system

Abstract

Disclosed is a method of detecting a behavioral or mental health condition. The method includes (a) receiving an input signal comprising a plurality of phonetic or lexical features of a speech of a subject. At least one of the plurality of phonetic or lexical features of the speech is associated with at least one language. The method also includes (b) selecting one or more acoustic models or natural language processing (NLP) models based at least in part on the plurality of phonetic or lexical features of the input signal. At least one of the acoustic models or NLP models is a multilingual model or a language independent model. The method also includes (c) detecting a result indicative of the presence or absence of the behavioral or mental health condition by processing the input signal with a joint or fusion model derived from the one or more acoustic models or natural language processing models.

Description

TECHNICAL FIELD

[0001] Cross-Reference This application claims the priority benefit of U.S. Provisional Patent Application No. 63 / 145,476 filed on February 3, 2021, which application is hereby fully incorporated herein by reference in its entirety.

BACKGROUND ART

[0002] Behavioral and mental health conditions are prevalent across the general population and may cause significant losses to society. While treatment is available for such conditions, many people may remain undiagnosed. Many screening systems for behavioral and mental health that use speech and / or natural language processing modeling have been devised for speakers of languages with large available data corpora, such as English. However, for languages that do not have large-scale data corpora, such analysis may be difficult or impossible to perform.

SUMMARY OF THE INVENTION

[0003] The present disclosure provides systems and methods for predicting the risk severity of a subject with respect to having behavioral and mental health conditions from utterances in multilingual scenarios (for example, by using a multilingual model, a language-agnostic model, that is, a language-independent model). The disclosed systems may enable mental health risk prediction for input utterances (in speech or text form) with content in unknown languages by using models that are trained in or implemented using known languages.

[0004] The disclosed system may perform analysis using acoustic models and / or natural language processing models. For example, an acoustic model may be used to analyze speech data from utterances, while an NLP model may be used to analyze text and / or speech data. The acoustic models used by this system may conform to or be based on one of many different architectures, and should not be interpreted as being limited to deep learning architectures that learn feature representations from data. The acoustic models used herein may also be based on or include architectures in which features have been expertly designed. The system may adaptively select from various acoustic and NLP models, or combinations thereof, to make predictions. The system may generate feedback based on the performance of the models, the availability of data, the topic, and the configuration used.

[0005] The system may first obtain a segment of speech data from a subject. The subject may be a human subject, which may be a patient. The system may then detect one or more languages ​​present in the speech. Based on the detected languages, the system may estimate model performance based on how well the model is likely to perform in those languages. Based at least partially on this estimate of model performance, the system may use a pre-model strategy selector to choose the combination of models to use. The system may then implement a joint model, a weighted combination of one or more models, an unweighted combination of one or more models, a rule-based model, or another type of model to predict the state. A post-model strategy selector may determine whether an alternative form of analysis is necessary based on confidence levels related to model performance.

[0006] The feedback mechanism may modify one or more components of the trial, or recommend one or more actions related to subjects, agents, or other users.

[0007] The system may generate an electronic report using the risk severity determined by the system for a behavioral or mental health condition. In some embodiments, the electronic report may include psychoeducational materials related to the behavioral or mental health condition if the determination indicates that the subject has or is likely to have such a condition.

[0008] In one embodiment, a method for detecting behavioral or mental health status is disclosed. This method comprises (a) receiving an input signal comprising multiple phonetic or lexical characteristics of a subject's utterance, at least one of which is associated with at least one language. The method also comprises (b) selecting one or more acoustic models or natural language processing (NLP) models based at least in part on the multiple phonetic or lexical characteristics of the input signal, at least one of which is a multilingual or language-independent model. The method also comprises (c) processing the input signal using a fused or joint model derived from one or more acoustic models or NLP models to detect results indicating the presence or absence of behavioral or mental health status.

[0009] In some embodiments, the input signal may include text or audio.

[0010] In some embodiments, at least one language is derived from a database of known languages.

[0011] In some embodiments, at least one language is one that has not previously been identified in the database of known languages.

[0012] In some embodiments, the method further comprises determining performance estimates of one or more acoustic or NLP models prior to (b). The performance estimates are associated with one or more of the speech or lexical characteristics of the input signal.

[0013] In some embodiments, performance estimates are determined using statistical modeling.

[0014] In some embodiments, processing an input signal using a fused model derived from one or more acoustic or NLP models includes (i) applying one or more weights to one or more acoustic or NLP models, and (ii) combining one or more weighted acoustic or NLP models.

[0015] In some embodiments, combining one or more weighted acoustic models or NLP models involves concatenating one or more outputs of the one or more weighted acoustic models or NLP models.

[0016] In some embodiments, the selection of one or more acoustic or NLP models is also based, at least in part, on prediction latency, the context of the input signal, or demographic information about the subject.

[0017] In some embodiments, one or more NLP models include one or more general-purpose language models, translation-based models, or multilingual models.

[0018] In some embodiments, (c) further includes fine-tuning at least one acoustic model or NLP model.

[0019] In some embodiments, behavioral or mental health conditions include depression, anxiety, stress, schizophrenia, bipolar disorder, or post-traumatic stress disorder (PTSD).

[0020] In some embodiments, one or more acoustic models include one or more acoustic multilingual models or acoustic language-independent models.

[0021] In some embodiments, one of the acoustic models is a supervised, semi-supervised, or unsupervised model.

[0022] In some embodiments, a joint or fusion model derived from at least one acoustic model or NLP model further comprises at least one demographic model or metadata-based model.

[0023] In some embodiments, the method further includes determining whether or not to provide feedback to the subject in (b).

[0024] In some embodiments, the feedback is to modify question prompts to the subject, modify a set of questions to be asked the subject, elicit further utterances from the subject, ask the subject relevant questions, prompt the subject to take one or more actions to increase the signal-to-noise ratio of the input signal, or modify the language of the interaction with the subject.

[0025] In one embodiment, a method for detecting behavioral or mental health is disclosed. The method comprises (a) receiving an input signal comprising multiple phonetic or lexical characteristics of a subject's utterance, at least one of which is associated with at least one language. The method also comprises (b) determining one or more confidence scales, one of which corresponds to the language of the input signal. The method also comprises (c) determining at least one confidence scale based on at least one of which is which, the confidence scale is related to the language performance of at least one of which is which Finally, the method comprises (e) detecting results indicating the presence or absence of a behavioral or mental health condition by processing an input signal using a fused or joint model derived from one or more acoustic or NLP models.

[0026] In one embodiment, a method for detecting behavioral or mental health is disclosed. The method comprises (a) receiving an input signal comprising multiple phonetic or lexical characteristics of a subject's utterance, at least one of which is associated with at least one language. The method also comprises (b) determining one or more confidence scales, one of which corresponds to the language of the input signal. The method also comprises (c) determining at least one confidence scale based on at least one of which is which, the confidence scale is related to the language performance of at least one of which is which The method also comprises (e) detecting results indicating the presence or absence of a behavioral or mental health condition by processing an input signal using a fused or joint model derived from one or more acoustic or NLP models. The method also comprises (f) iteratively selecting one or more improvement behaviors based on a confidence scale of the results and repeating steps (d) and (e) until a confidence threshold is achieved.

[0027] In one aspect, a method for detecting a behavioral or mental health condition is disclosed. The method comprises (a) receiving an input signal comprising a plurality of vocal characteristics or lexical characteristics of a subject's utterance. At least one of the plurality of vocal characteristics or lexical characteristics of the utterance is associated with at least one language. The method also comprises (b) determining one or more confidence measures. A certain confidence measure among the one or more confidence measures corresponds to the language of the input signal. The method also comprises (c) determining at least one reliability measure based on at least one of the one or more confidence measures. The reliability measure relates to the performance of a certain acoustic model or NLP model among a plurality of acoustic models or natural language processing (NLP) models for detecting a behavioral or mental health condition, with respect to the language of the at least one confidence measure. The method also comprises (d) selecting the one or more acoustic models or natural language processing (NLP) models based at least in part on the plurality of vocal characteristics or lexical characteristics of the input signal and the at least one reliability measure. At least one of the acoustic models or NLP models is a multilingual model or a language-independent model. The method also comprises (e) detecting a result indicating the presence or absence of a behavioral or mental health condition by processing the input signal using a fusion model or a joint model derived from the one or more acoustic models or NLP models. The method also comprises (f) iteratively selecting one or more improvement actions and repeating steps (d) and (e) until a reliability threshold is achieved, based at least in part on a reliability measure of the result. The method also comprises (g) determining one or more improvement actions directed at the subject or an agent based at least in part on the result, to improve the collection of the input signal.

[0028] In some embodiments, the confidence measure is generated using at least in part the metadata of the input signal.

[0029] In some embodiments, the method further comprises applying at least one label corresponding to at least one of one or more languages of an input signal.

[0030] In some embodiments, the at least one confidence measure is based at least in part on a length of a session in which the input signal is recorded, a topic of the input signal, or a quality of the input signal.

[0031] In some embodiments, the method further comprises: (i) determining whether one or more acoustic models or NLP models are unreliable; and (ii) if the one or more acoustic models or NLP models are unreliable, determining one or more improvement actions directed to a subject or an agent to improve collection of the input signal.

[0032] In some embodiments, a certain model among the one or more acoustic models or NLP models is a translation model.

[0033] In some embodiments, a certain acoustic model among the one or more of the acoustic models or NLP models is a multilingual acoustic model or a language-dependent acoustic model.

[0034] In some embodiments, the one or more acoustic models or NLP models include at least one metadata model or demographic model.

[0035] In some embodiments, the one or more acoustic models or NLP models comprise fusing outputs from at least two acoustic models or NLP models.

[0036] In some embodiments, results associated with the one or more improvement actions from (f) are provided in an electronic report.

[0037] In another aspect, the Disclosure provides a method for training a natural language processing model for detecting behavioral or mental health conditions, the natural language processing model comprising (i) a language model and (ii) a classifier, the method comprising (a) training the language model on a first encoded text, the first encoded text comprising text not relating to behavioral or mental health conditions; (b) fine-tuning the language model on a second encoded text and optionally on metadata information, the second encoded text comprising text relating to behavioral or mental health conditions; and (c) training a classifier to detect behavioral or mental conditions on a plurality of encoded speech samples from a plurality of subjects, the detection of a given encoded speech sample relating to a label indicating whether the subject who provided the encoded speech sample has a behavioral or mental health condition, and optionally on metadata information.

[0038] In another embodiment, a method is provided for detecting behavioral or mental health conditions. This method comprises (a) computer processing an input signal comprising a plurality of phonetic or lexical characteristics of a subject's utterance, wherein at least one of the plurality of phonetic or lexical characteristics of the utterance is associated with at least one language. The method also comprises (b) detecting results indicating the presence or absence of a behavioral or mental health condition using one or more acoustic models or models derived from natural language processing (NLP) models, wherein at least one of the acoustic models or NLP models is a multilingual model or a language-independent model.

[0039] Another aspect of this disclosure provides a non-transient, computer-readable medium comprising machine-executable code that, when executed by one or more computer processors, implements the systems described above or separately described herein.

[0040] Another aspect of this disclosure provides a system comprising one or more computer processors and computer memory coupled thereto. The computer memory comprises machine-executable code that, when executed by one or more computer processors, implements either of the methods described above or separately described herein.

[0041] In another aspect, the Disclosure provides a system comprising one or more computer processors and a memory comprising machine-executable instructions, which, when executed by one or more computer processors, enables one or more computer processors to realize an acoustic model configured to predict whether a subject has a behavioral or mental health condition of interest, at least in part, based on an input utterance comprising multiple segments from a subject. As will be understood, the Disclosure enables other and different embodiments, and some details of the Disclosure can be modified in various obvious ways without departing from the Disclosure. Accordingly, the drawings and specification are to be considered essentially illustrative and not restrictive. Incorporation by Reference

[0042] All published documents, patents, and patent applications referenced herein are incorporated by reference as if each individual published document, patent, or patent application were specifically and individually indicated. To the extent that the published documents and patents or patent applications incorporated by reference conflict with the disclosures contained herein, this specification is intended to supersede and / or take precedence over such conflicts.

[0043] The novel features of the present invention are specifically described using the appended claims. A deeper understanding of the features and advantages of the present invention will be obtained by referring to the embodiments for carrying out the invention illustrating exemplary embodiments utilizing the principles of the present invention, and to the appended drawings (hereinafter also referred to as "Figure" and "FIG"): [Brief explanation of the drawing]

[0044] [Figure 1] Figure 1 schematically illustrates a system, in accordance with the embodiment, configured to predict whether a subject has a behavioral or mental health condition in question, based at least partially on the subject's input utterances. [Figure 2] Figure 2 schematically illustrates a system configured to assess, screen, predict, or monitor a subject's behavioral or mental health using speech data, video data, and / or metadata relating to the embodiment. [Figure 3] Figure 3 illustrates a computer system programmed, or otherwise configured, to implement the method provided herein, in accordance with an embodiment. [Figure 4] Figure 4 schematically illustrates a system that uses a natural language processing ("NLP") model, in accordance with the embodiment, to predict whether or not a subject has a behavioral or mental health condition. [Figure 5] Figure 5 illustrates a process flow chart for performing multilingual mental health screening, in accordance with the embodiment. [Figure 6] Figure 6 illustrates a flowchart of the pre-model strategy selector, conforming to the embodiment. [Figure 7] Figure 7 shows an exemplary block diagram of an NLP model conforming to the embodiment. [Figure 8] Figure 8 illustrates an exemplary block diagram of an acoustic model conforming to an embodiment. [Figure 9] Figure 9 illustrates a block diagram of a post-model strategy selector in accordance with the embodiment. [Figure 10] Figure 10 illustrates a block diagram of a feedback mechanism according to the embodiment. [Figure 11] Figure 11 illustrates the label distribution for both the English and Spanish test sets from a multilingual mental health screening experiment, in accordance with the embodiment. [Modes for carrying out the invention]

[0045] While various embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided only as examples. Numerous modifications, alterations, and substitutions may be made by those skilled in the art without departing from the present invention. It is desirable to understand that various alternative forms to the embodiments of the present invention described herein may be adopted.

[0046] Wherever the terms “at least,” “greater than,” or “greater than or equal to” precede the first number or follow the last value in a set of two or more numbers, the terms “at least,” “greater than,” or “greater than or equal to” apply to each number in that set of numbers. For example, 1, 2, or 3 or more is equivalent to 1 or more, 2 or more, and 3 or more.

[0047] Wherever the terms “not greater than,” “less than,” or “less than or equal to” follow the last digit of a series of two or more numbers, the terms “not greater than,” “less than,” or “less than or equal to” apply to each digit in that series. For example, 3, 2, or 1 or less is equivalent to 3 or less, 2 or less, or 1 or less. overview

[0048] Disclosed is an end-to-end solution for predicting, detecting, or monitoring behavioral or mental health. Specifically, the system is configured to predict a speaker's behavioral or mental health regardless of the language present in their utterance. The speech data received from the speaker may include audio or text data.

[0049] The system may perform language-independent predictions by processing utterances using multiple acoustic models and natural language processing (NLP) models. These models may be general-purpose language models, multilingual models, or language-independent models.

[0050] The system may adapt to the input data and to new information generated during the machine learning process. For example, the system may first detect the language present in the utterance using one or more machine learning models. The pre-model strategy selector may then decide which machine learning algorithms to use to process the data, based at least on the input data and possibly on the detected language. The pre-model strategy selector may choose to apply a joint or fused model derived from one or more acoustic or NLP models to process the input utterance. After analysis by the machine learning models, the post-model strategy selector may decide to improve the prediction by taking corrective action or by using an alternative model, based on the confidence scores of one or more models used and / or the output of any of the models that may be used. Finally, the system may improve the prediction made by providing feedback to the subject or test agent, which may be done, for example, by changing the interaction with the subject in the screening process or by using knowledge gained about the training objective (e.g., the training agent).

[0051] The system is designed to make predictions based on lexical and non-lexical or acoustic features in the input utterance (which may be speech and / or text data), and can operate effectively with utterances from known or unknown languages, utterances containing one or more languages ​​(such as utterances with code-switching), or utterances containing specific dialects, pidgins, or creole languages.

[0052] Input utterances may include audio or text data uploaded to the user device (e.g., the subject's) as audio-based or text-based diary entries. The entries may or may not be multilingual in content. The disclosed system may perform an analysis of the diary entries to determine risk severity. Based on this risk severity, an application on the user device may suggest one or more recommended actions for the user. Recommended actions may be well-being or wellness exercises. The exercises may be related to cognitive behavioral therapy (CBT) or dialectical behavioral therapy (DBT). The application on the user device may provide medical care (e.g., conversation or text-based therapy through the application) or enable the user to contact telemedicine services.

[0053] One or more components of the disclosed system may be integrated into or accessible through other systems. For example, the analysis performed by the system may be accessible via an application programming interface (API) so that third-party developers can integrate the risk severity analysis into their applications. Such applications may include, for example, telemedicine, text-based therapy, mental or behavioral health monitoring (e.g., remote monitoring), or other types of applications. Subjects may be analyzed periodically, risk severity scores tracked, and patient progress over time recorded. Description of the Embodiment

[0054] Disclosed is a system that provides behavioral health predictions that may analyze speech input, which may be provided as audio or text data. Audio may be recorded in real time from human subjects. Audio may be provided via recordings. Text data may be entered in real time from human subjects. Text data may be provided as transcripts. Audio or text data may consist of utterances from one or more of many languages. Behavioral or mental health predictions may predict health conditions such as depression, anxiety, stress, schizophrenia, bipolar disorder, post-traumatic stress disorder (PTSD), or other disorders.

[0055] One or more languages ​​present in the speech data include English, Spanish, Chinese (e.g., Mandarin, Wu, Yue, Jing, Xiang, Gan, or Cantonese), Hindi, Bengali, Portuguese, Russian, Japanese, Punjabi, Marathi, Telugu, Turkish, Korean, French, German, Vietnamese, Tamil, Urdu, Javanese, Italian, Arabic, Gujarati, Persian, Bhutschuri, Hausa, Kannada, Indonesian, Polish, Yoruba, Malayalam, Odiya, Maithili, Burmese, Sundanese, Ukrainian, Igbo, Uzbek, Sindhi, Romanian, Tagalog, Dutch, Amharic, Pashto, Magahi, Thai, Saraiki, Khmer, Chhattisgari, Somali, Malay, Cebuano, and Nepali. This may include one or more of the following languages ​​or invented languages: Lu, Assamese, Sinhalese, Kurdish, Hurhurde, Bavarian, Azerbaijani, Greek, Chittagonian, Kazakh, Deccan, Hungarian, Albanian, Kinyarwanda, Danish, Finnish, Zulu, Lundi, Czech, Uyghur, Silhetti, Serbian, Croatian, Hungarian, Montenegrin, Haitian Creole, Swedish, Norwegian, Mon, Khosa, Belarusian, Konkani, Latin, Gaelic, Catalan, Occitan, Provençal, Maltese, Bosnian, Armenian, Slovenian, Hebrew, Yiddish, Aramaic, Ladino, Navajo, Native Hawaiian, Maori, Esperanto, Pig Latin, or other languages. The languages ​​included in the speech data may be Creole languages ​​(such as Haitian Creole) or pidgin languages. The languages ​​included in the speech data may be regional varieties or dialects (for example, Louisiana French, Indian English).

[0056] The language may be one that the system has encountered before. Alternatively, the language may be one that the system has never encountered before.

[0057] Speech samples (audio or text) may have any word length. For example, a text sample may be less than 10 words, less than 20 words, less than 30 words, less than 50 words, less than 100 words, less than 500 words, less than 1000 words, or less than 2000 words. For example, a text sample may be longer than 10 words, longer than 20 words, longer than 30 words, longer than 50 words, longer than 100 words, longer than 500 words, longer than 1000 words, or longer than 2000 words.

[0058] The speech sample may be of any length. For example, the speech sample may be less than 10 minutes, less than 20 minutes, less than 30 minutes, less than 50 minutes, less than 100 minutes, less than 500 minutes, less than 1000 minutes, or less than 2000 minutes. For example, the speech sample may be longer than 10 minutes, longer than 20 minutes, longer than 30 minutes, longer than 50 minutes, longer than 100 minutes, longer than 500 minutes, longer than 1000 minutes, longer than 2000 minutes.

[0059] Input speech samples may be active or passive speech samples. Active speech samples may be samples of speech taken from a subject in a clinical setting. For example, speech may be recorded while a subject is undergoing an investigation by a human or automated agent. In various embodiments, active speech samples are recorded when the subject knows that their elicited speech is being recorded for the purpose of mental health screening. Passive speech may be speech in conversation or observed speech when the subject is not being provided with a questionnaire or survey. For example, passive speech may be recorded from a conversation between a subject and a healthcare provider, or even with someone not participating in mental health screening.

[0060] Once utterances are collected, the system may identify one or more languages ​​present in the utterance sample. The system may include a software system or subsystem that can act as a language detection unit (or “language detector”) that detects one or more languages ​​present in the input utterance sample. A machine learning algorithm may generate one or more probability scores indicating the likelihood that the sample contains multiple languages, with each language corresponding to a probability score. For example, a machine learning algorithm may output a set of probabilities indicating that the sample has a 90% chance of containing English, a 70% chance of containing German, and a 3% chance of containing Arabic. The language detection unit may label the input utterances corresponding to the detected languages. The language detection unit may detect multiple languages ​​present in the utterance sample (i.e., code-switching between languages). The language detection unit may detect specific dialects, pidgins, or creoles of one or more languages ​​present.

[0061] The language detection unit may use metadata or demographic data to detect the languages ​​present. For example, the language detection unit may use as input other personal or personally identifiable information, or metadata that may contain non-personally identifiable information, along with speech or text data of utterances that include information about the subject's age, sex, race, nationality or country of origin, weight, medical history, address, occupation, marital status, socioeconomic status, education level, and number of children. This information may include typing speed, ambient noise, test circumstances or environment, time delay in answering questions, voice pitch, type of test being conducted, type of equipment used to acquire samples, duration of utterances, voice quality, subject equipment, agent equipment, or other information. Metadata may include vital sign information, such as heart rate, blood glucose level, blood pressure, or other physical measurements. The language detection unit may include an annotation unit that labels the utterance samples.

[0062] The system may use a performance estimation unit to estimate how well one or more acoustic or NLP models might perform to predict mental or behavioral health, at least in part, based on the quality of the input utterances. For example, the performance of a model may vary based on the quality of one or more languages, text, or acoustic signals present in the model (e.g., if the signals are incomplete or noisy), the length of the signals, the type of signals (acoustic, text, or both), and the acoustic features of the speaker's voice. The performance estimation module may also consider, when predicting the performance of a particular model, the language identity distribution, the length of the speech session or signal, the topic within the speech session, the quality of the utterances, the intonation or pitch of the subject's utterances, which dialect of a particular language the utterances belong to, the rate of speech, or the number of pauses in other vocal tics.

[0063] For example, a particular speech model may perform well with German speech but not with English speech. A specific NLP model may perform well analyzing speech with a Cockney accent but not with a Scottish accent. Another type of model may not perform well with code-switched samples (e.g., an Indian speaker switching between Hindi and English).

[0064] The performance estimation unit (used interchangeably with the "model performance estimator") may use statistical modeling (e.g., Bayesian modeling) or machine learning to estimate the performance of one or more acoustic or NLP models.

[0065] The pre-model strategy selector determines (conversely referred to as “strategy determination or selection”) different combinations of acoustic models and / or NLP models to be used to predict, monitor, or detect behavioral or mental health states from input utterances (e.g., using weighted combinations, unweighted combinations, joint models, or model fusion). The pre-model strategy selector may determine weighted combinations using input utterance samples. In some embodiments, the pre-model strategy selector may use additional data. For example, the pre-model strategy selector may use language detection data or model performance data to determine whether to use weighted combinations of models, joint models, or models with different configurations. In some embodiments, the pre-model strategy selector may use other data, such as demographic data or metadata. In some embodiments, along with acoustic and NLP models, weighted combinations of models may include demographic models and / or metadata models. In some embodiments, the strategy selector may implement a fusion model, or a model that combines representations (e.g., latent representations) of outputs from one or more acoustic models, NLP models, or other models. The pre-model strategy selector may determine the optimal combination of models that best perform a prediction, monitoring, or detection task. This optimal combination may include maximizing accuracy, recall, area under the receiver operating characteristic (ROC) curve (AUC), true positive rate, false positive rate, or other statistics related to quantifying the performance of a machine learning algorithm or combination of machine learning algorithms. The pre-model strategy selector may also determine whether to incorporate feedback into the mental or behavioral health screening process.

[0066] The pre-model strategy selector may devise the configuration of acoustic models, NLP models, and / or other models in one or more ways. The strategy selector may determine the order in which one or more acoustic models, NLP models, or other models are implemented. The strategy selector may determine the magnitude of the weights to apply to each selected model. The strategy selector may determine how the representations of the data are fused. For example, the representations may be fused using concatenation or max pooling. In addition, a multimodal algorithm may be used.

[0067] Machine learning models may process input data in accordance with a strategy selected by a pre-model strategy selector. These models may be supervised or unsupervised learning models. The models may be deep learning models. In some embodiments, the model includes preprocessing algorithms that filter or package the input utterances into data more suitable for machine learning analysis.

[0068] The model may be an NLP model. The model may include a general-purpose language model. The model may include an NLP translation model. The model may include a multilingual NLP model. The model used by the system may have the same architecture as other models used, or it may have a different architecture. The models used may or may not be limited to those listed. The model may include transformers, such as multilingual machine translation (M2M100), multilingual bidirectional and auto-regressive transformer (mBART), multilingual text-to-text (mT5), cross-lingual robust bidirectional encoder representations from transformer (XLM-RoBERTa)), Longformer, or a lite bidirectional encoder representations from transformer (ALBERT)).

[0069] The model may be an acoustic model. The model may include an acoustic model trained on a single language. The acoustic model may be fine-tuned on a single language. The model may include a multilingual acoustic model (e.g., trained on acoustic data from different languages). The model may include a language-independent acoustic model in which language-independent features are selected or learned. For example, these features may include (but are not limited to) speaker-based embedding features such as pitch, formant frequency, speech rate, i-vectors or x-vectors, and features learned from or from non-automatic speech recognition (ASR) tasks (e.g., sound classification).

[0070] The model may be a demographic model or a metadata model.

[0071] The model may be a fusion model. A fusion model may combine output data (e.g., representations) from one or more of the following: acoustic models, NLP models, demographic models, or metadata models. The output may be a score or a learned representation.

[0072] The system may process input using a fused model derived from multiple models (or submodels) used (e.g., acoustic models, NLP models, fused models, or demographic models). In such scenarios, each selected model may be trained independently. The model outputs may then be combined. The model outputs may be combined using methods such as concatenation, max pooling, or other methods.

[0073] The system may process input using a joint model derived from multiple models. In such scenarios, models (or sub-models), such as an acoustic model, an NLP model, or another model, are trained together or simultaneously to produce a joint model output.

[0074] The system may use a post-model strategy selector to compensate for low confidence resulting from the predicted output obtained from a combination of machine learning models. Confidence may be generated or computed by one or more of the models themselves and provided to the post-model strategy selector. The post-model strategy selector may be used as predictions are being made, and model parameters may be updated until the system's final prediction is valid. If confidence is low, the post-model strategy selector may take one or more actions. The post-model strategy selector may prompt a language change if an agent is managing the mental health screen. The post-model strategy selector may prompt human intervention in the screening process (e.g., from a human agent such as a clinician or healthcare provider). The post-model strategy selector may use an alternative model. The post-model strategy selector may change the prediction task (e.g., from depression to anxiety). Depending on model confidence and / or other factors, the post-model strategy selector may suggest one or more corrective actions to modify the analysis or testing procedure.

[0075] A feedback mechanism may provide feedback to an agent (human or automated) that provides a mental health screening to a subject following completion of the mental health screening or selection of a pre-model strategy by a pre-model strategy selector. An automated agent may be an application (e.g., a smartphone application). The feedback mechanism may take the results of a model or post-model strategy selector as input and output a recommended action. The feedback mechanism may be a rule-based mechanism or may use statistical methods or machine learning analysis. The feedback mechanism may, for example, prompt a language switch if the test is being conducted. The feedback mechanism may guide the patient to provide further verbal input. The feedback mechanism may prompt the agent to ask relevant questions. The feedback mechanism may take one or more actions to minimize the signal-to-noise ratio of the input sample. For example, the feedback mechanism may ask the patient to move to a quiet room or reconfigure the test equipment. The feedback mechanism may determine that an incorrect question has been asked and prompt the agent to change the question. Acoustic model

[0076] Figure 1 schematically illustrates System 100 in accordance with an embodiment. In other embodiments, acoustic models of different architectures may be used. System 100 can acquire input utterances through a microphone or array of microphones located on or connected to an Internet-connected device (e.g., via a Bluetooth connection). The device may be a wearable device (e.g., a smartwatch), a mobile phone, a tablet, a laptop computer, a desktop computer, a smart speaker, a home assistance device (e.g., an Amazon Alexa® device or a Google Home® device), or similar. The device may have a mental health application. The mental health application may visually or audibly prompt the subject to answer questions about the subject's work and home life, sleep, mood, medical history, and similar matters. The subject's responses to such prompts may be used as input utterances. System 100 may be implemented on a mobile application and may process input utterances locally on the subject's mobile device. Alternatively, or in addition, the mobile device can transmit the utterance to a remote location for processing. In some cases, processing may be performed partially on the local device and partially on a remote server.

[0077] Alternatively, or in addition, input utterances may be acquired through clinical encounters with healthcare professionals. For example, a voice recording device may capture a subject's utterances during a physician's consultation. The physician's consultation may be an in-person consultation or a telemedicine consultation conducted remotely. Other situations may include consultations with healthcare providers, medical teams, or medical managers by phone, online, in the office, or at home.

[0078] The subsystem in Figure 1 may be implemented in one or more computing devices. These computing devices may be servers, desktop or laptop computers, electronic tablets, mobile devices, smart speakers, smartwatches, wearable devices, or similar devices. The computing devices may be located in one or more locations. They may include general-purpose processors, graphics processing units (GPUs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), machine learning accelerators, or similar devices. The computing devices may also include memory, such as dynamic or static random-access memory, read-only memory, flash memory, hard drives, or similar devices. The memory may be configured to store instructions at runtime that train the computing device to train system 100, or to cause the computing device to predict whether a subject has a target behavior or mental health condition. The computing devices may also include network communication devices. These network communication devices can enable the computing devices to communicate with each other and with any number of user devices over a network. For example, a network communication device may enable the computing device implementing System 100 to communicate with a healthcare professional's mobile device regarding the subject's predicted behavioral or mental health status. The network may be a wired or wireless network. For example, the network may be a fiber optic network, an Ethernet® network, a satellite network, a cellular network, a Wi-Fi® network, a Bluetooth® network, or similar. In other implementations, the computing device may be several distributed computing devices accessible via the Internet. Such computing devices may be considered cloud computing devices. NLP Model Parameters

[0079] Figure 4 schematically illustrates a system 400 that uses an NLP model to predict whether a subject is in a behavioral or mental health state. System 400 may include an automatic speech recognition ("ASR") subsystem 405, an encoder subsystem 410, a language model subsystem 415, and a classification subsystem 425. The ASR subsystem 405 can generate a copy of the input utterance from the subject. In some cases, the ASR subsystem 405 may include a third-party ASR model, such as Google ASR. The third-party ASR may be a one-set hypothetical ASR, or it may take into account word uncertainty, or it may include word confusion information. In other cases, the ASR subsystem 405 may include a custom ASR model.

[0080] The subsystems and their components shown in Figure 4 may be implemented on one or more computing devices. These computing devices may be servers, desktop or laptop computers, electronic tablets, mobile devices, or similar. The computing devices may be located in one or more locations. They may include general-purpose processors, graphics processing units (GPUs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or similar. In addition, the computing devices may include memory, such as dynamic or static random-access memory, read-only memory, flash memory, hard drives, or similar. The memory may be configured to store instructions that enable the computing devices to implement the subsystem's functions at runtime. The computing devices may also include network communication devices. These network communication devices may enable the computing devices to communicate with each other and with any number of user devices over a network. The network may be a wired or wireless network. For example, the network may be a fiber optic network, an Ethernet® network, a satellite network, a cellular network, a Wi-Fi® network, a Bluetooth® network, or similar. In other implementations, the computing devices may be several distributed computing devices accessible via the Internet. Such computing devices may be considered cloud computing devices.

[0081] System Output The system 200 in Figure 2 can output an electronic report identifying whether a patient is at risk of a mental or behavioral condition. The electronic report can be configured to be displayed on the graphical user interface of an electronic device. The electronic device may belong to the patient, subject, healthcare provider, payer, healthcare team, family member of the subject or patient, or other stakeholders. The electronic report may include a quantification of the risk of a mental or physiological condition, such as a normalized score. The score can be normalized with respect to the entire population or a subgroup. The electronic report may also include a confidence level for the normalized score. The confidence level can indicate the reliability of the normalized score (i.e., the degree to which the normalized score is reliable).

[0082] Electronic reports can include visual graph elements. For example, if a patient has multiple scores obtained from multiple screening or monitoring sessions that occurred at different times, the visual graph element might be a graph showing the patient's scores progressing over time.

[0083] System 200 can output electronic reports to patients or patient-related contacts, healthcare providers, payers, or other third parties. System 200 can output electronic reports in substantially real time, even while screening, monitoring, or diagnosis is in progress. Electronic reports can be updated in substantially real time in response to changes in normalized scores or confidence levels during the screening, monitoring, or diagnosis process, and can be resent to the user.

[0084] In some cases, electronic reports may include one or more descriptors regarding the patient's mental state. The descriptors may be qualitative measures of the patient's mental state (e.g., "mild depression"). Alternatively, or further, the descriptors may be topics mentioned by the patient during screening. The descriptors may be displayed graphically, for example, as a word cloud.

[0085] The models described herein can be optimized for a specific purpose or based on the entity receiving the system's output. For example, a model may be optimized for its sensitivity in estimating whether a patient has a certain mental condition. Healthcare payers, such as insurance companies, may prefer such a model to minimize the amount of insurance payouts for patients who receive false positive diagnoses. In other cases, a model may be optimized for its specificity in estimating whether a patient has a certain mental condition. Healthcare providers may prefer such a model. The system may select an appropriate model based on the stakeholders to whom the output is to be sent. After processing, the system may send the output to the stakeholders.

[0086] The models described herein may instead be tuned or configured to process speech and other data according to a desired level of sensitivity or specificity determined by a clinician, healthcare provider, insurance company, or government regulatory body. Alternatively, or in addition, the models may be tuned, configured, or trained to optimize precision, recall, F1, equal error rate ("EER"), positive predictive value ("PPV"), negative predictive value ("NPV"), positive likelihood ratio ("LR+"), negative likelihood ratio ("LR-"), coincident correlation coefficient ("CCC"), Pearson correlation coefficient ("PCC"), mean squared error ("RMSE"), mean absolute error ("MAE"), or other relevant performance metrics.

[0087] Electronic reports may include "word clouds" or "topic clouds" extracted from textual transcripts of patient utterances. Word clouds may be visual representations of individual words or phrases, with the most frequently used words and phrases indicated using larger font sizes, different colors, different fonts, different typefaces, or any combination thereof. Since depressed patients generally utter certain words and phrases more frequently than non-depressed patients, expressing word and phrase frequency in this way can be useful. For example, depressed patients may use words or phrases that convey dark, black, or morbid humor. They may speak of feelings of worthlessness or failure, or use absolutist phrases such as "always," "never," or "completely." Depressed patients may also use first-person singular pronouns (e.g., "I," "me") more frequently and second- or third-person pronouns less frequently when compared to the general population. The system may be able to train a machine learning algorithm to perform semantic analysis of word clouds of depressed and non-depressed individuals, with the aim of being able to classify people as depressed or not depressed based on those word clouds. Word cloud analysis may also be performed using unsupervised learning. For example, the system may analyze unlabeled word clouds and search for patterns to divide people into groups based on their mental state. The generated words may show a decrease or increase in the risk of depression (i.e., this is associated with an increase or decrease in the risk of depression).

[0088] Similarly, electronic reports may include predicted personality traits of the patient. Personality traits (e.g., introversion or extroversion) can be inferred from the length of speech.

[0089] The electronic report may also include evidence-based psychoeducational materials and support strategies. These materials and strategies may be individually optimized for the patient's score. These materials and strategies may be provided directly to the patient in the form of videos, texts, or homework, or they may be provided to the patient's healthcare provider, who may lead the psychoeducational process.

[0090] Usage example The disclosed health screening systems described herein may be used to support the treatment of patients in regions of the world where mental health infrastructure is limited. These systems may provide a central infrastructure for managing health screening of patients or subjects from many countries. In addition, migration due to natural disasters, climate change, terrorism, and other factors can cause mass migration and movement of people to indigenous-speaking regions with different languages. Because the models used by the systems vary—ranging from language-agnostic, language-independent, or multilingual—the disclosed systems may be able to support patients in diverse regions, communities, and situations that have previously been underserved.

[0091] In addition, the system may be used to test the mental fitness of employees who have mentally demanding or stressful jobs. Clinicians or algorithms may be used to develop screening or monitoring methods for these conditions. For example, the system may be evaluated based on queries similar to those tested in the Minnesota Multiphasic Personality Inventory (NMPI) and MMPI-2.

[0092] This system may also be used to screen military personnel for mental fitness. For example, the system may implement an assessment using queries with subjects similar to those asked about Primary Care Post-Traumatic Stress Disorder for Diagnostic and Statistical Manual of Mental Disorders (PC-PTSD-5)) for testing for PTSD. In addition to PTSD, the system may screen military personnel for depression, panic disorder, phobias, anxiety, and hostility. The system may employ different surveys to screen military personnel before and after deployment. The system may categorize military personnel by occupation, and may also categorize them by branch of service, officer or non-commissioned officer status, sex, age, ethnicity, number of deployments / expeditions, marital status, medical history, and other factors.

[0093] The system may be used to evaluate prospective gun buyers, for example, by conducting background checks. The evaluation may be designed by a clinician or by an algorithm to assess the prospective buyer's mental suitability for possessing a firearm. This investigation may require using questions and related questions to determine whether the prospective gun buyer would be deemed by a court or other body to be a danger to themselves or others. Scoring

[0094] The models described herein may generate scores at various stages of assessing mental or behavioral health. The generated scores may be scaled or binary. Scaled scores may span a range of multiple values, while binary scores may represent one of two discrete values. The models may swap binary and scaled scores at various stages of assessment to monitor different mental states, or update specific binary and scaled scores for specific mental states during the assessment.

[0095] The scores generated by the system can be either binary or scaled scores, and may be generated after each response to each query in the assessment, or formulated based in part on previous queries. In the latter case, each marginal score acts not only to fine-tune the prediction of depression or another mental state, but also to make the prediction more robust. Marginal predictions may thus increase the confidence measure of the prediction of mental state after a certain number of queries and responses (correlated with a particular intermediate mental state).

[0096] For scaled scores, refining the scores may allow clinicians to determine with greater accuracy the severity of one or more mental states a patient is experiencing. For example, when multiple intermediate depressive states are observed, refining the scaled score may allow clinicians to determine whether the patient has mild, moderate, or severe depression. Furthermore, multiple iterations of scoring can add redundancy and robustness, assisting clinicians and administrators in eliminating false negatives. For example, initial predictions of mental state may be noisy because there are relatively few speech segments available for analysis, and the NLP algorithm may not have enough information to determine the semantic context of the patient's recorded speech. Even if a single peripheral prediction itself is a noisy estimate, refining the prediction by adding more measurements may reduce the overall variance in the system and result in a more accurate prediction. The predictions described herein may be more effective than predictions obtained simply by conducting surveys, because people may have an incentive to lie about their condition. Conducting such surveys can result in numerous false positives and false negatives, potentially causing patients who need treatment to slip through the cracks. Additionally, while trained clinicians may recognize voice-based and face-based biomarkers, they may be unable to analyze the large amounts of data available to the models disclosed herein.

[0097] Scaled scores may be used to describe the severity of a mental state. A scaled score may be, for example, a number between 1 and 5, or a number between 0 and 100, with higher numbers indicating a more severe or acute form of the mental state experienced by the patient. Scaled scores may include integers, percentages, or decimals. Conditions whose severity can be represented by scaled scores include, but are not limited to, depression, anxiety, stress, PTSD, phobic disorders, schizophrenia, and panic disorder. For example, a score of 0 for the depression-related aspect of the assessment may indicate no depression, a score of 50 may indicate moderate depression, and a score of 100 may indicate severe depression. A scaled score may be a mixture of multiple scores. A mental state may be represented as a mixture of sub-mental states, and a patient's composite mental state may be a weighted average of the individual scores from these sub-mental states. For example, a composite score for depression may be a weighted average of individual scores for anger, sadness, self-image, self-worth, stress, loneliness, isolation, and anxiety.

[0098] Scaled scores may be generated using models that employ multi-label classifiers. These classifiers may be, for example, decision tree classifiers, k-nearest neighbor classifiers, or neural network-based classifiers. The classifier may generate multiple labels for a particular patient at intermediate or final stages of the assessment, with the labels representing the severity or degree of a particular mental state. For example, a multi-label classifier may output multiple numbers, which may be normalized into probabilities using a softmax layer. The label with the highest probability may indicate the severity of the mental state experienced by the patient.

[0099] Scaled scores may be determined using regression models. Regression models may determine fitted values ​​from training examples, represented as a sum of weighted variables. These fitted values ​​may be used to estimate scores from patients using known weights. The weights may be based partly on features derived from audiovisual signals (e.g., speech-based biomarkers) and partly on patient information, such as features derived from patient demographics. Weights used to predict final or intermediate scores may be taken from previous intermediate scores.

[0100] Scaled scores may be scaled based on confidence scales. Confidence scales may be determined based on recording quality, the type of model used to analyze patient speech from the recordings (e.g., voice, visual, semantic), a temporal analysis related to which model was most frequently used during a particular period, and the timing of specific voice-based biomarkers within audiovisual samples. Multiple confidence scales may be used to determine intermediate scores. Confidence scales being assessed may be averaged to determine the weighting of a particular scaled score.

[0101] A binary score may reflect a binary result from the system. For example, the system may classify a user as either depressed or not depressed. The system may do this using a classification algorithm such as a neural network or an ensemble method. A binary classifier may output a number between 0 and 1. If a patient's score is above a threshold (e.g., 0.5), the patient may be classified as "depressed." If a patient's score is below the threshold, the patient may be classified as "not depressed." The system may generate multiple binary scores for multiple intermediate states of the assessment. The system may weight and sum the binary scores from the intermediate states of the assessment to generate an overall binary score for the assessment.

[0102] The outputs of the models described herein can be converted to calibrated scores, for example, scores with a range of 1. In addition, or instead, the outputs of the models described herein can be converted to clinically useful scores. Clinically useful scores may be qualitative diagnoses (e.g., high risk of severe depression). Clinically useful scores may instead be normalized qualitative scores normalized with respect to a general population or a specific subpopulation of patients. Normalized qualitative scores may express a percentage of risk relative to the general population or subpopulation.

[0103] The systems described herein may be able to identify a subject's mental state (e.g., mental or behavioral disorder) with less error (e.g., 10% less) or higher accuracy (e.g., 10% higher) than standardized mental health questionnaires or test tools. The error rate or accuracy may be established against benchmark criteria available to entities for identifying or assessing one or more medical conditions, including mental states. These entities may be clinicians, healthcare providers, insurance companies, or government regulatory bodies. The benchmark criteria may be independently validated clinical diagnoses.

[0104] Model-specific confidence scale The models described herein may use confidence scales. A confidence scale may be a measure of how well a score generated by a machine learning algorithm is at predicting a mental state, such as depression. The confidence scale may depend on the conditions under which the score was acquired. The confidence scale may be expressed as an integer, decimal, or percentage. Conditions may include the type of recording device, the surrounding space in which the signal is acquired, background noise, the specificity of the patient's utterances, the fluency of the speaker's language, the length of the patient's responses, the assessed truthfulness of the patient's responses, and the frequency of nonsensical words and phrases. Under conditions where the quality of the signal or utterances makes the analysis of the utterances more difficult, the confidence scale may have a smaller value. In some embodiments, the confidence scale may be added to the score calculation by weighting the calculated binary or scaled score by the confidence scale. In other embodiments, the confidence scale may be provided separately. For example, the system may tell a clinician that the patient has a depression score of 0.93 with a 75% confidence level.

[0105] The confidence level may also depend on the quality of the labels in the training data used to train a model that analyzes patient utterances. For example, if the labels are based on patient-completed surveys or questionnaires rather than on official clinical diagnoses, the label quality may be judged as low, and therefore the confidence level of the score may be low. In some cases, surveys or questionnaires may be judged to have a certain degree of dishonesty. In such cases, the label quality may be judged as low, and therefore the confidence level of the score may be low.

[0106] In particular, if the confidence scale is affected by the environment in which the evaluation is performed, the system may take various measures to improve the confidence scale. For example, the system may employ one or more signal processing algorithms to filter out background noise, or it may use impulse response measurements to determine how to remove the effects of reverberation caused by the objects and features of the environment in which the speech sample was recorded. The system may also use semantic analysis to find contextual clues and determine the features of missing or unintelligible words.

[0107] In addition, the system may use user profiles to group people based on behavior, ethnic background, gender, age, or other categories. Since people from similar groups may have similar voice-based biomarkers, the system may be able to predict depression with greater confidence, because people exhibiting similar voice-based biomarkers are likely to exhibit depression in a similar manner.

[0108] For example, people with depression stemming from different backgrounds may be categorized in various ways by slower speech, monotonous or low pitch fluctuations, excessive pauses, vocal timbre (raspy or hoarse voice), incoherent speech, lack of focus or loss of focus, concise answers, and stream-of-consciousness narrative. These voice-based biomarkers may belong to one or more segments of the patient being analyzed.

[0109] Multilingual mental health screening Figure 5 illustrates a process flow chart 500 for conducting a multilingual mental health screening according to an embodiment. The patient is tested by an agent or test provider, which may be a human test provider (e.g., a physician or clinician) or an automated test interface. The automated test interface may conduct a survey using questions designed to elicit utterances in voice or text form as input 501. The multilingual health screening process performs multiple operations on the input utterances 501 to monitor, detect, or predict mental or behavioral health status. The process is adaptive, and measurements of the predictive process may be taken at multiple stages to fine-tune the predictions.

[0110] Input 501 may be voice input or text input. Voice input may include one or more utterances, snippets of conversation, or elicitations. Elicitations may be several seconds to several minutes in length. Input may be text elicited and recorded from the subject. Input may also be a text transcript of speech produced by the subject. Text input may be of any length (e.g., from a few characters to several pages). Voice input may be of any duration (e.g., from a few seconds to several hours).

[0111] The language detector 502 may determine one or more languages ​​present in the input utterance 501. The language detector 502 may generate a probability distribution indicating which languages ​​are most likely to be present in the sample. In the case of a single-language utterance, this distribution may indicate candidate languages ​​to which the utterance may belong. In the case of a multilingual sample (e.g., a sample containing code-switching), the language detector 502 may indicate which languages ​​are present. If the language is unknown to the system, language selection analysis may select the most similar known language from the languages ​​that the system has learned to recognize through training.

[0112] The model performance estimator 503 may determine performance estimates for different models 505 for a given portion of utterances in a given language. These estimates may be used by the pre-model strategy selector 504, which selects a subset of models for calculating scores related to mental health risk. In some embodiments, the pre-model strategy selector 504 may not process an utterance if it determines that there is no model capable of effectively processing the provided utterance input. In such cases, the system may implement a feedback mechanism 509 at the point in which it makes such a determination. In the case of multilingual utterances, the model performance estimator 503 may generate multiple confidence values ​​for the multiple languages ​​present.

[0113] In some embodiments, the language detector 502 and the model performance estimator 503 are absent. In these embodiments, the pre-model strategy selector 504 may rely more heavily on feedback from the input 502 and the feedback mechanism 509 before devising a strategy.

[0114] The pre-model strategy selector 504 may select a subset of models 505 (or model configurations) to be used to make mental health predictions (for example, using joint modeling or model fusion). The pre-model strategy selector may use input data 501 along with feedback from the feedback mechanism 509, as well as linguistic information from the language detector 502 and the model performance estimator 503, to determine a subset of machine learning models 505 that will generate mental health predictions.

[0115] The pre-model strategy selector 504 may select a model 505 to use based on resource availability. For example, the pre-model strategy selector 504 may decide to use a translation model if automatic speech recognition (ASR) is available during mental health screening.

[0116] The pre-model strategy selector 504 may train a new model or adapt it by collecting labels or population statistics. The pre-model strategy selector 504 can train a model using population statistics or direct labels.

[0117] The pre-model strategy selector 504 may drive a model selection strategy, which may be done using one or more of the following: latency of predictions (e.g., single model vs. multiple models, automatic speech recognition, language detection, or depression detection); the circumstances under which the input 501 was collected (human-device, human-machine, etc.); business flow requirements; the monetary cost of using a particular model; or demographic data that may determine the data flow path. The pre-model strategy selector may determine whether the acoustic or NLP model configuration is reliable, and if so, may use a feedback mechanism 509 to provide one or more corrective actions to improve mental health screening.

[0118] Model 505 may be an NLP model 505A, an acoustic model 505B, another model (e.g., demographic, metadata) 505C, or a fusion model 505D. The model may use machine learning algorithms to generate intermediate outputs. These intermediate outputs may be weighted and combined according to instructions from the pre-model strategy selector 504.

[0119] NLP Model 505A may be translation-based models. These models may be sentence-to-spanish (e.g., English-Spanish, Spanish-English, English-Portuguese). These models may require less data for training than general-purpose models. These models may include many-to-many architectures that translate between 100 different language (M2M100) architectures.

[0120] NLP model 505A may be a multilingual model. These models may be trained simultaneously on large amounts of data across many languages. Multilingual models may be based on the architecture of a multilingual text-to-text transfer transformer (mT5) model.

[0121] Either the NLP model 505A or the acoustic model 505B may be fine-tuned for a specific task, for example, to detect a particular health condition. Fine-tuning may involve training a general-purpose model using data from a specific domain, and then performing supervised training on classification or regression.

[0122] The post-model strategy selector 506 may determine whether model 505 produces unreliable results. Based on information from input 501, pre-model strategy selector 504, or model 505, the post-model strategy selector 506 may select one or more corrective actions to improve predictions. These actions may be performed iteratively until the predictive results produced by model 505 are sufficiently reliable or valid. Corrective actions performed by the post-model strategy selector 506 may include switching language, soliciting human intervention, using prior information (e.g., using a distribution of PHQ scores across a specific demographic to provide screening results instead of using the output from model 505), using one or more alternative models 505, or predicting different metrics. The post-model strategy selector 506 may iteratively induce corrective actions until a confidence threshold is met or exceeded.

[0123] The results generator 507 may generate one or more reports 508 from the output of model 505. The results 508 may be an electronic report containing one or more charts or graphs describing the output of model 505. The results may be reviewed by an agent or subject. The results 508 of model 505 may be used to calculate additional scores or metrics used by a clinician. The results provided by the results generator 507 may be associated with improvement behaviors provided by the post-model strategy selector 506.

[0124] The feedback mechanism 509 may use information from the results 508 and / or the pre-model strategy selector 504 to recommend changes to the mental health screening procedure.

[0125] Figure 6 illustrates a flowchart of the pre-model strategy selector 504. The pre-model strategy selector 504 may take language detection results as input, which may include whether or not the input is the native language. Input to the pre-model strategy selector 504 may include the number of responses, the length of the responses, user metadata (e.g., demographics), and information about closed-loop sessions between patients and healthcare providers or automated test modules.

[0126] Figure 7 illustrates an exemplary block diagram of NLP model 505A in accordance with the embodiment. Model 505A may be trained 701. Then, a general-purpose NLP model 702, a multilingual NLP model 703, and a translation-based NLP model 704 may operate on input 501 to generate an intermediate output, which may be refined by a fine-tuned NLP model 705.

[0127] Figure 8 shows an exemplary block diagram of acoustic model 505B in accordance with the embodiment. Model 505A may be trained 801. Then, language-independent acoustic model 802 and multilingual acoustic model 803 may operate on input 501 to produce an intermediate output, which may be fine-tuned by a fine-tuned acoustic model 805.

[0128] Figure 9 illustrates a block diagram of the post-model strategy selector 506 in accordance with the embodiment. The post-model strategy selector 506 may take model confidence, number of responses, response length, user metadata (e.g., demographic, personal or non-personal information), voice / transcription quality, and topic score as input. The post-model strategy selector 506 may provide the results as input for future test behavior.

[0129] Figure 10 illustrates a block diagram of the feedback mechanism 509 in accordance with the embodiment. The feedback may be based on strategy selection 504, model 505, or other final report, or a combination of preceding components. The feedback mechanism 509 may prompt the test model to switch languages ​​when the screening is performed, or to ask one or more questions from the PHQ / GAD questionnaire. The feedback mechanism 509 may prompt the patient to provide further elicitations of speech. The feedback mechanism 509 may prompt the system to ask relevant questions. The feedback mechanism 509 may prompt the patient to perform one or more actions that increase the SNR of the speech signal, including moving to a quieter room to perform the screening. The feedback mechanism 509 may prompt the test agent to use a different set of questions.

[0130] quality control Situations may arise where the input utterances provided by patients are unacceptable. In such cases, the system described herein can flag the input utterances in real time. For example, a cooperative user may be unable to produce an utterance, or may produce one of suboptimal quality or quantity. An acoustic quality detector can analyze the utterance as it is collected and generate a real-time warning if the quality of the utterance (e.g., its volume) is too low. The system can also determine the word count in real time and provide a new set of prompts if the word count is insufficient. The new prompts may be designed to elicit longer or more responses. In another example, a user may attempt to manipulate the system (e.g., to gain an incentive or to avoid a diagnosis). In such cases, the ASR model can process the utterance to determine if it differs significantly from utterances from a "good" user. The input from the test user is then compared in real time to this model to see if the word patterns deviate significantly from what would be expected from a "good" user. This technique can capture users who play audio from another source instead of speaking live to the system, or users who speak but do not respond to the question asked. The system can then issue a warning to the user or tag the audio file.

[0131] Non-speech model In some cases, the systems described herein may include non-verbal models, including respiratory models, laughter models, demographic models, metadata models, and posture models. Modeling respiration may be useful in predicting anxiety or mania. Modeling laughter (or its absence) may be useful in predicting depression. Posture may also indicate certain behavioral or mental health conditions. The output of non-verbal models can be fused with the output of acoustic models.

[0132] Computer system This disclosure provides a computer system programmed to carry out the methods of this disclosure. Figure 3 shows a computer system 301 programmed or otherwise configured to implement the system 500 of Figure 5 or to perform the training process described herein.

[0133] The computer system 301 includes a central processing unit (CPU, also referred to herein as “processor” and “computer processor”) 305, which may be a single-core or multi-core processor, or multiple processors for parallel processing. The computer system 301 also includes memory or storage locations 310 (e.g., random-access memory, read-only memory, flash memory), an electronic storage unit 315 (e.g., hard disk), a communication interface 320 for communicating with one or more other systems (e.g., a network adapter), and peripheral devices 325, e.g., a cache, other memory, data storage, and / or an electronic display adapter. The memory 310, storage unit 315, interface 320, and peripheral devices 325 communicate with the CPU 305 via a communication bus (solid line), e.g., a motherboard. The storage unit 315 may be a data storage unit (or data repository) for storing data. The computer system 301 can be operably coupled to a computer network ("Network") 330 using a communication interface 320. Network 330 may be the Internet, the Internet and / or an extranet, or an intranet and / or extranet communicating with the Internet. Network 330 may optionally be a telecommunications network and / or a data network. Network 330 may include one or more computer servers that can enable distributed computing, such as cloud computing. Network 330 may optionally use computer system 301 to implement a peer-to-peer network, which may allow devices coupled to computer system 301 to operate as clients or servers.

[0134] The CPU 305 can execute a sequence of machine-readable instructions, which may be embodied in a program or software. The instructions may be stored in a memory location, for example, memory 310. The instructions may target the CPU 305, which can then be programmed or otherwise configured to implement the methods of this disclosure. Examples of operations performed by the CPU 305 may include fetching, decoding, executing, and writing back.

[0135] The CPU 305 may be part of a circuit, such as an integrated circuit. One or more other components of System 301 may be included in the circuit. In some cases, the circuit is an application-specific integrated circuit (ASIC).

[0136] The storage unit 315 can store files, such as drivers, libraries, and saved programs. The storage unit 315 can also store user data, such as user preferences and user programs. In some cases, the computer system 301 may include one or more additional data storage units located outside the computer system 301, for example, on a remote server communicating with the computer system 301 via an intranet or the internet.

[0137] Computer system 301 can communicate with one or more remote computer systems via network 330. For example, computer system 301 can communicate with a user's remote computer system. Examples of remote computer systems include personal computers (e.g., portable PCs), slate PCs or tablet PCs (e.g., Apple's iPad, Samsung's Galaxy Tab), telephones, smartphones (e.g., Apple's iPhone, Android devices, BlackBerry, or personal digital assistants). Users can access computer system 301 via network 330.

[0138] The methods described herein may be implemented by machine-executable code (e.g., computer processor) stored in an electronic storage location of the computer system 301, such as memory 310 or an electronic storage unit 315. The machine-executable code or machine-readable code may be provided in the form of software. In use, the code can be executed by the processor 305. In some cases, the code is retrieved from the storage unit 315 and stored in memory 310 for immediate access by the processor 305. In some cases, the electronic storage unit 315 may not be used, and the machine-executable instructions are stored in memory 310.

[0139] Code can be pre-compiled and configured for use with machines that have processors adapted to run the code, or it can be compiled at runtime. Code can be supplied in a programming language that may be chosen to allow it to run either pre-compiled or as-compiled.

[0140] Embodiments of the systems and methods provided herein, for example, computer system 301, may be embodied in programming. Various embodiments of the technology may typically be considered “products” or “goods,” taking the form of machine (or processor) executable code and / or related data carried on or embodied in some type of machine-readable medium. Machine executable code can be stored in electronic storage units, such as memory (e.g., read-only memory, random-access memory, flash memory) or hard disks. “Storage” type media may include any or all of the tangible memory of a computer, processor, or similar, or related modules thereof, such as various semiconductor memories, tape drives, disk drives, and similar, which may provide non-transient storage at any time for software programming. All or part of the software may be communicated over the Internet or other various telecommunication networks. Such communication may enable the loading of software from one computer or processor to another, for example, from a management server or host computer to an application server computer platform. Thus, other types of media that may carry software components include light waves, radio waves, electromagnetic waves, etc., used over various air-links that traverse physical interfaces between local devices via wired and optical fixed-line networks. Physical components that carry such waves, such as wired or wireless links, optical links, or similar, may also be considered media that carry software. Unless limited to non-transient, tangible “storage” media as used herein, the terms computer or machine-readable media refer to any medium involved in providing instructions to a processor for execution.

[0141] Therefore, machine-readable media, such as computer executable code, can take many forms, including but not limited to tangible storage media, carrier media, or physical transmission media. Non-volatile storage media include, for example, optical disks or magnetic disks, and storage devices in any computer or similar, such as those shown in the drawings, which may be used to implement databases. Volatile storage media include dynamic memory, such as the main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wiring and optical fibers, including wiring that includes buses in computer systems. Carrier media may take the form of electrical signals or electromagnetic signals, or acoustic or optical waves, such as those generated during radio frequency (RF) data communications and infrared (IR) data communications. Therefore, common forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs or DVD-ROMs, Blu-rays, any other optical media, punched card paper tapes, any other physical storage media having a pattern of holes, RAM, ROMs, PROMs, and EPROMs, FLASH-EPROMs, any other memory chips or cartridges, carriers for transmitting data or instructions, cables or links for transmitting such carriers, or any other medium from which a computer can read programming code and / or data. Many of these forms of computer-readable media may be involved in transporting one or more sequences of one or more instructions to a processor for execution.

[0142] The computer system 301 may include or communicate with an electronic display 335, which includes a user interface (UI) 340 that provides one or more queries to the user, for example, which may elicit utterances from the user. Examples of UIs include, but are not limited to, graphical user interfaces (GUIs) and web-based user interfaces.

[0143] The methods and systems of this disclosure may be implemented by one or more algorithms. The algorithms may be implemented as software when executed by the central processing unit 305. The algorithms may be, for example, any of the acoustic models, machine learning models, or training processes described herein.

[0144] While preferred embodiments of the present invention have been shown and described herein, it will be obvious to those skilled in the art that such embodiments are provided only as examples. The present invention is not intended to be limited by any specific example provided herein. While the present invention has been described with reference to the preceding specification, the descriptions and examples of embodiments herein are not intended to be constrained. Numerous modifications, alterations, and substitutions will be made by those skilled in the art without departing from the present invention. Furthermore, it will be understood that all aspects of the present invention are not limited to any specific descriptions, configurations, or relative proportions described herein, depending on various conditions and variables. It is desirable to understand that various alternative forms to the embodiments of the present invention described herein may be adopted when carrying out the present invention. Accordingly, the present invention is intended to also cover such alternative forms, modifications, variations, or equivalents. The appended claims define the scope of the present invention, and it is intended that methods and structures within the scope of the claims and their equivalents are covered. [Examples]

[0145] Speech-based depression prediction experiment This section describes a specific experiment and should not be construed as limiting other sections of this disclosure.

[0146] Speech-based depression prediction is important for future health management applications, but the need for large amounts of labeled training data can make it difficult to port to new languages. This experiment explores two novel alternative NLP techniques for depression prediction in a new language (Spanish) using only labeled training data from an existing language corpus (English). Both methods affect only the test data and not the training data. The English data (65,000 responses, training, and test) and the Spanish data (1,600 responses, test only) originate from spoken responses automatically copied in a similar behavioral health screening application. The machine translation technique uses the English "A Lite Bidirectional Encoder Representations from Transformers" (ALBERT) model to translate the Spanish test data from Spanish to English. The multilingual approach uses a multilingual text-to-text transfer transformer (mT5) model, in which case the model is constructed from parallel texts in over 100 languages ​​without direct, explicit translation. The results show that both methods achieve strong classification results compared to results from English training / testing alone, and the results are slightly better for machine translation than the multilingual approach.

[0147] Introduction and Background The prevalence of depression is widespread worldwide and has been steadily increasing over the past decade. Due to inadequate diagnostics of traits related to mental health status, there is significant pressure to utilize technologies in this field, particularly for screening. The Patient Health Questionnaire (PHQ-9), widely used for depression screening, lacks scalability and is difficult to use to support patients when their cognitive abilities are insufficient, especially when used independently. Conversational analysis in the mental health domain is another example of a method for measuring or screening depressive states. Various published literature has shown that such methods can be effective.

[0148] Digital technology may be used to generate an alternative to the PHQ-9, by enabling analysis of the patient's speech (either recorded as audio, copied as text, or both).

[0149] This disclosure focuses on natural language analysis. From statistical word frequency to n-gram techniques, word embeddings, language models, and transfer learning, NLP has advanced, enabling performance that significantly surpasses leading state-of-the-art NLP benchmarks.

[0150] Several trends are emerging in the field of NLP. One is transfer learning, which allows for the construction of general-purpose deep learning networks that are learned in most cases, enabling much better model performance on various downstream NLP tasks by predicting the occurrence of certain words in a given text passage (e.g., following words, word masking, or others). Another is the development of deep learning architectures, with transformer-type language models having recently achieved success. A third factor contributing to the latest breakthroughs in the field of NLP is data availability. Large corpora can help make models robust. Unfortunately, data for every language is not widely available. English is the most dominant language in the field of machine learning, and general, untargeted data is widely available. However, for several other languages ​​and downstream NLP tasks, data acquisition remains a challenge. In the past, there have been attempts to train separate language models for each language. Recent advances have made it possible to use multilingual pre-trained models that can be used for downstream tasks, which should be able to compensate for the lack of downstream labeled data.

[0151] The field of mental health screening is no exception to the problem of widely available data. Obtaining large language corpora can be difficult, for example, slowing the adoption of machine-based methods for detecting depression.

[0152] This disclosure relates to whether it is possible to train a classification language model on depression data labeled in English and to assess the state of depression on Spanish data. This experiment analyzes the ability of an English corpus to be used as a training set to predict the state of depression on a Spanish language test set. Thus, the following labeled data resources will be used: not only training and test sets of English text, but also a Spanish test set. Regarding methodology, this disclosure proposes a comparison of two of the most intuitive scenarios. The first scenario is to translate the Spanish data into English (automatically) and assess it on an English language model. The second scenario is to train a multilingual model, optimize it for the English corpus, and assess it on both English and Spanish assessment sets.

[0153] method data This disclosure uses American English and Spanish corpora as utterance sets. Both utterance sets were collected in the same manner; otherwise, data from different languages ​​could be confused with the methodology used to elicit utterances. In both corpora, participants were given monetary incentives. Users interacted with an app that posed questions on a variety of topics, such as “worries” and “family life.” Users responded by speaking freely. Corpus statistics are shown in Table 1. The large English Population (GP) corpus contains utterances from more than 15,000 sessions (65,000 responses). Training and test categories may not include overlapping speakers. Users' ages ranged from 18 to 65 years, with a mean age of 30. The average spoken response was approximately 160 words. Users answered 4 to 6 questions per session (average 4.5 questions).

[0154] The Spanish corpus includes over 300 sessions (1,600 responses). The question structure and technical environment are the same as the English corpus, and utterances were recorded using a computer within a time allocation of approximately 5 minutes per session. The average response length, measured in words, is slightly shorter than that of the English counterparts (Table 1).

[0155] During each session, users completed the PHQ-8 (or PHQ-9 after removing the suicidal tendencies question). Since each session was only a few minutes long, it was assumed that users did not change their state within the session. The PHQ-8 scores were then mapped for binary classification. Scores of 10 or higher were mapped to the presence of the state, and scores less than 10 were mapped to the absence of the state. [Table 1]

[0156] As mentioned above, the data labels represent the results of the PHQ-8 questionnaire. Figure 11 illustrates the label distribution for the English and Spanish test sets. Since the demographics are similar as well as the data collection methods, it might be expected that the labels would also be similar. For most PHQ scores, this is correct, and there is consistency between the two datasets. However, there are differences in the prevalence of these sets. The evaluation sets are in English and Spanish for 21% and 18% of the sets, respectively. The largest outlier is the training set, which has a prevalence of 28%. Model training

[0157] As described in the Data section, each patient interview is composed of multiple responses. Most NLP models (transformers) are initially built on a sequence of 512 input tokens. Due to limited time and resources, models for longer sequences were not trained (e.g., long-formers at the session level). As a result, if a session contains five responses, each response is treated as an independent record. All five responses were assigned the same PHQ session label.

[0158] This study compares two different methods. One is based on translating Spanish test data into English using a translation service (e.g., a cloud-based translation service). For this purpose, we use the English version of the ALBERT transformer language model. Its main advantage is that it has significantly fewer parameters than most other transformers (e.g., robustly optimized bidirectional encoder representations (RoBERTa)) and is more performance-oriented than its counterpart transformer. This method can be used without further training or tuning the original model if a translation model between the two languages ​​is available. Classification training was performed only on English data, while evaluation was performed not only on the English set but also on the Spanish-to-English translation set. The training phase was performed on a small GPU, 3 x 12GB, for which the batch size was 9, the learning rate was 1e-6, and training required 5 epochs.

[0159] Another approach is based on a pre-trained multilingual model. This experiment uses a pre-trained encoder / decoder MT5 model, trained on approximately 100 different language sets, including English and Spanish. Because NLP language models have grown considerably in terms of parameter size, and due to limited hardware resources, the disclosed methods do not compare similar models (English vs. multilingual) in terms of parameter count, which is a common practice in the NLP field. This experiment uses the 3-billion-parameter mT5 model because its T5 counterpart, the GLUE benchmark, is very similar to the ALBERT benchmark. This minimizes the possibility of misinterpreting the final performance results by attempting to use a model that performs similarly on language tasks in English. mT5 training was performed using only a pre-trained encoder, which was used as a feature extractor before the classifier. This method assumes that the encoder output is language-independent and can be effectively used for multilingual classification purposes. A single supervised training method without any fine-tuning of the language model was also applied here. The mT5 model was trained on 65,000 English responses, and evaluation was performed on English development and test sets as well as Spanish test data. The main training characteristics for the mT5 model were as follows: 1 epoch, 1e-5, and a batch size of 25.

[0160] result Mental health datasets are typically highly imbalanced, and the disclosed experiments use the Area Under the Receiver Operating Characteristic Curve (ROC AUC) metric to measure model performance on classification tasks. In addition, the disclosed experiments supplement results with equally specific error rate values ​​for specificity and sensitivity. This disclosure demonstrates the feasibility of constructing depression screening models for different languages ​​trained on an English corpus. The ALBERT and mT5 models provide comparable results to their English test counterparts. Both the development set and the test English set return very similar results, with a ROC AUC of ~0.73. It is important to note that such results are expected based on the assumption that the BLEU benchmark is also comparable. For the Spanish test set, the ALBERT architecture was slightly more effective than mT5 at the response level, but after data aggregation, mT5 significantly outperformed the English one. Aggregation to the session level is the mean of the predicted values ​​at the response level. In addition, the correlation coefficient of the predicted values ​​at the session level is 0.91. [Table 2]

[0161] When comparing the binary predictions of both models to the true labels, the similarity overlap is -92% (233 + 14 + 27 + 24). The largest difference occurs when ALBERT generates more false positive predictions (14 sessions) compared to 4 sessions by mT5. When mT5 predicts 0 and ALBERT predicts 1, artificially replacing mT5's prediction with ALBERT's prediction results in a 0.005 improvement in ROC AUC across all three trial subsets and the development subset. This indicates that further performance improvements on the evaluation set are possible through a more sophisticated fusion method. Our internal best performance on the English trial set is an ROC AUC of 0.84 or higher. [Table 3]

[0162] Discussion and Conclusion When training an NLP model using a corpus of English data, it is possible to predict depression in a different language (Spanish) in at least two ways. The model performance is effective in both methods. There is a correlation of 0.92 between the two methods proposed in this disclosure, and both models show equally identical performance at the response level (ROC AUC of -0.74), but the multilingual model stands out at the session level.

Claims

1. One or more computer processors, The system comprises a computer memory coupled to one or more computer processors, The aforementioned computer memory, when executed by the one or more computer processors, (a) A process of receiving an input signal that includes multiple speech or lexical features of a subject's speech, At least one of the aforementioned plurality of phonetic features or lexical features is related to at least one language, (b) A step of generating a probability distribution that indicates one or more languages ​​that may be present in the input signal using a language detector, (c) A step of determining the performance estimate of the input signal in one or more languages ​​for multiple models using a model performance estimator, (d) Based at least in part on the probability distribution representing the one or more languages ​​that may be present in the input signal and the performance estimates for the multiple models, A step of selecting one or more models from the plurality of models using a pre-model strategy selector, The aforementioned model includes a natural language processing (NLP) model, The aforementioned selection is, Selection of a general-purpose language model, Selection of a translation-based model, and A process including at least one selection of a multilingual model, (e) A process of generating predictions of behavioral or mental health status using one or more models with a result generator, A system for detecting behavioral or mental health conditions, having machine-executable code that implements processes including the following.

2. The system according to claim 1, wherein the pre-model strategy selector selects one or more models from the plurality of models based on resource availability.

3. The system according to claim 1, which provides one or more corrective measures for improving screening when the plurality of models are determined to be unreliable.

4. The selection of one or more models includes selecting a further fine-tuned natural language processing model, At least one of the general-purpose language model, the multilingual model, and the translation-based model generates an intermediate output. The system according to claim 1, wherein the intermediate output is processed by the finely tuned natural language processing model.

5. The pre-model strategy selector further selects one or more acoustic models from the plurality of models, The system according to claim 1, wherein the prediction of the behavioral or mental health state is generated using a fusion model or integrated model derived from the one or more acoustic models and the one or more natural language processing models.

6. Selecting one or more acoustic models from the plurality of models is: This includes at least one of the following: the selection of a language-independent acoustic model and the selection of a multilingual acoustic model. Furthermore, the selection includes selecting a finely tuned acoustic model. At least one of the language-independent acoustic model and the multilingual acoustic model generates an intermediate output. The system according to claim 5, wherein the intermediate output is processed by the finely tuned acoustic model.

7. The system according to claim 1, further comprising a feedback mechanism for recommending changes to mental health screening procedures.

8. The system according to claim 5, wherein the fusion model or the integrated model is further derived from one or more demographic models and / or metadata models.

9. The selection of one or more models is Prediction latency, The circumstances under which the input was collected, Business flow requirements, The monetary cost associated with using a particular model, or Demographic data that can determine the data flow path The system according to claim 1, which is at least partially based on at least one of the following.

10. The system according to claim 1, wherein the one or more models are general-purpose models that are fine-tuned using data from a specific domain.

11. The system according to claim 1, wherein the behavioral or mental health condition includes at least one of depression, anxiety, stress, schizophrenia, bipolar disorder, and post-traumatic stress disorder (PTSD).

12. The system according to claim 1, wherein the result of step (e) is provided as an electronic report.

13. The pre-model strategy selector is Number of answers, Length of the answer, User metadata, and Information regarding closed-loop sessions between the patient and the provider or system. The system according to claim 1, which receives at least one of the following as input.

14. If it is determined that the selected one or more models have produced unreliable results, The system according to claim 1, comprising selecting one or more improvement actions to improve predictions using a post-model strategy selector.

15. The post-model strategy selector is Model reliability, Number of answers, Length of the answer, User metadata, Audio / copy quality, and Topic score The system according to claim 14, which receives at least one of the following as input.

Images