Artificial intelligence technology for neurotoxicity detection

Abstract

Detection of neurotoxicity-related abnormalities caused by specific drug therapy is of great significance to recognition of various neurological disorders. The present disclosure relates to detecting neurotoxicity related disorders by receiving a response set corresponding to one or more query sets and a set of components using an interface containing the query sets. As disclosed herein, the techniques may leverage one or more attributes of the set of responses, including consistency, complexity, grammar and spelling correctness, and time taken during the response. The disclosed techniques may pre-process responses, extract features, and based on one or more metrics corresponding to one or more attributes of one or more responses provided by a subject during a session with an interface. Metrics generated on a baseline session and subsequent sessions may be used to determine trends of the metrics, which trends may be used to assess the degree of neurotoxicity produced by a subject, and may trigger alarms or prophylactic measures regarding potential neurotoxicity to reduce the likelihood of further neurotoxicity.

Description

Cross-reference to related applications

[0001] This application claims priority and benefit to U.S. Provisional Application No. 63 / 527,011, filed July 15, 2023, the entire contents of which are incorporated herein by reference for all purposes. Background Technology

[0002] Different drug treatments may carry a significantly higher risk of neurotoxicity due to the presence of harmful chemicals or toxic components. For example, chimeric antigen receptor T-cell therapy is used to treat subjects with various types of liquid tumors, such as lymphoma, leukemia, and multiple myeloma. For instance, some randomized controlled trials involving subjects who had received this chemotherapy treatment found that 25% of subjects experienced neurotoxicity, with 8% of these experiencing severe neurotoxicity. Neurotoxicity can cause damage to the brain or its peripheral nervous system. Persistent neurotoxicity leads to nerve cell death, causing irreversible damage to the brain and triggering symptoms such as cognitive impairment, motor impairment, or sensory impairment. Therefore, subjects may develop mental illnesses, such as anxiety, confusion, and depression.

[0003] The primary “treatment” for neurotoxicity lies in detecting the neurotoxicity and subsequently reducing or stopping exposure to the toxic substance (or harmful chemical) that causes it. However, early and reliable detection remains a significant challenge due to the wide range of symptoms that may be associated with neurotoxicity. Summary of the Invention

[0004] Some aspects and features of this disclosure relate to identifying neurotoxicity by analyzing one or more responses to one or more queries obtained via an interface on a user device. The interface on the user device may present a query set and be configured to receive a set of components from the user corresponding to a set of responses to the query set. A specific set of responses corresponding to the query set may be received from the user device at a backend or server for analysis. The specific set of responses may be processed using one or more artificial intelligence techniques to generate one or more metrics. At least one of the one or more metrics may each be based on: the degree of consistency between responses provided by the user in a given session of the interface, or between responses provided by the user in one or more other sessions; the complexity or sophistication of the responses provided by the user in the given session of the interface; the degree of conformity of the responses provided by the user in the given session of the interface to grammatical rules and / or correct spelling; one or more durations of time spent by the user providing responses during the given session; one or more durations of time the user pauses when providing two consecutive responses during the given session; and the cumulative duration of time the user pauses when providing responses during the given session.

[0005] The consistency among one or more responses in a response set can be evaluated by generating a distribution of one or more positions in a multidimensional space corresponding to one or more tokens within each of the one or more responses, and by comparing the distributions corresponding to the one or more responses. This consistency can be represented as an evaluation of the consistency among the responses in the response set based on the comparison using a metric generated using, for example, a similarity metric (e.g., a numerical metric).

[0006] In some cases, a metric can represent the complexity or sophistication of one or more responses corresponding to a given query. For example, one or more large language models (LLMs) can be used to generate a metric representing the complexity of the response.

[0007] A metric associated with a subject's time response can be generated using the delay between presenting one or more queries and receiving one or more corresponding responses. For example, an incremental time interval can be calculated for each query, set to be equal to the time between presenting the query and providing a response. The metric associated with the time response can be defined or associated with a statistic (e.g., mean, median, mode, etc.) based on multiple incremental time intervals. As another example, the metric can be defined or associated with the time between the initial presentation of at least a portion of a query set and the receipt of responses to all queries. It should be understood that the metric can be transformed and / or normalized based on past metrics and / or data associated with the subject and / or other metrics associated with other subjects.

[0008] The metric may indicate or be based on the duration of the session, or assess the cumulative time the user actively participated in the session. For example, the cumulative time may be counted as all time periods during which the application or webpage presenting the query is visible (e.g., in contrast to another application or webpage obscuring part or all of the interface of the query application or webpage, and / or in contrast to situations where the device presenting the application or webpage is currently or has previously been in a dormant state).

[0009] The metric can be based on the frequency of pauses detected during the reception of input corresponding to each response in one or more independent responses, the frequency of pauses detected between responses, and / or the duration of one or both types of pauses. For example, for a given response, each time window during the duration of the response input can be characterized as "active input" (e.g., when the user is typing part of the response) or "pause" (when no such input is received). For each response, the percentage of time windows that fall into the "pause" category can be calculated, and the metric can be defined as a statistic (e.g., mean percentage, median percentage, or mode percentage) generated based on percentages associated with multiple queries.

[0010] A comprehensive score can be generated based on the one or more measures. The comprehensive score may include statistics generated based on multiple measures among the one or more measures. For example, the comprehensive score may be defined as, or may be defined as, the mean, median, or mode based on these measures. In some cases, the comprehensive score is based on one or more relative measures. For example, at least one of the one or more measures may each be normalized based on other measures of a population (e.g., a healthy population, a population that has received or intends to receive a specific drug or a specific type of drug, a population exhibiting a specific symptom, etc.). As another example, a relative measure may include the difference between the measure corresponding to a subject's most recent session (or set of sessions) and the measure corresponding to one or more of their past sessions. Such relative measures may include the first derivative, the second derivative, etc.

[0011] In some cases, the comprehensive score can be generated using a weighted metric (or a value calculated accordingly). As an example, during the subject's initial session, weights can be assigned to each metric based on a comparison of the subject's metric value with group metric values ​​(e.g., assigning a higher weight when a metric is at the upper end of the group's metric distribution, or vice versa). As another example, weights can be dynamically assigned based on the degree of change of the metric between recent sessions (e.g., assigning a higher weight when a metric shows greater change compared to the subject's other metrics between recent sessions). As yet another example, weights can be assigned based on group data (e.g., data reflecting the degree of change of each metric between sessions or the degree of change for a subgroup).

[0012] One or more scores corresponding to the one or more metrics can be used to assess neurotoxicity levels by accumulating scores from two or more sessions and determining whether the scores or the composite score meet one of three conditions corresponding to negligible neurotoxicity, non-serious neurotoxicity, or severe neurotoxicity. If the condition is determined to be met, presentation, transmission, or execution related to a potential neurotoxicity alert can be triggered, or preventative measures expected to reduce the likelihood of further neurotoxicity can be triggered. Triggering conditions can be configured to be met when, for example, the composite score exceeds a threshold, the change in the composite score exceeds a threshold, the slope of the composite score exceeds a threshold, a metric exceeds a threshold, the slope of a metric exceeds a threshold, and / or any combination of the above occurs. In some cases, outlier detection techniques can be used to predict when a specific metric or composite score will become an outlier and to ignore the use of such a metric or composite score in subsequent processing (e.g., by excluding the specific metric from the calculation of the corresponding composite score, or by excluding the specific composite score from the condition assessment). One or more thresholds included in the conditions can be absolute or relative thresholds. For example, a relative threshold can be defined as a specific percentage of the composite score or metric value associated with the subject's previous sessions, or a statistic based thereon. As another example, relative thresholds can be defined based on other comprehensive score sets and / or other metric sets associated with other subject sets.

[0013] In some embodiments, a system is provided that includes one or more data processors and a non-transient computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of the methods of this disclosure.

[0014] In some embodiments, a computer program product tangibly implemented in a non-transitory machine-readable storage medium is provided, comprising instructions configured to cause one or more data processors to perform some or all of the methods or processes of this disclosure.

[0015] In some embodiments, a system is provided that includes one or more means, some or all of which are used to perform one or more methods or processes of this disclosure.

[0016] The terminology and expressions used herein are for illustrative purposes only and are not intended to limit the scope of the invention. No equivalents of the shown and described features or portions thereof are intended to be excluded, but it should be understood that various modifications may be possible within the scope of the claimed invention. Therefore, it should be understood that although the claimed invention has been specifically disclosed through embodiments and optional features, those skilled in the art can make modifications and variations to the concepts disclosed herein, and such modifications and variations should be considered to fall within the scope of the invention as defined in the appended claims. Attached Figure Description

[0017] The various embodiments will be described below with reference to the accompanying drawings. It should be noted that the drawings are not drawn to scale, and elements with similar structures or functions are represented by similar reference numerals throughout the drawings. It should also be noted that the drawings are for illustrative purposes only and are not intended to provide an exhaustive description of the present disclosure, nor do they limit the scope of the invention.

[0018] Figure 1 This shows an example distribution of different neurotoxic side effects experienced by the subjects due to different drug treatments.

[0019] Figure 2 This shows one or more examples of different types of damage experienced by the subjects due to neurotoxicity.

[0020] Figure 3 This displays an exemplary interface of a neurotoxicity detector on a user device, which performs a neurotoxicity assessment on a subject by asking multiple questions and recording and evaluating the characteristics of the corresponding responses.

[0021] Figure 4 This refers to various analyses performed on the subject's corresponding responses to the interface received from the subject's user device.

[0022] Figure 5 This describes an exemplary preprocessing procedure for the corresponding response received from the subject's user device.

[0023] Figure 6 This represents an example of a process for calculating one or more comprehensive scores related to different measures of the subject's corresponding response during a single session.

[0024] Figure 7 This is an example illustration of determining the evaluation result of the corresponding response in inter-session analysis based on one or more ratings associated with different metrics.

[0025] Figure 8This example chart shows how a score calculated for a metric across different sessions is compared to a baseline score for that metric to assess a subject’s level of neurotoxicity.

[0026] Figure 9 An example illustration shows how the neurotoxicity level of the subject can be assessed using the rate of change of the calculated score of the metric across different sessions.

[0027] Figure 10 This illustration shows an example of how to model the dynamic trends of the scoring group to assess the level of neurotoxicity of the subjects by performing an inter-subject analysis on the one or more scores of one or more subjects.

[0028] Figure 11 This illustrates one or more exemplary implementations of a management protocol developed for a subject based on whether a specific neurotoxicity condition is met.

[0029] Figure 12 This is an example flowchart illustrating a neurotoxicity detection process that predicts neurotoxicity in a subject by analyzing the corresponding responses received from the subject's user device.

[0030] Figure 13 This is an example illustration of a computer system containing one or more examples of executable neurotoxicity evaluators. Detailed Implementation

[0031] This disclosure relates to new techniques, methods, and systems for assessing neurotoxicity levels in a subject by analyzing one or more responses received from a user interface on a user device. The user interface may be configured to present one or more queries to a user and receive the user's corresponding responses, storing the queries and related responses in a user device or a cloud server storage device. One or more queries may include open-ended questions configured to receive responses in unstructured text in natural language. Therefore, for each query, the interface may include a text-based natural language response to the query, and a corresponding text box may be configured to allow the user to input a text-based natural language response to the query. Furthermore, the interface may be configured to receive the subject's spoken voice response to the query via a microphone on the user device, and the complete response may be stored in an audio file on the user device and subsequently transmitted to the cloud server storage device. In this example, when the subject responds to the query by speaking into the microphone on the user device, one or more speech processing techniques or artificial intelligence-based speech processing techniques may be used to analyze the response. In another example, the audio signal can be transcribed into text, and the complexity or precision of the text response can be analyzed using artificial intelligence (AI)-based natural language processing according to the response analysis method of this disclosure.

[0032] Response analysis methods may include using one or more AI techniques to generate one or more scores and / or metrics to assess: (1) the consistency between responses provided by users in a given session using the interface on a user device; (2) the complexity or sophistication of responses provided by users in a given session using the interface on a user device; (3) the degree to which responses provided by users in a given session using the interface on a user device conform to grammatical rules and / or are spelled correctly; (4) the amount of time a user spends providing responses during a given session; and (5) the number of pauses or the cumulative amount of time a user pauses while providing responses during a given session, etc.

[0033] In some cases, the properties of neural signals can also be used, or alternatively, to generate metrics and / or scores used according to the techniques disclosed herein. For example, one or more EEG (electroencephalogram) signals may be acquired during the session and / or at other time points. These EEG signals can be evaluated to, for example, predict levels of startle, focus, effort, concentration, and / or confusion. For instance, the signal (or a portion thereof) may be converted to detect one or more intensities within the beta band for assessing focus, effort, concentration, and / or confusion; or the signal (or a portion thereof) may be converted to detect one or more intensities within the gamma band for assessing startle. This analysis may include quantifying statistics (e.g., maximum, median, mode, mean, variance, standard deviation, or minimum) related to all or part of the beta band signal intensity during a portion or throughout the session. For example, variance statistics can be used to infer the extent to which subjects are able to maintain their attention on a task. Another example is that median, mode, or mean statistics can be used to infer overall startle levels. In some cases, similar analyses relating to neural signals can be performed even outside of a session. Such analyses help infer general cognitive abilities (e.g., maximum, median, mode, or mean levels of startle response, focus, effort, attention span, and / or confusion), and / or provide data to normalize any statistics, variables, or data corresponding to one or more sessions. In such cases, the analysis may be performed using data from outside the session, during which the subject is presumed to be awake. In some cases, similar or different analyses may be performed when the subject is presumed to be asleep. For example, EEG data, subject input data, and / or motion data may be used to infer that the subject is asleep. One or more features (e.g., features associated with the gamma band) may be evaluated to assess startle response, which may then be used, for example, to normalize session measures and / or contribute to the score.

[0034] In some cases, data features acquired by one or more sensors may also be used, or alternatively, to generate metrics and / or scores used according to the techniques disclosed herein. The sensors may include, for example, cameras or motion sensors such as accelerometers or gyroscopes. Relevant statistics may indicate the degree of movement during a session, shortly after the session ends (e.g., within subsequent 10-second, 30-second, 1-minute, 5-minute, 10-minute, or 30-minute intervals), and / or outside of sessions. For example, a statistic may assess daily step count (scoring rules may be configured, for example, to assess that a decrease in step count is associated with an increased probability of neurotoxicity). As another example, a statistic may predict the degree of tremor (scoring rules may be configured, for example, to predict that a stronger tremor is associated with an increased probability of neurotoxicity).

[0035] The rating can be configured to show a positive correlation between the likelihood of neurotoxicity and the level of focus, effort, concentration, and / or confusion assessed.

[0036] Supplemental or alternative assessments may characterize the degree of difference between one or more of the above ratings or metrics and the metric ratings corresponding to one or more previous sessions of the user. In some examples, one or more of these assessment methods may include using the same AI model to calculate the rating for each metric each time, or by incorporating assessment-specific dynamic factors to train a specialized assessment AI model to calculate the rating for each metric each time.

[0037] The one or more AI techniques may include the use of one or more trained machine learning models, which may include generative models, neural networks, long short-term memory models, converter models, moving average models (e.g., autoregressive integrated moving average (ARIMA) models), models using self-attention (such as ChatGPT), etc. In the example of using two or more trained machine learning models to generate different metric scores, the two or more trained machine learning models may belong to the same or different types described above; and in the example where the machine learning model is the neural network, these may be the same network architecture or different network architectures as disclosed herein.

[0038] AI technologies may include a preprocessing flow that removes stop words using a library and / or converts various words or phrases into corresponding lexical units. The lexical units can then be converted into vectors using an encoding model, such as a term frequency-inverse document frequency (TF-IDF) vectorizer or word2vec. The encoding model may be configured to perform the conversion based on the frequency of a given lexical unit (or a set of two or more related lexical units) in a specific response (or a combination of two or more responses), and / or the frequency of the given lexical unit or related lexical units in an underlying response set, which may be generated based on one or more responses from two or more users. A lexical unit that appears frequently in one user's response set but less frequently in the underlying response sets from two or more users can be interpreted as relatively important for conveying the semantics or meaning of the underlying set. Furthermore, the underlying training dataset can be analyzed to identify relatively frequent pairs of lexical units or combinations of three or more lexical units, or a set of responses from one or more responses, within the same response.

[0039] Subsequently, a distance metric that displays semantic similarity between word pairs or three or more word groups can be used to determine: the degree of consistency between responses provided by the user in a given session with the interface or between responses provided by the user in one or more other sessions; the complexity or precision of responses provided by the user in the given session with the interface; the degree to which responses provided by the user in the given session with the interface conform to grammatical rules and / or are spelled correctly; the amount of time the user spends providing responses during the given session; or the number of times or the cumulative amount of time the user pauses while providing responses during the given session.

[0040] In one example, consistency between a user's responses to the same response within a single session or across multiple sessions can be assessed by comparing lexical units in the response. For example, each lexical unit can be assigned a position in the multidimensional space based on a baseline dataset. Response consistency can be assessed based on an analysis of the distribution of lexical units in the multidimensional space: comparing a first positional distribution corresponding to a lexical unit position in a specific response or session with a second positional distribution corresponding to the lexical unit positions in different specific responses or different specific sessions; and calculating statistics based on the distance between lexical unit presentations (e.g., distance across responses within a single session or across responses across multiple sessions). These statistics determine the degree of consistency between responses provided by the user in a specific session with the interface and between responses provided by the user in one or more different specific sessions.

[0041] In one example, a generative model can be used to determine consistency between a subject's responses to one or more queries in a single session, or between the same responses across one or more sessions. The generative model can be used to predict a response to a query based on a subject's responses to one or more other queries in the same session or one or more previous sessions. Stop words can be removed from the predicted responses generated by the generative model and the user's actual responses, and lexical units are then generated and projected to different locations in the multidimensional space. Response consistency can be assessed based on the degree of difference between the distribution of lexical units in the subject's actual responses in the multidimensional space and the distribution of lexical units in the predicted responses corresponding to the generative model in the multidimensional space. Response consistency can be alternatively or additionally assessed by calculating a distance metric between the projections of the subject's actual responses in the multidimensional space and the projections of the predicted responses corresponding to the generative model in the multidimensional space.

[0042] In one example, one or more machine learning models or rule-based models can be used to determine the absolute or relative complexity or precision of one or more responses. For example, the model can be trained and configured to associate one or more lexical units or combinations thereof with a level of complexity. For instance, the model can be trained to associate lexical units from a first subset of training data (e.g., scientific literature) with complexity measures higher than those from a second subset of training data (e.g., social media posts).

[0043] In one example, the model can be trained in an unsupervised manner to learn how various features of the content correspond to complexity. In another example, one or more rules can be defined to calculate the complexity or precision of one or more responses based on: the syllable distribution of each word, the word distribution of each sentence, the frequency of various words or phrases used in the baseline dataset responding to the query, or the type of punctuation used.

[0044] In one example, a machine learning or rule-based model can evaluate the complexity or sophistication of a response to a query. For instance, a machine learning model can be trained in a supervised or unsupervised manner to identify important features that correspond to different levels of complexity or sophistication in the response. For example, in the training set, one or more variables such as response length, average number of letters per word, etc., can be used to calculate a complexity metric, which can then be used by the model to identify new features that correspond to the complexity or sophistication of the query response. In another example, a subject's response can be fed into a general model, which can then evaluate the subject's intelligence quotient (IQ) for providing the response.

[0045] In one example, alert criteria can be defined, which specifies when to issue an alert to the service provider based on the score of one or more metrics of the response. For example, the alert criteria may include a threshold that generates an alert when a metric or score exceeds a threshold corresponding to the level of no neurotoxicity or negligible neurotoxicity. The workflow for generating the alert may include sending an email, text message, or online message, or a combination thereof, to the server, which may contain a subject identifier, one or more responses, and one or more corresponding metric scores for determining that the subject's neurotoxicity level exceeds the threshold for negligible neurotoxicity.

[0046] In one example, once a subject completes a response to one or more queries on the user device, one or more scores for one or more metrics can be automatically calculated on the subject's user device. In one example, the service provider's user device can transmit the response to one or more queries to a provider's service running in the cloud, where one or more scores for one or more metrics are calculated. In one example, the subject may be arranged to appear in person at a healthcare service provider's practice, using the service provider's computer system to provide a response to one or more queries, where one or more scores for one or more metrics are calculated on the service provider's local device. In one example, the service provider's computer system can transmit the response to one or more queries to a service provider's service running in the cloud, where one or more scores for one or more metrics are calculated.

[0047] Once the neurotoxicity assessor determines that the subject's neurotoxicity level exceeds the threshold for negligible neurotoxicity, the artificial intelligence model may use a rule-based expert system to develop a comprehensive management plan for the subject (when the assessment is completed in the provider's practice). This plan includes, but is not limited to: (1) preparing one or more laboratory test orders; (2) adjusting the treatment plan or recommending termination of treatment if necessary; (3) submitting these to the physician for review and approval; and (4) sending the laboratory orders to the designated laboratory and the updated treatment plan to the designated pharmacy after approval by the physician. In this case, when the remote assessment is performed on the user's device, the management plan may include arranging an appointment between the subject and the service provider, transmitting the assessment analysis to the service provider, and confirming the appointment time with the subject after approval by the service provider's assistant. The scores of the one or more metrics, the analysis of these scores, and the inferences about the subject's neurotoxicity level may be stored in the service provider's local server or cloud server's electronic medical record (EMR) system.

[0048] Figure 1This example study 100 shows the distribution of subjects who developed neurotoxic side effects due to various drug treatments. The neurotoxic treatments can be broadly categorized into two types: chemotherapy and antibiotics. Many other therapies can also cause neurological damage, and each should be evaluated before administration. For example, CAR T-cell therapy can be used to treat subjects with different types of liquid tumors, such as lymphoma, leukemia, and multiple myeloma. One or more drug treatments from treatment regimen set 102 can be administered to one or more subjects 104. In this example study 100, approximately 25% of the subjects experienced neurotoxicity levels corresponding to those in neurotoxicity 108, 8% of the subjects receiving the treatment experienced severe neurotoxicity levels corresponding to those in severe neurotoxicity 110, and the remaining 65% of subjects may have no neurotoxicity levels or negligible neurotoxicity levels corresponding to those in subject 106 who did not experience neurotoxicity.

[0049] Figure 2 This illustrates one or more examples of different types of damage experienced by the subject due to neurotoxicity. These damages may result from persistent levels of neurotoxicity in the subject. Neurotoxicity can cause damage to the subject's brain or peripheral nervous system; persistent neurotoxicity will lead to nerve cell death, resulting in irreversible damage to the brain. In example 200, the subject having a neurotoxicity level corresponding to neurotoxicity 204 may experience cognitive impairment 206, motor impairment 208, or sensory impairment 210, etc. Therefore, the subject may suffer from mental illnesses such as anxiety, confusion, or depression.

[0050] Figure 3 An example interface 300 for a neurotoxicity detector on a user device is displayed, which performs a neurotoxicity assessment on a subject by asking multiple questions and recording and evaluating the characteristics of the corresponding responses. Various embodiments relate to new techniques, methods, and systems for assessing neurotoxicity. Specifically, a computing device such as a mobile device, desktop computer, or laptop computer may provide the example interface 300, which is configured to present multiple queries to a user and receive the user's corresponding responses. Interface 300 may present a query set 302a, 302b, ..., 302n to the subject. All of the above queries may be displayed to the subject at once or sequentially in a random manner. The queries may be or may include open-ended questions configured to receive text-based natural language responses. Thus, interface 300 may include natural language text 304 for each query and / or a corresponding text box 306 configured to receive the response in text-based natural language.

[0051] For each query, the interface 300 may additionally or alternatively include different input modes, such as storing audio of the query (e.g., “Query 1” 302a), which the subject can listen to by pressing component 308, and providing a corresponding response (e.g., “Response 1”) by pressing component 310. The subject can then speak into the microphone of the user device to record the response. Alternatively, the response can be typed into an input text box provided for each query. After completing the response, the subject can indicate completion by, for example, clicking or touching the “OK” button 312. When the response includes an audio signal, the audio signal can be analyzed directly or indirectly by transcribing it into text that can be analyzed according to the response analysis techniques disclosed herein. The subject can continuously provide responses (e.g., “Response 2”, ..., “Response n”) corresponding to the queries (e.g., 302b, ..., 302n) by clicking on the corresponding components on the interface 300s to select text-based and speech-based input options.

[0052] The timing of the subject's interaction with the interface can be scheduled, for example, once or twice a month, or midway between subsequent drug treatment doses, or as recommended by a healthcare professional. The content of the query set may depend on several factors, including the subject's knowledge level, exposure level, education background, experience, age, language, location, etc. Different query sets may be needed to evaluate the response of a particular subject.

[0053] Figure 4 This displays an exemplary evaluation 400 performed on one or more responses received from the interface 300 on the subject's user device. These evaluations may be performed using response analysis techniques, which may include using one or more artificial intelligence (AI) techniques to evaluate different aspects of the responses provided by the subject. The evaluation may include: determining consistency 403 (i.e., the degree of consistency between responses provided by the subject in a given session on the interface), complexity 405 or precision (i.e., the complexity or precision of the responses provided by the subject in a session on the interface 300), grammatical / spelling accuracy 407 (i.e., the degree to which the responses provided by the subject in a given session on the interface conform to grammatical rules and / or are correctly spelled), the time length t1, t2, ..., tn spent by the subject for each of the corresponding responses provided in the n queries, and the amount of time spent by the user providing a response during a single session (i.e., t1n (=∑t)). i))409, Pause time (i.e., the pause time when a subject provides a response to two consecutive queries (e.g., t12, t23, ..., t(n-1)n, where t12 represents the pause time spent by the subject between response 1 and response 2)), and the cumulative pause time ∑t when a user provides responses to all n queries during a given session. i(i+1) .

[0054] To assess the complexity of the response, the subject may be asked questions such as, “Have you experienced any changes in your motor abilities?” The subject may respond in one or more different ways: (1) “Yes”; (2) “Yes, I cannot effectively control my movements”; (3) “Yes, I always feel clumsy and my hands tremble when using the soldering iron.” The first of these responses is simple, the second is relatively complex, and the third is the most complex of the three example responses. Based on the complexity of the response, the first response may be assigned a low complexity score, the second a moderate complexity score, and the third a high complexity score. High complexity scores are expected in the initial phase of the drug treatment, while low scores are expected in the later phases of the drug treatment due to deterioration in mental health.

[0055] To assess the consistency of the responses, two or more questions with identical answers can be posed in different ways: (1) “Who is the current president of the United States?”; (2) “Who was the winner of the last US presidential election?”; (3) “What is the name of the husband of the current First Lady of the United States?”. The answers to these three questions are identical. If the subjects respond to all three questions with identical answers, these responses will be considered consistent and may be awarded a high consistency score. If the responses differ from each other, the consistency score may be significantly lower compared to the case where the answers are identical.

[0056] Additional or alternative evaluations may characterize the degree of difference between each of the above-mentioned variables and its corresponding variable from a previous session of the same user. In some cases, each of these evaluations may be performed using the same AI model or a separate evaluation-specific AI model to derive a performance metric for that particular evaluation. One or more performance metrics may then be used to generate a score.

[0057] Figure 5This describes an exemplary preprocessing flow 500 for one or more responses 520 received from the subject's user device. These responses 520 may be received in text form 502 or audio form 504 in response to one or more queries 510 during a single session between the subject and the interface 300. The preprocessing steps may be performed using one or more artificial intelligence (AI) technologies. For example, if one or more of the responses are in audio format, the speech-to-text conversion 506 may be performed to convert the audio input 504 into its corresponding transcribed text 508. Speech-to-text conversion may be performed using one or more machine learning (ML) models, such as recurrent neural networks (RNNs), long short-term memory (LSTM) networks, and transformer-based models, or automatically using services such as Google® Cloud Speech-to-Text, Amazon® Transcribe, and Microsoft® Azure Speech to Text. The text input 502 or the transcribed text input 508 may be recorded as a response in one or more lists of responses 520 related to the queries corresponding to one or more lists of queries 510. Further preprocessing 530 can be performed on one or more responses 520, which may involve, for example, stop word removal 532 (e.g., a, an, the, is, etc.) and / or lexicalization 534 (including converting various words or phrases into corresponding lexical units, for example, using a library). These lexical units can then be converted into vectors using encoding techniques 536 (e.g., Term Frequency-Inverse Document Frequency (TF-IDF) vectorizers, Word2vec, skip-gram, GloVe). The encoding technique 536 can be configured to perform the conversion based on the frequency of a given lexical unit (or related lexical units) in a specific response (or combination of responses), and / or based on the frequency of a given lexical unit (or related lexical units) in an underlying response set, which may be generated based on responses from numerous users and multiple evaluations by the same user.

[0058] This should be understood as meaning that audio input can be processed without being converted to text. For example, audio files can be processed to generate one or more statistics related to pitch, pauses, the reliability of various detected pauses and response times (compared to higher variability), and / or pitch variability.

[0059] Figure 6This describes an example method 600, according to some embodiments of the present disclosure, for calculating one or more scores related to different metrics of the subject's responses received from the interface 300 on the subject's user device during a single session. Preprocessing 530 of one or more responses 520 may generate multidimensional encoded vectors, which may be input to one or more machine learning / natural language processing techniques 605a, 605b, ..., 605c that, based on features, may generate one or more evaluation metrics for the responses 520 provided by the subject. The features may include, for example, identifying lexical units that are common in a particular subject's response set but less common in the underlying dataset, such lexical units may be interpreted as relatively important in conveying the meaning of the response set. Furthermore, the underlying dataset may be analyzed to identify lexical pairs or lexical groups that occur relatively frequently in the same response, response set, web page, etc. (and / or within a given range of distance from each other), and the lexical pairs and / or lexical groups may then be evaluated based on the analysis results of the underlying dataset to quantify, for example, predicted response consistency, response complexity, etc.

[0060] Evaluation may include metrics such as response consistency 611, response complexity 613, and grammatical and spelling accuracy 615. The one or more ML / NLP models may include one or more trained models, such as generative models, neural networks, long short-term memory models, transformer models, moving average models (e.g., autoregressive ensemble moving average (ARIMA) models), models using self-attention (e.g., ChatGPT), etc. In cases where different metrics are generated using the different trained machine learning models: two, more, or all different trained machine learning models may belong to the same model type and / or contain the same architecture; and / or two, more, or all different trained machine learning models may belong to the different model types and / or contain different architectures. The (evaluation) metrics may also include response time measurements 617, including the time lengths t1, t2, ..., tn spent by the subject answering each of the n queries, the amount of time tmn spent by the user providing a response n during m sessions, and the pause times t12, t23, ..., tn when the subject provides a response to two consecutive queries. (n-1)n And the cumulative time ∑t during which a user pauses while providing a response within a given session. i(i+1) .

[0061] In some cases, consistency between responses can be assessed by comparing the terms in the responses (responses to a query presented in a single session with the interface 300, or responses across different sessions). For example, terms can be mapped to corresponding positions in a multidimensional space using encoding techniques based on the underlying dataset (using vectorizers such as TF-IDF, Word2vec, skip-gram, GloVe, etc.). These positions can be represented as vectors in the multidimensional space (e.g., multidimensional Euclidean space). The consistency of the responses can be assessed based on: the positions of the terms across the space; a comparison of a first position distribution (e.g., terms corresponding to a specific response or session) with a second position distribution (e.g., terms corresponding to different specific responses or different specific sessions); calculating one or more statistics based on the distance between term representations (e.g., distances across responses within a single session or across responses from multiple sessions).

[0062] In some cases, generative models can be used to predict the response to a query based on responses to one or more other queries (e.g., from the same session or one or more previous sessions). For each predicted and actual response, stop words can be removed and lexical units generated. Each lexical unit can be projected to a position in a multidimensional space (using a vectorizer such as TF-IDF, Word2vec, skip-gram, GloVe, etc.). This position can be represented as a vector in the multidimensional space (e.g., multidimensional Euclidean space). The response consistency 611 can be evaluated based on the degree of difference between the lexical distributions of one or more actual responses and the lexical distributions of one or more corresponding predicted responses. Alternatively or additionally, response consistency can be evaluated based on the distance between the projections of lexical units in the actual responses and the projections of lexical units in the predicted responses.

[0063] In some cases, machine learning models or rule-based models can be used to evaluate the absolute or relative complexity (or precision) of one or more responses.613 For example, a model can be trained and configured to associate one or more lexical units or combinations thereof with a level of complexity. For instance, the model could be trained to associate lexical units from a first subset of training data (e.g., scientific literature) with complexity measures higher than those from a second subset of training data (e.g., social media posts). In another example, the model could be trained unsupervised to learn how various features of the content correspond to complexity. In yet another example, one or more rules can be defined to compute the complexity or precision of one or more responses based on: the syllable distribution of each word, the word distribution of each sentence, the frequency of various words or phrases used in the baseline dataset responding to the query, or the type of punctuation used. In some cases, response complexity 613 can be measured by treating the vectors in the response as a set of vectors and evaluating the mean, variance, or spatial distribution of the set of vectors, or by performing principal component analysis (PCA) and / or independent component analysis (ICA) to identify critical directions in subspaces of the multidimensional space.

[0064] In some cases, machine learning or rule-based models can evaluate the complexity or sophistication of a response to a query. For example, a machine learning model can be trained in a supervised or unsupervised manner to identify important features that can be used to correspond to different levels of complexity or sophistication of the response. For instance, in the training set, a complexity metric can be calculated using one or more variables such as response length, average number of letters per word, and then the model can identify new features that correspond to the complexity or sophistication of the query response. In another example, a response can be fed into a general model, requesting the general model to evaluate the IQ or grade level of the person who provided the response.

[0065] In some cases, two or more of these assessments can be performed (e.g., by the same AI model or by individually assessing a specific AI model) to derive a metric for that particular assessment. These metrics can then be used to generate scores. For example, score generation 620a, 620b, ..., 620p are p scores generated corresponding to the aforementioned assessments. Each score can be a scalar or a vector with a dimension greater than 1. For each session with the interface 300, the scores can be stored in storage device 630 (e.g., non-transient storage) for further querying, comparison, or aggregation of scores via aggregator 640.

[0066] Figure 7This is an example illustration of determining the evaluation result of the corresponding response in an inter-session analysis based on one or more ratings associated with different metrics. In this example 700, a baseline session rating 702 for a certain metric (which may represent the average of ratings from multiple sessions of one or more subjects) can even be used before the subject begins medication treatment. In some cases, the subject may have already started medication treatment before using the interface 300 on the user device to conduct an evaluation session. In such cases, the first session during the medication treatment process can be used as the subject's baseline session. Metric ratings for the subject's one or more subsequent sessions 704a, 704b, ..., 704n can be generated and stored in a storage device corresponding to the session identifier. The rating of the current session 704n can be compared with the ratings of one or more previous sessions 704a, 704b, ..., 704(n-1). Inter-session analysis 710 can be performed on a specific subject using an AI model to generate one or more rating patterns corresponding to one or more metrics (consistency, complexity, grammar / spelling, response time). The patterns and trends of one or more ratings can be determined, for example... Figure 7 Charts 720a, 720b, ..., 720p display a pattern of graphs, where each of one or more charts shows a pattern or trend of a single rating. For example, graph 720a shows the trend of change in rating 1 during one or more sessions for a particular subject. Similar trend graphs can be generated for ratings 2 through p during one or more sessions for a particular subject. Graphs 720b and 720p each show the trend of change in ratings 1 and p during one or more sessions for a particular subject.

[0067] Figure 8Example chart 800 illustrates how a score calculated for a metric across different sessions is compared to a baseline score for that metric to assess a subject's level of neurotoxicity. The score on the y-axis (referred to as the scoring axis) can be any of one or more scores corresponding to any of the one or more attributes (consistency, complexity, grammar / spelling, response time). All these scores for one or more sessions for a specific subject can be recorded via interface 300 on a user device. Scores for the first few sessions within the one or more sessions may represent transient responses within a transient range 810, while scores for the remaining sessions after the transient range 810 may represent steady-state responses within a steady-state range 820. Region 803 in chart 800 may correspond to a range of score values ​​with no neurotoxicity or negligible neurotoxicity. Region 805 in chart 800 may correspond to a range of score values ​​with non-severe neurotoxicity. In contrast, region 807 may correspond to a range of score values ​​with severe neurotoxicity. Once the transient response of a metric is complete, and the steady-state score of a particular subject is in one of regions 803, 805, or 807, and remains in the same region throughout the remainder of the one or more sessions until the start of the current session, the subject can be categorized with a label corresponding to that region, indicating negligible neurotoxicity, non-serious neurotoxicity, or severe neurotoxicity, to reflect the subject's level of neurotoxicity. In one example, the baseline score of a metric may be normalized to a numerical value, and the scores in subsequent sessions of the one or more sessions may be normalized relative to the baseline score.

[0068] If, at the start of the initial few sessions for a particular subject, a score is located in a first region associated with any one of the three regions 803, 805, or 807, subsequently shifts to a second region associated with a region different from the first region, and remains in the second region during the remaining sessions of the one or more sessions, then the particular subject can be classified as having the neurotoxicity level corresponding to the label in the second region. For example, if a particular subject's steady-state score remains in region 803 (e.g., no neurotoxicity or negligible neurotoxicity) until the fifth session, and then the score changes at the start of the sixth session, causing the score to shift to region 807 (e.g., severe neurotoxicity), and remains in the same region during the sixth, seventh, and eighth sessions, then it can be reasonably inferred that the particular subject has the severe neurotoxicity level.

[0069] In one example, if a subject's steady-state score begins in the first of the three regions 803, 805, or 807 and remains in that region for one or more sessions, then the steady-state score shifts to the second of the three regions 803, 805, or 807 (different from the first region) and remains in that region for one or more sessions, and finally the steady-state score shifts back from the second region to the first region and remains in that region for one or more remaining sessions: this may be an outlier. The reasons this scenario may represent an outlier could include one or more factors: the subject's surrounding environment during the session, signal interference from other people's voices near the subject, loss of network connection, or a transient psychological condition that may be unrelated to the drug treatment but could adversely affect the subject's response. Outliers can be detected using statistical techniques such as z-scores, modified z-scores, or the Tukey method, which identify outliers based on the standard deviation of the mean or median of the score sequence. Distance-based methods (such as k-nearest neighbors) can also be used to detect outliers by measuring the distance between each point in the multidimensional space corresponding to the score of the metric and its k nearest neighbors; if the distance exceeds a threshold, the point is considered an outlier. Outliers can also be detected by first clustering the scores of the corresponding attributes in the training dataset; outliers are scores that do not belong to any cluster or belong to only a few clusters. Supervised learning models (such as support vector machines (SVMs) and random forests) can also be configured during training to detect outliers.

[0070] An alert may be generated if a subject's steady-state score remains within the steady-state region 820 for two consecutive sessions in region 805 (corresponding to non-severe neurotoxicity) or region 807 (corresponding to severe neurotoxicity). The alert may be sent to the subject's user device and may also include a suggestion that the subject schedule a consultation session with a neurologist to discuss and confirm the level of neurotoxicity predicted by the region-based approach 800. During the consultation session with the neurologist, the subject may also discuss one or more management options, including adjusting the treatment regimen to slow the progression of the neurotoxicity or, if possible, restoring it to the negligible neurotoxicity level.

[0071] In another example, most steady-state scores on one or more metrics may indicate no neurotoxicity or negligible neurotoxicity, but at least one score may indicate severe neurotoxicity. An alert can still be generated under a conservative yet proactive mechanism, as it is prudent for the subject to consult a neurologist and confirm the absence or negligible neurotoxicity. This is preferable to a situation where no alert is generated and the subject actually suffers from severe neurotoxicity.

[0072] A composite score for a specific subject can also be generated from one or more score sets from a single session. This composite score can be, for example, the mean, weighted mean, median, mode, or geometric mean of the one or more score sets from a single session. A composite score sequence for a specific subject across one or more sessions can be generated, and the level of neurotoxicity can be inferred using the graphical method 800. The mean or moving average of the composite score sequence can also be used to infer the level of neurotoxicity for a specific subject.

[0073] Figure 9 One or more example charts 900 are shown to illustrate how the rate of change of the score calculated across different sessions can be used to assess the neurotoxicity level of the subject. In this case, the rate of change or derivative of the score (or composite score) across multiple sessions can be used to predict the neurotoxicity level of a particular subject. In example chart 902, the rate of change of the score across multiple sessions remains low, and its value fluctuates around the x-axis. Therefore, the absolute value of the ratio of the total positive class area to the total negative class area |(total positive class area) / (total negative class area)| can be close to 1, where the total positive class area and the total negative class area are the areas of the positive and negative regions between the Δ score curve and the horizontal axis. Therefore, the subject may be classified as having no neurotoxicity or negligible neurotoxicity 902.

[0074] In example chart 904, the rate of change of the ratings across multiple sessions remains large in the initial session and gradually decreases in later sessions. In this chart 904, the rating derivative is negative and below the x-axis in about half of the sessions, and slightly positive and slightly above the x-axis in about half of the sessions. Therefore, the absolute value of the ratio of the total positive class area to the total negative class area can be close to 0.5. In some cases, configurable conditions ensure that the initial decrease (e.g., a fairly consistent decrease) is sufficient to result in a classification of severe neurotoxicity. For example, empirical data may indicate that an initial decrease is sufficient to classify as predicting severe or irreversible neurotoxicity. In other cases, configurable conditions ensure that the initial negative derivative is insufficient to indicate that a classification of any irreversible or physiologically detectable neurotoxicity should be applied. Therefore, if subsequent rating changes tend towards zero or a positive value, a classification of no neurotoxicity or negligible neurotoxicity can be applied. In still other cases, configurable conditions ensure that the cumulative derivative indicates a classification of predicting non-severe neurotoxicity should be applied.

[0075] In Figure 906, the rate of change of a certain score remains consistently high and negative across all sessions. In this figure, the derivative of the score is negative and well below the x-axis for most sessions, with only a few sessions showing a slightly positive value above the x-axis. Therefore, the absolute value of the ratio of total positive class area to total negative class area |(total positive class area) / (total negative class area)| is likely close to 0. Thus, this subject may be classified as having severe toxicity.

[0076] Figure 10 The neurotoxicity level of test subject 1004 is assessed by comparing the scores of test subject 1004 with those of m other subjects 1010a, 1010b, ..., 1010m who may have developed neurotoxicity, as well as the score of baseline subject 1002. Inter-session analysis of the baseline subject 1002 can be generated over a period of time using method 700. The baseline subject may not have received any drug treatment or may have only received placebo treatment. In some cases, the baseline subject 1002 may not even be a real person, but a hypothetical person generated using ML clustering techniques. Inter-subject analysis 1030 can be performed by comparing the inter-session scores of test subject 1004 with the inter-session scores of the baseline subject 1002 and the inter-session scores of one or more subjects (subject 1 1010a to subject m 1010m). Inter-subject analysis 1030 for test subject 1004 can be performed by generating trends and patterns of different scores corresponding to various attributes using one or more statistical techniques. Trends and patterns in different scores can be presented in the form of charts 1020a, 1020b, ..., 1020p, where the lines in each chart show the scoring trend of a particular subject. When compared with other m subjects 1010a, 1010b, ..., 1010m who may have already experienced neurotoxicity, and with the baseline subject 1002 who did not receive drug treatment, the patterns of scores 1, 2, ..., p of test subject 1004 may indicate a declining trend in the quality of their attribute. Distance measures can be used to determine the degree of proximity of test subject 1004 to the baseline subject 1002 or one or more of the m subjects; based on this, the level of neurotoxicity of test subject 1004 can be predicted.

[0077] Figure 11This displays one or more actions triggered when neurotoxicity is detected in a subject. The inter-session analysis 710 of the subject can be further combined with the inter-subject analysis 1030 at 1102, and then a comparator 1104 detects whether a condition is met. This condition may include a score range or a predefined threshold corresponding to negligible neurotoxicity 803, non-serious neurotoxicity 805, or severe neurotoxicity 807. Based on the comparison result, one or more actions may be triggered, such as including an alert message 1106, recommendations for one or more preventative measures 1108, and / or triggering one or more response measures 1110. Alert criteria can be defined, specifying whether to generate the alert 1106 based on one or more metric scores. For example, the alert criteria may include a threshold that generates the alert 1106 when a metric or score exceeds the threshold. The alert may be generated when a subject's score enters a cluster associated with increased neurotoxicity. The alert may also be generated when a subject's score changes from being associated with one cluster to being associated with another cluster. In some cases, toxicity levels can be estimated for each cluster, and an alert can be generated when a subject's score is associated with a cluster whose assessed toxicity level differs from a predefined (e.g., received or learned) threshold. Generating the alert 1106 may include, for example, sending an email, SMS message, or online message, wherein any or all messages may contain a subject identifier, one or more responses (and / or queries), one or more metrics, and / or one or more scores. In some cases, each of one or more metrics and / or one or more scores may be output (e.g., transmitted to or presented on the subject's and / or corresponding healthcare provider's device). Such output may be performed automatically upon completion of the response processing, at a predetermined time, or upon request by the user or healthcare provider.

[0078] The metrics and / or scores can be used to automatically trigger one or more response options 1110, such as preparing one or more laboratory test orders, modifying prescriptions, arranging appointments between subjects and healthcare providers, etc. In some cases, orders or prescriptions can be automatically sent to designated laboratories or pharmacies, while simultaneously arranging and confirming appointments between subjects and healthcare providers. In some cases, the orders, prescriptions, or appointment requests can be prepared by an AI-based expert system; a request is then sent to the healthcare provider for approval or editing of the response measures. Once the response measures are approved by the healthcare provider, the orders or prescriptions can be sent to the designated laboratories or pharmacies, respectively. Similarly, appointments with neurologists can only be confirmed upon approval by the neurologist. Therefore, various examples can utilize statistical techniques or artificial intelligence to rapidly and efficiently detect the neurotoxicity levels in subjects and, where possible, initiate response measures to prevent or reverse the neurotoxic effects.

[0079] Figure 12The flowchart illustrates an example of method 1200, which determines the neurotoxicity level of a subject by utilizing an interface 300 on a user device associated with the subject. The blocks in the flowchart of method 1200 are arranged in a specific order, but this order can be adjusted; for example, some blocks may be executed before others, and some blocks may be executed simultaneously. These blocks can be executed via hardware or software, or a combination thereof. The process 1200 may include providing an interface in module 1202 containing a query set and a set of components to receive a set of responses corresponding to the query set. The query set may be presented in text form or in one or more audio recordings containing a readout of the query. The set of responses may be received in text or audio form. In module 1204, the set of responses may be received from the user device to a server, which may be located locally on the user device or a remote server in the cloud. The server performs preprocessing steps and a feature generation process to produce important features from the set of responses. In module 1206, one or more artificial intelligence techniques may be used to process the features corresponding to a specific set of responses to generate one or more metrics.

[0080] In module 1208, the one or more metrics can be analyzed to determine whether a certain condition is met. In module 1210, if the condition of module 1208 is met, an action can be triggered, which may include presentation, transmission, or action corresponding to a potential neurotoxicity alarm, or one or more preventive measures that may reduce the neurotoxicity level of the subject (or, if possible, restore it to a negligible neurotoxicity level).

[0081] Figure 13 This is an example illustration of a computer system 1300 that can implement various embodiments of the present disclosure. For example, the techniques described above (such as providing a user interface, presenting a query set, receiving a response set, response preprocessing, feature generation, score generation, score analysis, and triggering measures, etc.) can be implemented in computer-executable instructions (e.g., organized in program module 1304). Program module 1304 may contain routines, programs, objects, components, and data structures that perform the tasks described herein and implement data types for implementing the techniques described above. The functionality described herein can be performed at least in part by one or more hardware logic components.

[0082] To provide specific background information for each technical solution, Figure 13The following description is intended to provide a brief overview of the suitable computer system 1300 in which various technical approaches can be implemented. While the description is based on the general context of computer-executable instructions that can run on a single or multiple computers, those skilled in the art will recognize that novel implementations can also be combined with other program modules and / or as a combination of hardware and software. The computer system 1300 for implementing the various technical approaches includes a processing unit 1308 having one or more processors (also referred to as microprocessors), a computer-readable storage medium (which is any physical device or material that allows data to be stored and retrieved electronically and / or optically) such as a data storage unit 1320 (computer-readable storage media / media also include disks, optical disks, solid-state drives, external storage systems, and flash drives), and a system bus 1322. The system bus 1322 may serve as an interface between system components (including, but not limited to, system memory 1324) and the processing unit 1308. This type of system bus 1322 can be any of a variety of bus architecture types, and can be further interconnected to a memory bus (with or without a controller) and peripheral buses (such as PCI, PCIe, AGP, LPC, etc.) through any of a variety of commercially available bus architectures.

[0083] Figure 13 An example configuration of a classic computer is shown, which may be other commercially available microprocessors, such as single-processor, multi-processor, single-core, and multi-core processing and / or storage circuits. Furthermore, those skilled in the art will understand that the novel system and method described can be implemented in other computer system configurations, including minicomputers, mainframes, and personal computers (e.g., desktop computers, laptops, tablets, etc.), handheld computing devices, microprocessor-based or programmable consumer electronics, etc., wherein each type of device may be co-coupled with one or more associated devices.

[0084] In some technical solutions, the computer system 1300 may be one of several computers employed in a data center and / or computing resources (hardware and / or software) to support cloud computing services for portable and / or mobile computing systems such as wireless communication devices, cellular phones, and other mobile-enabled devices. Cloud computing services include, but are not limited to, Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), Storage as a Service (SaaS), Desktop as a Service (DAS), Data as a Service (DAS), Security as a Service (SAS), and Application Programming Interfaces (APIs) as a Service (APIs). In some cases, the system memory 1324 may include computer-readable storage (physical storage) media, such as volatile memory (e.g., random access memory (RAM) 1326) and non-volatile memory (e.g., read-only memory (ROM) 1328). The Basic Input / Output System (BIOS) may be stored in non-volatile memory and includes the basic routines that facilitate data and signal communication between internal components of the computer system 1300 (e.g., during startup). The volatile memory also includes high-speed RAM (e.g., static RAM) for caching data.

[0085] By way of example and not limitation, system memory 1324 may also contain program modules 1304 (which may include client applications, web browsers, middleware applications, relational database management systems (RDBMS), etc.), program data 1306, and operating system 1302. For example, operating system 1302 may contain multiple versions of Microsoft Windows®, Apple Macintosh®, and / or Linux operating systems, various commercially available UNIX® or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google Chrome OS, etc.), and / or mobile operating systems such as iOS, Windows® Phone, Android OS, BlackBerry® OS, and Palm® OS. All or part of operating system 1302, program modules 1304, and / or program data 1306 may also be cached in memory (e.g., the volatile memory and / or non-volatile memory, such as RAM 1326 or ROM 1328). It should be understood that the disclosed architecture can be implemented using various commercially available operating systems or combinations of operating systems (e.g., virtual machines).

[0086] In other examples, the computer system 1300 may have additional features or functions. For example, the computer system 1300 may also include additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. Computer-readable media may include at least two types of computer-readable media: computer storage media and communication media. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data.

[0087] System memory 1324 and data storage 1320, which includes both removable and non-removable storage, are examples of computer storage media. In addition to RAM 1326 and ROM 1328, computer storage media include, but are not limited to, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile optical disc (DVD), or other optical storage devices, cassette tape, magnetic tape, disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store target information and can be accessed by computer system 1300. Furthermore, the computer-readable medium may contain computer-executable instructions that, when executed by the processing unit 1308, can perform the various functions and / or operations described herein. In contrast, communication media may embody computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms.

[0088] The computer system 1300 may also include one or more input / output (I / O) devices 1332. One or more input devices among the one or more input / output devices 1332 may be, for example, a keyboard, mouse, pen, voice input device, touch input device, etc. One or more output devices among the one or more I / O devices 1332 may also be, for example, a monitor, speaker, printer, etc. These devices are well known in the art and will not be described in detail here. The computing device 1300 may also include one or more network interfaces 1330 to establish communication, enabling the computer system 1300 to communicate with other systems or devices (e.g., via a network). These networks may include wired networks and wireless networks. Here, the computer system 1300 is an example of a suitable device or system and is not intended to limit the scope or functionality of the various embodiments described.

[0089] Other common computer systems, environments, and / or configurations applicable to the described embodiments include, but are not limited to, personal computers, server computers, handheld or portable devices, multiprocessor systems, microprocessor-based systems, set-top boxes, game consoles, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments containing any of the aforementioned systems or devices. Some or all components of computer system 1300 may be implemented in a cloud computing environment, enabling resources and / or services to be provided via computer networks for selective use by user devices.

[0090] Some embodiments of this disclosure include a system comprising one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of the one or more methods and / or some or all of the one or more processes disclosed herein. Some embodiments of this disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium, including instructions configured to cause the one or more data processors to perform some or all of the one or more methods and / or some or all of the one or more processes disclosed herein.

[0091] The terminology and expressions used herein are descriptive and not restrictive, and their use is not intended to exclude any equivalent forms of the shown and described features or portions thereof. However, it should be recognized that various modifications may be possible within the scope of the invention as defined by the appended claims. Therefore, it should be understood that although the claimed invention has been clearly disclosed through specific embodiments and optional features, those skilled in the art can modify and adapt the concepts disclosed herein, and such modifications and adaptations should be considered within the scope of the invention as defined by the appended claims.

[0092] This disclosure provides only preferred exemplary embodiments and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the description of preferred exemplary embodiments will provide those skilled in the art with a feasible description of implementing various embodiments. It should be understood that various changes can be made to the function and arrangement of the elements without departing from the spirit and scope set forth in the appended claims.

[0093] The specific details provided in this disclosure are intended to provide a thorough understanding of the embodiments. However, it should be understood that these embodiments can be implemented without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form to avoid obscuring the embodiments with unnecessary details. In other cases, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments.

Claims

1. A method comprising: Provide an interface to a user device associated with a user, the interface including one or more query sets and a set of components for receiving a set of responses corresponding to the query sets; Receive a specific response set corresponding to a specific query set from the user equipment; The specific set of responses is processed using one or more artificial intelligence techniques to generate one or more metrics, wherein at least one of the metrics is based on: The degree of consistency between the responses provided by the users in a given session on the interface; The complexity or sophistication of the responses provided by the user in the given session on the interface; The degree to which the user's response in the given session on the interface conforms to grammatical rules and / or is spelled correctly; The amount of time the user spends providing a response during the given session; or The user provides the number of pauses or the cumulative amount of time during the given session; Determine whether the conditions are met based on one or more of the metrics; as well as In response to determining that the conditions are met, a presentation, transmission, or action corresponding to an alert regarding potential neurotoxicity or preventative measures to reduce the possibility of further neurotoxicity is triggered.

2. The method of claim 1, wherein at least one of the one or more measures is each based on: The degree of difference between each of the one or more metrics and the metrics corresponding to one or more previous sessions of the user.

3. The method according to claim 1, further comprising: A comprehensive score is generated by aggregating each of two or more metrics for a given session.

4. The method according to claim 1, further comprising: Each response in the specific response set is preprocessed by generating one or more corresponding tokens associated with each response in the specific response set.

5. The method of claim 1, wherein one or more metrics are generated based on the degree of consistency between the responses of the particular response set and evaluated in the following manner: A first position distribution and a second position distribution are generated by assigning each lexical unit in one or more lexical units associated with each response in the specific response set to a position in a multidimensional space; Perform a comparison of the first position distribution with respect to the second position distribution; and Based on the comparison, one of the one or more metrics is calculated to evaluate consistency.

6. The method of claim 1, wherein the complexity or sophistication of the response is evaluated using a Large Language Model (LLM).

7. The method of claim 1, wherein the conditions include negligible neurotoxicity, non-serious neurotoxicity, or severe neurotoxicity.

8. A system comprising: One or more data processors; as well as A non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform operations including the following steps: Provide an interface to a user device associated with a user, the interface including one or more query sets and a set of components for receiving a set of responses corresponding to the query sets; Receive a specific response set corresponding to a specific query set from the user equipment; The specific set of responses is processed using one or more artificial intelligence techniques to generate one or more metrics, wherein at least one of the metrics is based on: The degree of consistency between the responses provided by the users in a given session on the interface; The complexity or sophistication of the responses provided by the user in the given session on the interface; The degree to which the user's response in the given session on the interface conforms to grammatical rules and / or is spelled correctly; The amount of time the user spends providing a response during the given session; or The user provides the number of pauses or the cumulative amount of time during the given session; Determine whether the conditions are met based on one or more of the metrics; as well as In response to determining that the conditions are met, a presentation, transmission, or action corresponding to an alert regarding potential neurotoxicity or preventative measures to reduce the possibility of further neurotoxicity is triggered.

9. The system of claim 8, wherein at least one of the one or more metrics is each based on: The degree of difference between each of the one or more metrics and the metrics corresponding to one or more previous sessions of the user.

10. The system of claim 8, further comprising: A comprehensive score is generated by aggregating each of two or more metrics for a given session.

11. The system of claim 8, further comprising: Each response in the specific response set is preprocessed by generating one or more corresponding tokens associated with each response in the specific response set.

12. The system of claim 8, wherein one or more metrics are generated based on the degree of consistency between the responses of the particular response set and evaluated in the following manner: A first position distribution and a second position distribution are generated by assigning each lexical unit in one or more lexical units associated with each response in the specific response set to a position in a multidimensional space; Perform a comparison of the first position distribution with respect to the second position distribution; and Based on the comparison, one of the one or more metrics is calculated to evaluate consistency.

13. The system of claim 8, wherein the complexity or sophistication of the response is evaluated by utilizing a Large Language Model (LLM).

14. The system of claim 8, wherein the conditions include negligible neurotoxicity, non-serious neurotoxicity, or severe neurotoxicity.

15. A computer program product tangibly embodied in a non-transitory machine-readable storage medium, comprising instructions configured to cause one or more data processors to perform operations, said operations including: Provide an interface to a user device associated with a user, the interface including one or more query sets and a set of components for receiving a set of responses corresponding to the query sets; Receive a specific response set corresponding to a specific query set from the user equipment; The specific set of responses is processed using one or more artificial intelligence techniques to generate one or more metrics, wherein at least one of the metrics is based on: The degree of consistency between the responses provided by the users in a given session on the interface; The complexity or sophistication of the responses provided by the user in the given session on the interface; The degree to which the user's response in the given session on the interface conforms to grammatical rules and / or is spelled correctly; The amount of time the user spends providing a response during the given session; or The user provides the number of pauses or the cumulative amount of time during the given session; Determine whether the conditions are met based on one or more of the metrics; as well as In response to determining that the conditions are met, a presentation, transmission, or action corresponding to an alert regarding potential neurotoxicity or preventative measures to reduce the possibility of further neurotoxicity is triggered.

16. The computer program product of claim 15, wherein at least one of the one or more metrics is each based on: The degree of difference between each of the one or more metrics and the metrics corresponding to one or more previous sessions of the user.

17. The computer program product of claim 15, further comprising: A comprehensive score is generated by aggregating each of two or more metrics for a given session.

18. The computer program product of claim 15, further comprising: Each response in the specific response set is preprocessed by generating one or more corresponding tokens associated with each response in the specific response set.

19. The computer program product of claim 15, wherein one or more metrics are generated based on the degree of consistency between the responses of the particular response set and evaluated in the following manner: A first position distribution and a second position distribution are generated by assigning each lexical unit in one or more lexical units associated with each response in the specific response set to a position in a multidimensional space; Perform a comparison of the first position distribution with respect to the second position distribution; and Based on the comparison, one of the one or more metrics is calculated to evaluate consistency.

20. The computer program product of claim 15, wherein the complexity or sophistication of the response is evaluated using a Large Language Model (LLM).

Images