Amphasia grading evaluation system under multi-language scene

Abstract

The invention relates to the technical field of medical data mining, in particular to an aphasia grading evaluation system under a multi-language scene. The method comprises the following steps: firstly, acquiring language data of a patient in a dialogue type scene and non-language data of the patient in an instruction type scene through a data acquisition module; furthermore, in a performance analysis module, according to the language expression smooth performance of the language data of the patient in the dialogue scene, obtaining a language organization ability coefficient; according to the principal component features of the linguistic feature data of the patient in the instruction type scene, combining the principal component features of the non-linguistic clue data to obtain a semantic understanding ability coefficient; and finally, aphasia grading is performed on the patients through a grading evaluation module. According to the method, on the basis of the language expression condition in the traditional analysis language data, the principal component of the non-language data of the patient is combined, the language expression understanding ability of the patient is supplemented and corrected from the non-language perspective, and the accuracy and comprehensiveness of hierarchical evaluation are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical data mining, and in particular to an aphasia grading assessment system in a multilingual scenario. Background Art

[0002] Aphasia is a language disorder typically caused by damage to the brain, particularly to areas associated with language function. Aphasics can cause difficulty speaking fluently, understanding language, and writing. Aphasia often manifests differently in different communication scenarios, impacting daily life. Early diagnosis and treatment are crucial to helping patients regain their language function.

[0003] The current assessment system for aphasia patients is generally based on standardized language tests, which conduct symptom assessments on the patient's speech output characteristics. However, aphasia patients may exhibit behaviors such as speech pauses and repetitions during the assessment. At the same time, these behaviors are accompanied by some non-speech data (such as facial expressions, gestures, etc.). These non-speech data reflect the patient's aphasia symptoms. Ignoring these factors can easily lead to inaccurate aphasia grading assessments. Summary of the Invention

[0004] In order to solve the technical problem that existing assessment methods only focus on the patient's language characteristics, resulting in inaccurate graded assessment, the purpose of the present invention is to provide an aphasia graded assessment system in a multilingual scenario. The technical solutions adopted are as follows:

[0005] Data acquisition module: acquires the patient's language data in conversational scenarios and non-language data in instructional scenarios; the non-language data includes language feature data and non-language clue data;

[0006] Performance analysis module: obtains a language organization ability coefficient based on the language expression fluency performance of the patient's language data in the conversational scenario; obtains a semantic comprehension ability coefficient based on the principal component characteristics of the patient's language feature data in the instructional scenario, combined with the principal component characteristics of the non-verbal clue data;

[0007] Grading assessment module: compares the language organization ability coefficient and the semantic comprehension ability coefficient with corresponding preset normal indicators to grade the patient's aphasia.

[0008] Furthermore, the method for obtaining the language organization ability coefficient includes:

[0009] Extracting data related to the fluency of language expression of each speech segment from the language data; obtaining the fluency of language expression of each speech segment based on the data related to the fluency of language expression;

[0010] According to the overall characteristics of the language expression fluency of all speech segments of the patient in the conversational scenario, combined with the time interval between each speech segment and the previous speech of the conversation person, the patient's language organization ability coefficient is obtained; the overall characteristics of the language expression fluency are positively correlated with the language organization ability coefficient; the time interval is negatively correlated with the language organization ability coefficient.

[0011] Furthermore, the method for obtaining the fluency of language expression includes:

[0012] The data related to language expression fluency include: pronunciation speed, vocabulary repetition rate, total number of words and number of pauses; the language expression fluency of each speech segment is obtained based on the pronunciation speed, vocabulary repetition rate, total number of words and number of pauses of each extracted speech segment; the pronunciation speed and the total number of words are both positively correlated with the language expression fluency; the vocabulary repetition rate and the number of pauses are both negatively correlated with the language expression fluency.

[0013] Furthermore, the instruction-type scenario includes an instruction feedback action scenario and an instruction opinion statement scenario.

[0014] Furthermore, the method for obtaining the semantic understanding ability coefficient includes:

[0015] fusing the principal component scores of the patient in different instruction-type scenarios to obtain the principal component scores of the patient's non-verbal cue data;

[0016] The product of the principal component score of the language feature data and the principal component score of the non-language clue data is used as the semantic understanding ability coefficient.

[0017] Furthermore, the non-verbal clue data includes at least: changes in the patient's facial expression in the instruction feedback action scenario, and whether the action is hesitant or slow;

[0018] The patient's facial expression changes, gesture swing amplitude and logical coherence in the scenario of stating the instructions.

[0019] Furthermore, the method for obtaining the principal component score of the non-verbal clue data includes:

[0020] The principal component scores of the patient's non-verbal clue data in the instruction feedback action scenario and the principal component scores of the patient's non-verbal clue data in the instruction opinion statement scenario are integrated to obtain the principal component scores of the patient's non-verbal clue data; the principal component scores of the patient in the instruction feedback action scenario and the principal component scores in the instruction opinion statement scenario are both negatively correlated with the principal component scores of the non-verbal clue data.

[0021] Furthermore, the language feature data at least includes: whether the patient has redundant dialogues when executing instructions in instruction-type scenarios, whether the instructions are answered in a detailed and fluent manner, and the vocabulary of the instructions.

[0022] Furthermore, the method for grading aphasia in a patient includes:

[0023] When the language organization ability coefficient and the semantic comprehension ability coefficient are both greater than or equal to the corresponding preset normal indicators, the patient is determined to have grade 4 aphasia;

[0024] When the language organization ability coefficient is greater than or equal to the corresponding preset normal index, and the semantic comprehension ability coefficient is less than the corresponding preset normal index, the patient is determined to have grade 3 aphasia;

[0025] When the language organization ability coefficient is less than the corresponding preset normal index, and the semantic comprehension ability coefficient is greater than or equal to the corresponding preset normal index, the patient is determined to have grade 2 aphasia;

[0026] When the language organization ability coefficient and the semantic comprehension ability coefficient are both less than the corresponding preset normal indicators, the patient is determined to have grade 1 aphasia.

[0027] Furthermore, the conversational scenarios include single-person daily conversation scenarios and multi-person social interaction scenarios.

[0028] The present invention has the following beneficial effects:

[0029] The present invention first obtains the patient's language data in a conversational scenario and non-verbal data in a command-type scenario through a data acquisition module, providing a data basis for subsequent analysis; further, in the performance analysis module, based on the language expression fluency performance of the patient's language data in the conversational scenario, obtains a language organization ability coefficient to characterize the patient's language organization ability, providing a basis for subsequent accurate aphasia grading assessment; further, based on the principal component characteristics of the patient's language feature data in the command-type scenario and the principal component characteristics of the non-verbal clue data, obtains a semantic comprehension ability coefficient, obtaining the semantic comprehension ability coefficient from the perspective of non-verbal features, providing more basis for the grading assessment of aphasia patients and improving the accuracy of the assessment results; at the same time, in the process of calculating the semantic comprehension ability coefficient, the principal component analysis algorithm is used to remove feature redundancy, reduce data dimensions, and improve assessment efficiency; finally, the grading assessment module compares the language organization ability coefficient and the semantic comprehension ability coefficient with the corresponding preset normal indicators to grade the patient. The present invention improves the accuracy and comprehensiveness of the grading assessment by combining the principal components of the patient's non-verbal data with the language expression situation in the traditional analysis of language data, and supplements and corrects the patient's language expression and comprehension ability from a non-verbal perspective. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 This is a system block diagram of an aphasia grading assessment system in a multilingual scenario provided by one embodiment of the present invention;

[0032] Figure 2 A flowchart of aphasia grading determination provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features and effects of an aphasia grading assessment system in a multilingual scenario proposed by the present invention. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics of one or more embodiments may be combined in any suitable form.

[0034] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0035] The following describes in detail a specific scheme of an aphasia grading assessment system in a multilingual scenario provided by the present invention with reference to the accompanying drawings.

[0036] See also Figure 1 , which shows a system block diagram of an aphasia grading assessment system in a multilingual scenario provided by an embodiment of the present invention. The system includes: a data acquisition module 101, a performance analysis module 102 and a grading assessment module 103.

[0037] Data collection module 101: acquires the patient's language data in a conversational scenario and non-language data in a command scenario; the non-language data includes language feature data and non-language clue data.

[0038] In an embodiment of the present invention, different language scenarios are set and language data and non-language data of patients in different scenarios are collected with the help of multimodal sensors.

[0039] In one embodiment of the present invention, the set language scenarios include:

[0040] Single-person daily conversation scenario: Have a conversation with the patient on topics in daily life (such as family, food, etc.).

[0041] Multi-person social interaction scenario: The patient chats with multiple people to respond to the opinions of different roles.

[0042] Instruction feedback action scenario: Give the patient some specific instructions (such as "Please pass me that book, etc.").

[0043] Instruction opinion statement scenario: Give an opinion and ask the patient to express his or her views (such as "What do you think").

[0044] Among them, conversational scenarios include single-person daily conversation scenarios and multi-person social interaction scenarios, focusing on evaluating patients' language organization ability; instructional scenarios include instruction feedback action scenarios and instruction opinion statement scenarios, focusing on evaluating patients' semantic comprehension ability.

[0045] During the evaluation process, multimodal sensors (microphones, cameras, etc.) are used to collect language data (such as voice data) and non-verbal data from patients in different scenarios, and different behavioral change nodes are represented on the timeline. This allows for a comprehensive quantitative analysis of the patient's performance in multiple dimensions to minimize the possibility of misjudgment. Non-verbal data includes language feature data (such as whether the response to instructions is detailed and fluent, the vocabulary of the instruction response, etc.) and non-verbal clue data (such as facial expressions, gestures, and other behaviors). Non-verbal data can be manually collected and statistically analyzed by relevant personnel to provide a data basis for subsequent analysis.

[0046] It should be noted that the setting methods of various language scenes are technical means well known to those skilled in the art. The number of dialogues or instructions in each scene can be set by the implementer, but the number of scene settings for different patients needs to be consistent, and no limitation is imposed here.

[0047] Performance analysis module 102: obtains the language organization ability coefficient based on the language expression fluency performance of the patient's language data in the conversational scenario; obtains the semantic comprehension ability coefficient based on the principal component characteristics of the patient's language feature data in the instructional scenario, combined with the principal component characteristics of the non-verbal clue data.

[0048] The patient's language characteristics are the most intuitive manifestation of aphasia symptoms, and can also show the patient's language organization ability and semantic comprehension ability to the greatest extent. The patient's non-verbal data is generally used to supplement the language description. Therefore, first determine the patient's language organization ability based on the patient's voice characteristics and language feedback performance in different scenarios, and then supplement and correct the patient's language comprehension ability based on the non-verbal data, thereby improving the accuracy of the patient's aphasia grade assessment.

[0049] Considering that the fluency of the patient's language expression in conversational scenarios reflects the patient's language organization ability, we first obtain the language organization ability coefficient based on the language expression fluency performance of the patient's language data in conversational scenarios to characterize the strength of the patient's language organization ability and provide a basis for subsequent accurate aphasia grading assessment.

[0050] Preferably, in one embodiment of the present invention, considering that language data cannot directly quantify the expression fluency feature, relevant data of the language expression fluency of each speech segment in the language data is first extracted; based on the relevant data of the language expression fluency, the language expression fluency of each speech segment is obtained, and from the perspective of the language expression fluency, a basis is provided for analyzing the patient's language organization ability and obtaining the language organization ability coefficient. The greater the language expression fluency, the stronger the language organization ability.

[0051] Considering the time interval between each speech segment of the patient and the previous speech of the person being spoken to, it represents the patient's reaction time to questions and conversations, and also shows the patient's language organization ability. The longer the time interval, the longer the reaction time and the worse the language organization ability. Therefore, the time interval between each speech segment and the previous speech of the person being spoken to is also analyzed;

[0052] Based on this, the patient's language organization ability coefficient was obtained according to the overall characteristics of the patient's language expression fluency in all speech segments in conversational scenarios, combined with the time interval between each speech segment and the previous speech of the conversational person; the overall characteristics of language expression fluency were positively correlated with the language organization ability coefficient; the time interval was negatively correlated with the language organization ability coefficient.

[0053] As an example, the mean of the language expression fluency of all speech segments of the patient in a conversational scenario is taken as the overall expression fluency, which is used to represent the overall characteristics of the language expression fluency of all speech segments of the patient in the conversational scenario; the sum of the time intervals between each speech segment of the patient and the previous speech of the conversational person is taken as the overall reaction time, which represents the time interval of all speech segments; the ratio of the overall expression fluency to the overall reaction time is taken as the language organization ability coefficient of the corresponding patient; the overall reaction time is in the denominator.

[0054] It should be noted that each patient's response or speech is considered a speech segment; the difference between the start time of each speech segment and the end time of the previous speech of the interlocutor is taken as the time interval between each speech segment and the previous speech of the interlocutor; in the process of calculating the language organization ability coefficient, only the numerical value of the data is taken;

[0055] In other embodiments of the present invention, the implementer may also fuse the mean, mode and median of the language expression fluency by weighted summation, such as with weights of 0.5, 0.25 and 0.25, to obtain the overall expression fluency; the language organization ability coefficient may also be further normalized, specifically by using linear normalization.

[0056] Preferably, in one embodiment of the present invention, considering that the faster the pronunciation speed, the lower the word repetition rate, the higher the total number of words in the speech segment and the fewer the number of pauses, the more fluent the patient's language expression is and the higher the fluency of language expression;

[0057] Based on this, the relevant data of language expression fluency include: articulation speed, vocabulary repetition rate, total number of words and number of pauses; the language expression fluency of each speech segment is obtained based on the articulation speed, vocabulary repetition rate, total number of words and number of pauses of each extracted speech segment; the articulation speed and total number of words are positively correlated with language expression fluency; the vocabulary repetition rate and number of pauses are both negatively correlated with language expression fluency.

[0058] As an example, the product of the difference between the total number of words and the number of pauses in each speech segment and the ratio of the pronunciation speed to the word repetition rate is used as the language expression fluency of each speech segment.

[0059] It should be noted that speech recognition technologies such as the Transformer model or the long short-term memory network model are used to convert speech data into text, count the total number of words, and take the ratio of the total number of words to the length of the speech segment as the pronunciation speed; JieBa word segmentation tools are used for word segmentation, and text analysis tools such as dictionary building are used to count repeated words, and the ratio of the number of repeated words to the total number of words is taken as the vocabulary repetition rate; the number of times the energy in the speech segment is lower than the energy threshold and lasts for more than 200ms is counted as the number of pauses. The energy threshold can be set to 20% of the maximum energy, and the speech energy of each sampling point is the square of the amplitude of each sampling point.

[0060] In other embodiments of the present invention, the implementer may also use the product of the articulation speed and the total number of words as the numerator, the sum of the product of the number of pauses and the word repetition rate and the preset normal constant 1 divided by zero as the denominator, and use the ratio of the fractions as the fluency of language expression; the differences between the articulation speed, word repetition rate, total number of words and number of pauses may also be fused by weighted summation. The algorithms and weighted summation methods used in the method of obtaining the relevant data of language expression fluency are both existing technologies and will not be described in detail.

[0061] Next, we analyze the patient's non-verbal data in the directive scenario to assess their semantic comprehension ability. Semantic comprehension ability is closely related to the patient's verbal feedback in the directive scenario. Because patients' semantic comprehension ability has characteristics of different dimensions, we need to use principal component analysis (PCA) to extract the main features of different patients' language characteristics in the directive scenario to reduce the interference of redundant information.

[0062] The principal component features represent the core characteristics of the patient's language and non-verbal performance in directive scenarios. They have removed redundant information from the data and highlighted the most important dimensions for semantic comprehension ability. Therefore, based on the principal component features of the patient's language feature data in directive scenarios and combined with the principal component features of the non-verbal clue data, the semantic comprehension ability coefficient is obtained from the perspective of non-verbal features, providing more basis for the graded assessment of aphasia patients and improving the accuracy of the assessment results. At the same time, in the process of calculating the semantic comprehension ability coefficient, the principal component analysis algorithm is used to remove feature redundancy, reduce data dimensions, and improve assessment efficiency.

[0063] Preferably, in one embodiment of the present invention, considering that the principal component score directly quantifies the principal component characteristics, the principal component characteristics are represented by the principal component score; considering that the instruction-type scenario is divided into the instruction feedback action scenario and the instruction opinion statement scenario, and the two instruction-type scenarios have different focus points on the patient's non-verbal cues, the principal component scores of the patient's non-verbal cue data are first calculated separately, and then the principal component scores of the patient in different instruction-type scenarios are integrated to obtain the principal component scores of the patient's non-verbal cue data;

[0064] As an example, the language feature data may include at least: whether the patient's instructions in a command-based scenario contain unnecessary dialogue, whether the patient's answers are precise and fluent, and the vocabulary of the instructions. The principal component score of the language feature data may be obtained by:

[0065] First, the patient's language feature data for each test in the instruction feedback action scenario and the instruction opinion statement scenario were collected; for example, [0, 0, 12], where 1 represents yes and 0 represents no, 1 represents no unnecessary dialogue and 1 represents a detailed and fluent instruction response, and [0, 0, 12] represents unnecessary dialogue and an incomplete and sparse instruction response. The vocabulary size of the instruction response is 12. The vocabulary size can be obtained by converting speech data into text using speech recognition technology, creating a dictionary after word segmentation, and counting the number of words in the dictionary. Whether there is unnecessary dialogue during the execution of instructions and whether the instruction response is detailed and fluent are recorded by relevant personnel.

[0066] Further standardize the different language feature data by subtracting the mean of each feature data and then dividing it by its standard deviation to eliminate the differences in the dimensions and value ranges of different features;

[0067] We then performed principal component analysis to calculate the covariance matrix, eigenvalues, and eigenvectors. We selected the top N principal components that explain most of the variance and set the principal component with a cumulative variance contribution of 70%. This yielded the following explanations of semantic understanding in aphasic patients from different dimensions (principal components):

[0068] The first principal component (PC1): represents the comprehension of vocabulary or sentence structure;

[0069] The second principal component (PC2): represents the dimension related to language fluency and grammatical correctness;

[0070] The third principal component (PC3): represents the dimensions related to language fluency and vocabulary;

[0071] Since each principal component corresponds to a dimension of semantic comprehension ability, the principal component score of the language feature data is obtained to indicate the strength of the patient's ability to understand voice commands. Specifically, the eigenvector of the principal component is multiplied by the standardized data matrix, and the product is used as the score vector of each principal component. The mean of the data in the score vector is used as the average score of each principal component. Different principal component weights are set according to the variance contribution rate, and the principal component score is obtained by weighted summation of the average scores.

[0072] For example, the data matrix after standardization Eigenvector of principal component 1 Eigenvectors of principal component 2 The score vector of the principal component obtained is The average score corresponding to v1 is 0.37, and the average score corresponding to v2 is -0.09. Assuming that the variance contributions of principal components 1 and 2 are 0.56 and 0.24 respectively, which are 0.7 and 0.3 after normalization, the principal component score C = 0.37 × 0.7 + (-0.09) × 0.3 = 0.24.

[0073] Preferably, in one embodiment of the present invention, considering non-verbal cues also helps the assessor understand the patient's intentions. If these non-verbal cues are ignored when assessing aphasia, the patient's semantic comprehension ability may be underestimated. Therefore, further consideration of non-verbal cues is necessary to supplement semantic comprehension ability.

[0074] In the command-feedback action scenario, patients are assessed for their semantic comprehension ability by executing the instructions. Ideally, this scenario does not involve verbal feedback. If the patient's facial expression shows confusion or wandering eyes after receiving the instructions, this indicates that the patient may not understand the meaning of the instructions and has poor semantic comprehension ability. Furthermore, since the patient needs to execute the instructions, if they understand the instructions, they will show their intention to execute them. Otherwise, they may show hesitation and avoidance. Therefore, the non-verbal cues that may be generated include both facial expressions and behaviors.

[0075] It should be noted that the patient's understanding behavior is not limited to being able to understand or not understanding. There may also be situations where the patient does not understand at first and then gradually understands. Therefore, it is necessary to evaluate the final result of the patient's execution of the instructions.

[0076] As an example, non-verbal clue data include at least: changes in the patient's facial expressions in the instruction feedback action scenario, and whether the actions are hesitant and slow; for example, changes in facial expressions (0 indicates no obvious changes in expression, 1 indicates subtle changes in expression, and 2 indicates obvious changes in expression, such as confusion and tension), whether the actions are hesitant and slow (0 indicates smooth execution, 1 indicates slight delay in action, and 2 indicates obvious delay or pause in action), [0, 0] indicates that the patient has no changes in expression, no delay in action, and the patient fully understands the instructions.

[0077] The same method as obtaining the principal component score of language feature data was used to input the non-verbal clue data in the instruction feedback action scenario to obtain the principal component score of the patient's non-verbal clue data in the instruction feedback action scenario, which represents the richness of non-verbal clues. The higher the score, the richer the patient's non-verbal clues for understanding instructions, reflecting that the patient's understanding of instructions is lower and the corresponding semantic comprehension ability is worse.

[0078] Considering that the instructions and opinions are open-ended, patients not only need to understand the instructions but also actively organize language to express their own opinions. If the patient's expression is natural when answering, it means that the patient can express himself confidently after understanding the instructions and has strong semantic comprehension ability. On the contrary, exaggerated and dull expressions indicate that the patient may have difficulties in semantic comprehension and needs time to process information, and is trying to use expressions to compensate for the difficulty in language organization. Similarly, excessive, random, and uncoordinated gestures may mean that the patient needs additional physical assistance to compensate for the lack of semantic understanding during the expression process. The patient's logical connection also reflects the patient's information processing and expression ability.

[0079] Based on this, the patient's facial expression changes, gesture swing amplitude, and logical coherence in the instruction point statement scenario are evaluated. For example, facial expression changes (0: natural, 1: slightly exaggerated, 2: very exaggerated / sluggish), gesture swing amplitude (0: normal, 1: slightly casual, 2: obviously uncoordinated or excessive), and logical coherence (0: clear logic, no interruptions, 1: less than two interruptions, 2: more than two logical interruptions); [1, 1, 1] means that the patient's expression is slightly exaggerated, gestures are slightly casual, and there are occasional (less than two) logical interruptions.

[0080] Similarly, the same method as obtaining the principal component scores of the language feature data is used to input the non-verbal clue data in the instruction opinion statement scenario to obtain the principal component scores of the patient's non-verbal clue data in the instruction opinion statement scenario; the higher the principal component score, the richer the non-verbal clues, reflecting the lower the patient's understanding of the instructions and the worse the corresponding semantic comprehension ability.

[0081] Furthermore, the principal component scores of the patient's non-verbal cue data in the instruction feedback action scenario and the principal component scores of the patient's non-verbal cue data in the instruction opinion statement scenario are fused to obtain the principal component scores of the patient's non-verbal cue data.

[0082] The larger the patient's principal component scores in the instruction feedback action scenario and the principal component scores in the instruction opinion statement scenario, the worse the patient's semantic comprehension ability. Therefore, the patient's principal component scores in the instruction feedback action scenario and the principal component scores in the instruction opinion statement scenario are negatively correlated with the principal component scores of the non-verbal clue data.

[0083] As an example, the product of the principal component scores of the non-verbal clues of the two instruction-type scenes is subjected to an inverse negative correlation mapping with the sum of a preset positive constant 0.1 divided by zero, and the result is used as the principal component score of the non-verbal clue data.

[0084] The product of the principal component scores of the language feature data and the principal component scores of the non-language clue data is taken as the semantic comprehension ability coefficient.

[0085] It should be noted that in other embodiments of the present invention, the principal component scores of the language feature data and the principal component scores of the non-language clue data may be fused by weighted summation or the like to obtain the semantic understanding ability coefficient; the semantic understanding ability coefficient may also be further normalized, such as linear normalization; the variance contribution rate may be selected between 70% and 90% to select the principal component;

[0086] Implementers can also set other language feature data, such as the number of correct sentences, the number of effective words (excluding stop words, catchphrases, etc.), the number of redundant dialogues (such as repeated instruction dialogues), etc., to replace or add them; in the instruction feedback action scenario, other non-verbal clue data such as the number of expression changes, the number of eye wanderings, the number of action pauses, whether active execution, etc. can also be set to replace or add them; in the instruction opinion statement scenario, other non-verbal clue data such as the number of logical interruptions, the number of expression changes, etc. can also be set to replace or add them.

[0087] In another embodiment of the present invention, the implementer may also multiply the eigenvector of the principal component with the standardized data matrix, sum the score vectors corresponding to all principal components, take the modulus, and use the modulus value as the corresponding principal component score.

[0088] Considering that in the instruction-feedback action scenario, the more consistent the patient's actions are with the instruction requirements, and in the instruction-statement scenario, the shorter the patient's thinking or incubation time is when expressing an opinion, the stronger the patient's semantic comprehension ability is. Implementers can also use the product of the number of times the instruction requirements are met, the inverse of the incubation time and value of all opinions, and the principal component scores of the language feature data and the principal component scores of the non-verbal cue data as the semantic comprehension ability coefficient. Alternatively, a weighted summation method can be used to integrate and obtain the semantic comprehension ability coefficient.

[0089] Grading assessment module 103: compares the language organization ability coefficient and the semantic comprehension ability coefficient with the corresponding preset normal indicators to grade the patient's aphasia.

[0090] After analyzing the patient's language data and non-language data in different scenarios in the performance analysis module 102, the patient's language organization ability coefficient and semantic comprehension ability coefficient are evaluated from the perspective of language expression and non-language clues, and the basis for aphasia classification is obtained. Therefore, the language organization ability coefficient and semantic comprehension ability coefficient are further compared with the corresponding preset normal indicators to automatically classify the patient's aphasia.

[0091] Preferably, in one embodiment of the present invention, the method for grading aphasia in a patient comprises: Figure 2 , which shows a flow chart of aphasia classification determination provided by one embodiment of the present invention, specifically including:

[0092] When the language organization ability coefficient and semantic comprehension ability coefficient are both greater than or equal to the corresponding preset normal indicators, the patient is judged to have grade 4 aphasia;

[0093] When the language organization ability coefficient is greater than or equal to the corresponding preset normal index, and the semantic comprehension ability coefficient is less than the corresponding preset normal index, the patient is judged to have grade 3 aphasia;

[0094] When the language organization ability coefficient is less than the corresponding preset normal index, and the semantic comprehension ability coefficient is greater than or equal to the corresponding preset normal index, the patient is judged to have grade 2 aphasia;

[0095] When the language organization ability coefficient and the semantic comprehension ability coefficient are both lower than the corresponding preset normal indicators, the patient is judged to have grade 1 aphasia.

[0096] Figure 2 In the scale, A represents the difference between the language organization ability coefficient and the corresponding preset normal index, and B represents the difference between the semantic comprehension ability coefficient and the corresponding preset normal index. The aphasia levels are ranked from high to low, and the symptoms are from mild to severe, with level 4 aphasia having the mildest symptoms and level 1 aphasia having the most severe symptoms.

[0097] It should be noted that the two preset normal indicators are obtained by collecting language data and non-language data from a number of normal people to obtain the language organization ability coefficient and the semantic understanding ability coefficient, and the mean of a certain proportion of the minimum data is used as the preset normal indicator. For example, data from 100 normal people is collected, and the mean of the minimum 10% of the data is set as the preset normal indicator for the corresponding type of data.

[0098] It should be noted that the aphasia grading results obtained are only for reference by doctors and other relevant personnel, and provide some data support for relevant personnel to intervene or treat patients.

[0099] In summary, in order to address the technical problem that the existing assessment method only focuses on the patient's language features, resulting in inaccurate graded assessment, the present invention proposes an aphasia graded assessment system in a multi-language scenario. The present invention first obtains the patient's language data in a conversational scenario and non-language data in an instructional scenario through a data acquisition module; further, in a performance analysis module, the language organization ability coefficient is obtained based on the language expression fluency performance of the patient's language data in the conversational scenario; the semantic understanding ability coefficient is obtained based on the principal component characteristics of the patient's language feature data in the instructional scenario, combined with the principal component characteristics of the non-language clue data; finally, the language organization ability coefficient and the semantic understanding ability coefficient are compared with the corresponding preset normal indicators through a graded assessment module to grade the patient's aphasia. The present invention improves the accuracy and comprehensiveness of the assessment by supplementing and correcting the patient's language expression and comprehension ability from a non-language perspective based on the traditional analysis of the language expression in the language data and combining it with the principal components of the patient's non-language data.

[0100] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0101] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A multilingual aphasia grading assessment system, characterized by: The system comprises: Data acquisition module: acquires the patient's language data in conversational scenarios and non-language data in instructional scenarios; the non-language data includes language feature data and non-language clue data; Performance analysis module: obtains a language organization ability coefficient based on the language expression fluency performance of the patient's language data in the conversational scenario; obtains a semantic comprehension ability coefficient based on the principal component characteristics of the patient's language feature data in the instructional scenario, combined with the principal component characteristics of the non-verbal clue data; Grading assessment module: compares the language organization ability coefficient and the semantic comprehension ability coefficient with corresponding preset normal indicators to grade the patient's aphasia.

2. The aphasia grading assessment system in a multilingual setting according to claim 1, characterized in that: The method for obtaining the language organization ability coefficient includes: Extracting data related to the fluency of language expression of each speech segment from the language data; obtaining the fluency of language expression of each speech segment based on the data related to the fluency of language expression; According to the overall characteristics of the language expression fluency of all speech segments of the patient in the conversational scenario, combined with the time interval between each speech segment and the previous speech of the conversation person, the patient's language organization ability coefficient is obtained; the overall characteristics of the language expression fluency are positively correlated with the language organization ability coefficient; the time interval is negatively correlated with the language organization ability coefficient.

3. The aphasia grading assessment system in a multilingual setting according to claim 2, characterized in that: The method for obtaining the language expression fluency includes: The data related to language expression fluency include: pronunciation speed, vocabulary repetition rate, total number of words and number of pauses; the language expression fluency of each speech segment is obtained based on the pronunciation speed, vocabulary repetition rate, total number of words and number of pauses of each extracted speech segment; the pronunciation speed and the total number of words are both positively correlated with the language expression fluency; the vocabulary repetition rate and the number of pauses are both negatively correlated with the language expression fluency.

4. The aphasia grading assessment system in a multilingual setting according to claim 1, characterized in that: The instruction-type scenarios include instruction feedback action scenarios and instruction opinion statement scenarios.

5. The aphasia grading assessment system in a multilingual setting according to claim 4, characterized in that: The method for obtaining the semantic understanding ability coefficient includes: fusing the principal component scores of the patient in different instruction-type scenarios to obtain the principal component scores of the patient's non-verbal cue data; The product of the principal component score of the language feature data and the principal component score of the non-language clue data is used as the semantic understanding ability coefficient.

6. The aphasia grading assessment system in a multilingual setting according to claim 5, characterized in that: The non-verbal clue data includes at least: changes in the patient's facial expression in the instruction feedback action scenario, and whether the action is hesitant or slow; The patient's facial expression changes, gesture swing amplitude and logical coherence in the scenario of stating the instructions.

7. The aphasia grading assessment system in a multilingual setting according to claim 5, characterized in that: The method for obtaining the principal component score of the non-verbal clue data includes: The principal component scores of the patient's non-verbal clue data in the instruction feedback action scenario and the principal component scores of the patient's non-verbal clue data in the instruction opinion statement scenario are integrated to obtain the principal component scores of the patient's non-verbal clue data; the principal component scores of the patient in the instruction feedback action scenario and the principal component scores in the instruction opinion statement scenario are both negatively correlated with the principal component scores of the non-verbal clue data.

8. The aphasia grading assessment system in a multilingual setting according to claim 5, characterized in that: The language feature data at least includes: whether the patient has redundant dialogues when executing instructions in instruction-type scenarios, whether the instructions are answered in a detailed and fluent manner, and the vocabulary of the instructions.

9. The aphasia grading assessment system in a multilingual setting according to claim 1, characterized in that: The method for grading aphasia in a patient comprises: When the language organization ability coefficient and the semantic comprehension ability coefficient are both greater than or equal to the corresponding preset normal indicators, the patient is determined to have grade 4 aphasia; When the language organization ability coefficient is greater than or equal to the corresponding preset normal index, and the semantic comprehension ability coefficient is less than the corresponding preset normal index, the patient is determined to have grade 3 aphasia; When the language organization ability coefficient is less than the corresponding preset normal index, and the semantic comprehension ability coefficient is greater than or equal to the corresponding preset normal index, the patient is determined to have grade 2 aphasia; When the language organization ability coefficient and the semantic comprehension ability coefficient are both less than the corresponding preset normal indicators, the patient is determined to have grade 1 aphasia.

10. The aphasia grading assessment system in a multilingual setting according to claim 1, characterized in that: The conversational scenarios include single-person daily conversation scenarios and multi-person social interaction scenarios.

Images