A cognitive warning method, device, storage medium and equipment

Abstract

The present application discloses a cognitive early warning method, device, storage medium and equipment, the method comprising: first, obtaining the process data and result data generated by the target subject when performing cognitive classification, wherein the process data includes the voice data, video data, and eye movement data generated by the target subject during the cognitive classification process, and the result data is the electronic scale data obtained by the target subject after cognitive classification, then, extracting the characteristic data of the target subject from the process data and the result data, and determining the classification result of the target subject accordingly, and then, when it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, sending cognitive abnormality early warning prompt information to the target associated user of the target subject. Thereby, the cognitive classification result of the target subject can be determined more accurately, and when the cognitive abnormal classification is determined, the early warning prompt information is immediately sent to the associated user of the target subject.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a cognitive early warning method, device, storage medium and equipment. Background Art

[0002] With the increasing aging of society, cognitive abnormalities in the elderly have become a problem that cannot be ignored. In order to improve the living standards of the elderly, early classification screening and intelligent early warning of cognitive abnormalities play a key role. When it is determined that the elderly have cognitive abnormalities, early warnings can be given to their family members or other people with whom they have a relationship in a timely manner, so that corresponding measures can be taken as soon as possible to adjust the elderly's cognitive conditions, such as through targeted rehabilitation training to adjust the elderly's cognitive conditions, etc., to improve the living standards of the elderly.

[0003] At present, when the cognitive results of the elderly (i.e., subjects) are classified, they are mainly classified through one-on-one tests between the examiner and the subject, which mainly depends on the current test situation of the subject. The examiner will give the cognitive situation classification results based on the test results of the subject and give rehabilitation training program suggestions. However, this classification method, on the one hand, depends on the subjective classification results of the examiner, which is subjective and inconsistent. Different examiners are often difficult to unify in the consistency of judgment on cognitive classification questions. On the other hand, it only relies on the current test results of the subject for classification, which cannot fully reflect the cognitive performance of the subject, and the rehabilitation training program suggestions given also depend on the experience of the examiner, which has certain limitations. Therefore, when the existing classification method is used to classify the cognitive status of the subject, the accuracy of the classification results is low, and the corresponding preventive measures are not taken in the first time, which is not conducive to timely improving the cognitive status and living standards of the elderly. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a cognitive warning method, device, storage medium and equipment, which can improve the accuracy of the authentication classification results when cognitive classification of subjects is performed, and can send warning prompt information to related users in a timely manner, so as to improve the cognitive status and living standards of the elderly as soon as possible.

[0005] The present application embodiment provides a cognitive early warning method, including:

[0006] Acquire process data and result data generated by the target subject to be classified during cognitive classification, wherein the process data includes voice data, video data, and eye movement data generated by the target subject during cognitive classification, and the result data is electronic scale data obtained by the target subject after cognitive classification;

[0007] Extracting characteristic data of the target subject from the process data and result data;

[0008] Determining a cognitive classification result of the target subject according to the characteristic data of the target subject;

[0009] When it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, cognitive abnormality warning prompt information is sent to the target associated user of the target subject.

[0010] In a possible implementation, extracting characteristic data of the target subject from the process data and result data includes:

[0011] extracting audio features of the speech data from the speech data;

[0012] extracting video features of the video data from the video data;

[0013] extracting eye movement features of the eye movement data from the eye movement data;

[0014] extracting a scale test feature of the electronic scale data from the electronic scale data;

[0015] The audio features, the video features, the eye movement features and the scale test features are spliced ​​to obtain feature data of the target subject.

[0016] In a possible implementation, extracting the audio features of the speech data from the speech data includes:

[0017] Acoustic features, linguistic features, duration features, and spectrogram image features are extracted from the speech data and concatenated to obtain audio features of the speech data.

[0018] In a possible implementation, extracting the video features of the video data from the video data includes:

[0019] Lip shape features, facial expressions and other features are extracted from the video data and spliced ​​to obtain video features of the video data.

[0020] In a possible implementation, extracting eye movement features of the eye movement data from the eye movement data includes:

[0021] The gaze duration of different areas, the jump frequency of different areas, and the movement trajectory characteristics of the eye gaze point are extracted from the eye movement data and spliced ​​to obtain the eye movement characteristics of the eye movement data.

[0022] In a possible implementation, extracting the scale test features of the electronic scale data from the electronic scale data includes:

[0023] The test scores of each item in the neuropsychology scale are extracted from the electronic scale data and concatenated to obtain the scale test features of the electronic scale data.

[0024] In a possible implementation, when the target subject is cognitively classified for the first time, determining the cognitive classification result of the target subject according to the characteristic data of the target subject includes:

[0025] Using the characteristic data of the target subject, constructing pseudo-cohort data;

[0026] Inputting the characteristic data of the target subject into a first cognitive classification model to determine a cognitive classification result of the target subject;

[0027] Among them, the first cognitive classification model is constructed on the basis of a pre-constructed cross-sectional data classification model using the characteristic data of the pseudo-cohort data; the cross-sectional data classification model is constructed using the characteristic data generated by the most recent cognitive classification of M subjects collected in advance; and M is a positive integer greater than 1.

[0028] In a possible implementation, the constructing pseudo-cohort data using the characteristic data of the target subject includes:

[0029] Acquire historical cohort data generated when K subjects were previously cognitively classified as candidate cohort data; wherein K is a positive integer greater than 1;

[0030] The similarity between the characteristic data of the target subject and the characteristic data of all the candidate cohort data is calculated, and the candidate cohort data corresponding to the similarity that meets the preset conditions is used as the pseudo cohort data of the target subject.

[0031] In a possible implementation, the constructing pseudo-cohort data using the characteristic data of the target subject includes:

[0032] Acquire historical cohort data generated when L subjects perform cognitive classification as candidate cohort data; L is a positive integer greater than 1;

[0033] The pseudo-cohort data selection model is used to select candidate cohort data that meet preset matching conditions from the candidate cohort data as the pseudo-cohort data of the target subject.

[0034] In a possible implementation, when the target subject is subjected to cognitive classification for the Nth time, where N is a positive integer greater than 1, determining the cognitive classification result of the target subject according to the characteristic data of the target subject includes:

[0035] Inputting the characteristic data of the target subject into a pre-constructed second cognitive classification model to determine a cognitive classification result of the target subject;

[0036] Among them, the second cognitive classification model is constructed on the basis of a pre-constructed cross-sectional data classification model, using the characteristic data of the previous N-1 historical cohort data of the target subject; it is constructed using the characteristic data generated by the most recent cognitive classification of M subjects collected in advance.

[0037] In a possible implementation, the cross-sectional data classification model is constructed as follows:

[0038] Obtaining sample process data and sample result data generated by sample subjects when they perform cognitive classification;

[0039] Extracting characteristic data of the sample subject from the sample process data and the sample result data;

[0040] Training the initial cross-sectional data classification model according to the characteristic data of the sample subjects and the cognitive classification identification labels corresponding to the sample subjects to generate the cross-sectional data classification model;

[0041] The initial cross-sectional data classification model includes an input embedding layer, an encoding layer, and a decoding layer.

[0042] In a possible implementation, when the target subject is subjected to cognitive classification for the first time and it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, the method further includes:

[0043] Inputting the characteristic data of the target subject into the Input Embedding layer of the cross-sectional data classification model to obtain a first low-dimensional feature vector;

[0044] Inputting the first low-dimensional feature vector into a pre-constructed first rehabilitation training program recommendation model to obtain a rehabilitation training program recommended to the target subject;

[0045] Among them, the first rehabilitation training program recommendation model is constructed using the first low-dimensional feature vector corresponding to the cross-sectional data generated by the most recent cognitive classification of P subjects collected in advance; P is a positive integer greater than 1.

[0046] In a possible implementation, when the target subject is subjected to cognitive classification for the Nth time and it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, the method further includes:

[0047] Inputting the feature data of the target subject into the Input Embedding layer of the second cognitive classification model to obtain a second low-dimensional feature vector and a third low-dimensional feature vector;

[0048] Inputting the second low-dimensional feature vector and the third low-dimensional feature vector into a pre-constructed second rehabilitation training program recommendation model to obtain a rehabilitation training program recommended to the target subject;

[0049] Among them, the second rehabilitation training program recommendation model is constructed on the basis of the pre-constructed first rehabilitation training program recommendation model, using the second low-dimensional feature vector generated by the target subject's previous N-1 cognitive classifications; the first rehabilitation training program recommendation model is constructed using the low-dimensional feature vectors generated by the most recent cognitive classification of P subjects collected in advance.

[0050] The present application also provides a cognitive warning device, including:

[0051] A first acquisition unit is used to acquire process data and result data generated by the target subject to be classified when performing cognitive classification, wherein the process data includes voice data, video data, and eye movement data generated by the target subject during the cognitive classification process, and the result data is electronic scale data obtained by the target subject after cognitive classification;

[0052] A first extraction unit, used to extract characteristic data of the target subject from the process data and result data;

[0053] A classification unit, used to determine a cognitive classification result of the target subject according to the characteristic data of the target subject;

[0054] The early warning unit is used to send cognitive abnormality early warning prompt information to the target associated user of the target subject when it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification.

[0055] In a possible implementation, the first extraction unit includes:

[0056] A first extraction subunit, configured to extract audio features of the speech data from the speech data;

[0057] A second extraction subunit, configured to extract video features of the video data from the video data;

[0058] A third extraction subunit, configured to extract eye movement features of the eye movement data from the eye movement data;

[0059] A fourth extraction subunit, configured to extract a scale test feature of the electronic scale data from the electronic scale data;

[0060] The splicing subunit is used to splice the audio features, the video features, the eye movement features and the scale test features to obtain the feature data of the target subject.

[0061] In a possible implementation, the first extraction subunit is specifically used for:

[0062] Acoustic features, linguistic features, duration features, and spectrogram image features are extracted from the speech data and concatenated to obtain audio features of the speech data.

[0063] In a possible implementation, the second extraction subunit is specifically used for:

[0064] Lip shape features, facial expressions and other features are extracted from the video data and spliced ​​to obtain video features of the video data.

[0065] In a possible implementation, the third extraction subunit is specifically used for:

[0066] The gaze duration of different areas, the jump frequency of different areas, and the movement trajectory characteristics of the eye gaze point are extracted from the eye movement data and spliced ​​to obtain the eye movement characteristics of the eye movement data.

[0067] In a possible implementation, the fourth extraction subunit is specifically used for:

[0068] The test scores of each item in the neuropsychology scale are extracted from the electronic scale data and concatenated to obtain the scale test features of the electronic scale data.

[0069] In a possible implementation, when the target subject is subjected to cognitive classification for the first time, the classification unit includes:

[0070] A construction subunit, used to construct pseudo-cohort data using the characteristic data of the target subject;

[0071] A classification subunit, configured to input the characteristic data of the target subject into a first cognitive classification model to determine a cognitive classification result of the target subject;

[0072] Among them, the first cognitive classification model is constructed on the basis of a pre-constructed cross-sectional data classification model using the characteristic data of the pseudo-cohort data; the cross-sectional data classification model is constructed using the characteristic data generated by the most recent cognitive classification of M subjects collected in advance; and M is a positive integer greater than 1.

[0073] In a possible implementation, the construction subunit includes:

[0074] A first acquisition subunit is used to acquire historical cohort data generated when K subjects were previously cognitively classified as candidate cohort data; K is a positive integer greater than 1;

[0075] The calculation subunit is used to calculate the similarity between the characteristic data of the target subject and the characteristic data of all the candidate queue data, and use the candidate queue data corresponding to the similarity that meets the preset conditions as the pseudo queue data of the target subject.

[0076] In a possible implementation, the construction subunit includes:

[0077] The second acquisition subunit is used to acquire historical cohort data generated when L subjects perform cognitive classification as candidate cohort data; L is a positive integer greater than 1;

[0078] The selection subunit is used to select candidate cohort data that meets preset matching conditions from the candidate cohort data by using the pseudo cohort data selection model as the pseudo cohort data of the target subject.

[0079] In a possible implementation, when the target subject performs cognitive classification for the Nth time, where N is a positive integer greater than 1, the classification unit is specifically configured to:

[0080] Inputting the characteristic data of the target subject into a pre-constructed second cognitive classification model to determine a cognitive classification result of the target subject;

[0081] Among them, the second cognitive classification model is constructed on the basis of a pre-constructed cross-sectional data classification model, using the characteristic data of the previous N-1 historical cohort data of the target subject; the cross-sectional data classification model is constructed using the characteristic data generated by the most recent cognitive classification of M subjects collected in advance.

[0082] In a possible implementation, the device further includes:

[0083] The second acquisition unit is used to acquire sample process data and sample result data generated by the sample subjects when performing cognitive classification;

[0084] A second extraction unit, used to extract characteristic data of the sample subject from the sample process data and the sample result data;

[0085] A training unit, used to train the initial cross-sectional data classification model according to the characteristic data of the sample subjects and the cognitive classification identification labels corresponding to the sample subjects, so as to generate the cross-sectional data classification model;

[0086] The initial cross-sectional data classification model includes an input embedding layer, an encoding layer, and a decoding layer.

[0087] In a possible implementation, when the target subject is subjected to cognitive classification for the first time and it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, the device further includes:

[0088] A first input unit, used to input the characteristic data of the target subject into an Input Embedding layer of the cross-sectional data classification model to obtain a first low-dimensional feature vector;

[0089] A second input unit is used to input the first low-dimensional feature vector into a pre-constructed first rehabilitation training program recommendation model to obtain a rehabilitation training program recommended to the target subject;

[0090] Among them, the first rehabilitation training program recommendation model is constructed using the first low-dimensional feature vector corresponding to the cross-sectional data generated by the most recent cognitive classification of P subjects collected in advance; P is a positive integer greater than 1.

[0091] In a possible implementation, when the target subject is subjected to cognitive classification for the Nth time and it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, the device further includes:

[0092] a third input unit, configured to input the feature data of the target subject into an Input Embedding layer of the second cognitive classification model to obtain a second low-dimensional feature vector and a third low-dimensional feature vector;

[0093] A fourth input unit, configured to input the second low-dimensional feature vector and the third low-dimensional feature vector into a pre-constructed second rehabilitation training program recommendation model to obtain a rehabilitation training program recommended to the target subject;

[0094] Among them, the second rehabilitation training program recommendation model is constructed on the basis of the pre-constructed first rehabilitation training program recommendation model, using the second low-dimensional feature vector generated by the target subject's previous N-1 cognitive classifications; the first rehabilitation training program recommendation model is constructed using the low-dimensional feature vectors generated by the most recent cognitive classification of P subjects collected in advance.

[0095] The embodiment of the present application also provides a cognitive warning device, including: a processor, a memory, and a system bus;

[0096] The processor and the memory are connected via the system bus;

[0097] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementations of the above-mentioned cognitive early warning method.

[0098] An embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned cognitive warning method.

[0099] An embodiment of the present application also provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes any one of the implementation modes of the above-mentioned cognitive warning method.

[0100] The present application provides a cognitive warning method, device, storage medium and equipment. First, the process data and result data generated by the target subject to be classified during cognitive classification are obtained, wherein the process data includes the voice data, video data and eye movement data generated by the target subject during cognitive classification, and the result data is the electronic scale data obtained by the target subject after cognitive classification. Then, the characteristic data of the target subject is extracted from the process data and the result data, and the cognitive classification result of the target subject is determined according to the characteristic data of the target subject. Then, when it is judged that the cognitive classification result of the target subject belongs to the preset abnormal classification, the target associated user of the target subject is sent a cognitive abnormal warning prompt information. It can be seen that when the embodiment of the present application performs cognitive classification on the target subject, the voice data, video data, eye movement data and other information related to cognitive function generated by the target subject during cognitive classification are considered, so that the cognitive classification result of the target subject can be determined more accurately, and then when it is determined that there is a cognitive abnormal classification, the cognitive abnormal warning prompt information can be immediately sent to the associated user of the target subject, so as to timely improve the target subject's cognition and living standards and avoid the occurrence of dangerous situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0101] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0102] Figure 1 A flowchart of a cognitive warning method provided in an embodiment of the present application;

[0103] Figure 2 A schematic diagram of the structure of a cross-sectional data classification model provided in an embodiment of the present application;

[0104] Figure 3 A schematic diagram of the structure of the first cognitive classification model or the second cognitive classification model provided in an embodiment of the present application;

[0105] Figure 4 A schematic diagram of the structure of a first rehabilitation training program recommendation model provided in an embodiment of the present application;

[0106] Figure 5 A schematic diagram of the structure of a second rehabilitation training program recommendation model provided in an embodiment of the present application;

[0107] Figure 6 A schematic diagram of the composition of a cognitive warning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0108] In the existing classification methods of cognitive results, the basic information and test result data of the subjects are usually collected through paper quality tables, but the process information of the test cannot be collected. In the specific classification process, the classification is mainly carried out through one-on-one tests between the examiner and the subject. The examiner will give the cognitive status classification results based on the test results of the subject and give rehabilitation training program suggestions. However, this classification method not only relies on the subjective classification results of the examiner, which is subjective and inconsistent, but also mainly relies on the current test results of the subject for classification, and does not consider the data information and historical classification information generated during the test. Therefore, it is impossible to fully reflect the cognitive performance of the subject. In addition, the rehabilitation training program suggestions given by the examiner are only based on their own classification experience, which has certain limitations, resulting in low accuracy of cognitive classification results, and when determining the classification of cognitive abnormalities, corresponding preventive measures are not taken in the first time, which is not conducive to timely improving the cognition and living standards of the elderly.

[0109] To solve the above defects, the embodiment of the present application provides a cognitive warning method, first, obtain the process data and result data generated by the target subject to be classified during cognitive classification, wherein the process data includes the voice data, video data, and eye movement data generated by the target subject during cognitive classification, and the result data is the electronic scale data obtained by the target subject after cognitive classification, then, extract the characteristic data of the target subject from the process data and the result data, and determine the cognitive classification result of the target subject according to the characteristic data of the target subject, and then, when it is judged that the cognitive classification result of the target subject belongs to the preset abnormal classification, send cognitive abnormality warning prompt information to the target associated user of the target subject. It can be seen that when the embodiment of the present application performs cognitive classification on the target subject, the voice data, video data, eye movement data and other information related to cognitive function generated by the target subject during cognitive classification are considered, so that the cognitive classification result of the target subject can be determined more accurately, and then when the cognitive abnormal classification is determined, the cognitive abnormality warning prompt information can be immediately sent to the associated user of the target subject, so as to timely improve the target subject's cognition and living standards and avoid the occurrence of dangerous situations.

[0110] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0111] First embodiment

[0112] See also Figure 1 , is a flow chart of a cognitive early warning method provided in this embodiment, the method comprising the following steps:

[0113] S101: Obtaining process data and result data generated by the target subject to be classified during cognitive classification, wherein the process data includes voice data, video data, and eye movement data generated by the target subject during cognitive classification, and the result data is the electronic scale data obtained by the target subject after cognitive classification.

[0114] In this embodiment, any subject (usually an elderly person) who uses this embodiment to implement cognitive classification and early warning is referred to as a target subject. In order to accurately determine the cognitive status classification result of the target subject, and when it is determined that the cognitive classification result belongs to a preset abnormal classification, cognitive abnormality early warning prompt information is promptly sent to its associated user, and targeted rehabilitation training program suggestions are recommended to improve the elderly's cognition and living standards as soon as possible. In this embodiment, when using an electronic scale to perform cognitive classification on the target subject, it is first necessary to obtain the process data and result data generated by the target subject to be classified during cognitive classification.

[0115] Among them, the process data may include multimodal data such as voice data (such as voices emitted when answering questions from the examiner), video data, eye movement data, etc. generated by the target subject during the cognitive classification process. The result data may be electronic scale data obtained by the target subject after cognitive classification, such as the option answers selected by the target subject, memory drawing scores, etc. Then, the acquired process data and result data may be intelligently analyzed and processed in the subsequent step S102, so as to further determine the cognitive status classification result of the target subject based on the processing results.

[0116] S102: Extract characteristic data of the target subject from the process data and result data.

[0117] In this embodiment, after obtaining the process data and result data generated by the target subject during cognitive classification through step S101, the existing or future feature extraction methods can be further used to extract features from these process data and result data, that is, feature extraction is performed on the voice data, video data, eye movement data and electronic scale data generated by the target subject during cognitive classification to generate feature data that can characterize these process data and result data as the feature data of the target subject, which is used to execute the subsequent step S103 to determine the classification result of the cognitive status of the target subject.

[0118] Specifically, an optional implementation manner is that the specific implementation process of step S102 may include the following steps S1021-S1025:

[0119] S1021: Extracting audio features of the speech data from the speech data.

[0120] In this implementation, the acquired voice data generated by the target subject during cognitive classification may include but is not limited to the voice of the target subject answering cognitive classification questions asked by the examiner, or other random voices spontaneously generated during the answering process that are unrelated to the cognitive classification questions.

[0121] It should be noted that the present application does not limit the method for acquiring the target subject's voice data. For example, a built-in or external sound receiving device can be used to collect the conversational voice made by the target subject during the cognitive classification process. For example, a built-in sound receiving device such as a microphone of a computer, tablet or mobile phone used in the cognitive classification process can be used for collection, or an external sound receiving device such as a voice recorder or a clip-on microphone can be used for collection.

[0122] On this basis, existing or future speech feature extraction methods can be used to perform feature extraction on the acquired target subject speech data. Specifically, an optional implementation method is to extract acoustic features, linguistic features, duration features, and spectrogram image features from the target subject speech data frame by frame (such as 0.01s), and after calculating the statistics (including maximum value, minimum value, average value, etc.), perform feature splicing to obtain the audio features of the speech data, and set it to n1 dimension.

[0123] The acoustic features of the target subject's speech data include, but are not limited to, Mel Frequency Cepstral Coefficents (MFCCs), FBank (Filter bank) features, etc. The linguistic features of the target subject's speech data include, but are not limited to, vocabulary, syntax, etc. The duration features of the target subject's speech data include, but are not limited to, the target subject's speaking duration and pause duration during cognitive classification.

[0124] S1022: Extract video features of the video data from the video data.

[0125] In this implementation, the video data generated by the target subject when performing cognitive classification may include but is not limited to the target subject's mouth, facial expressions and other body movement information collected by multi-angle camera equipment during the test, and then feature extraction can be performed on the video data to generate video features corresponding to the video data.

[0126] It should be noted that the present application does not limit the method for acquiring the video data of the target subject. For example, a built-in or external camera device can be used to capture the actions, expressions, etc. produced by the target subject during the cognitive classification process. For example, a built-in camera device such as a camera on a computer, tablet or mobile phone used in the cognitive classification process can be used to collect the data.

[0127] On this basis, existing or future video feature extraction methods can be used to extract features from the acquired target subject video data. Specifically, an optional implementation method is to extract lip shape features and facial expressions from the target subject video data frame by frame (such as 0.01s) to characterize the dynamic changes of lip shape and facial expressions (such as changes in facial expressions such as joy, anger, sadness, and happiness), and perform feature splicing to obtain video features of the video data, and set it to n2 dimensions.

[0128] S1023: Extracting eye movement features of the eye movement data from the eye movement data.

[0129] In this implementation, the eye movement data generated by the target subject when performing cognitive classification may include but is not limited to the target subject's eye movement information collected by multi-angle camera equipment during the test, and then feature extraction can be performed on the eye movement data to generate eye movement features corresponding to the eye movement data.

[0130] It should be noted that the present application does not limit the method for acquiring the target subject's eye movement data. For example, a built-in or external camera device can be used to capture the target subject's eye gaze area and duration during the cognitive classification process. For example, a built-in camera device such as a camera on a computer, tablet or mobile phone used in the cognitive classification process can be used to collect data, and a timer can be used for timing.

[0131] On this basis, existing or future feature extraction methods can be used to extract features from the acquired target subject's eye movement data. Specifically, an optional implementation method is to extract the target subject's gaze duration in different areas such as up, down, left, and right, the jump frequency in different areas, and the movement trajectory characteristics of the eye gaze point from the target subject's eye movement data to characterize the changes in eye attention, and perform feature splicing to obtain the eye movement features of the eye movement data, and set it to n3 dimensions.

[0132] S1024: Extracting scale test features of the electronic scale data from the electronic scale data.

[0133] In this implementation, the electronic scale data obtained after cognitive classification of the target subject may include but is not limited to the test scores of the target subject on neuropsychological scales such as MoCAB, MMSE, IADL, HAMA, HAMD, clock drawing test, and picture description during the test, and then feature extraction can be performed on these electronic scale data. Specifically, an optional implementation method is to extract the test scores of each neuropsychological scale from the electronic scale data of the target subject, and splice these scores as scale test features of the electronic scale data, and set them to n4 dimensions.

[0134] It should be noted that this embodiment does not limit the execution order of S1021, S1022, S1023 and S1024, and these four steps can be executed sequentially or simultaneously.

[0135] S1025: Concatenate audio features, video features, eye movement features, and scale test features to obtain feature data of the target subject.

[0136] In this implementation, after obtaining audio features (n1 dimension), video features (n2 dimension), eye movement features (n3 dimension), and scale test features (n4 dimension) through steps S1021-S1024, the four can be further spliced, for example, the four can be directly spliced ​​without any processing, or the four can be weighted spliced, etc., and the vector data obtained after splicing is used as the feature data of the target subject, and it is defined as Fea_all and set to n dimension. It can be seen that if the feature data of the target subject is obtained by directly splicing the audio features (n1 dimension), video features (n2 dimension), eye movement features (n3 dimension), and scale test features (n4 dimension), then n=n1+n2+n3+n4.

[0137] S103: Determine the cognitive classification result of the target subject according to the characteristic data of the target subject.

[0138] In this embodiment, after extracting the characteristic data Fea_all of the target subject from the process data and the result data in step S102, the characteristic data Fea_all can be further processed, and the cognitive classification result of the target subject can be determined according to the processing result. Among them, the cognitive classification result of the target subject can include the normal classification or abnormal classification of the target subject's cognition, and the abnormal classification can be further divided into language abnormality, writing abnormality, eye movement abnormality, etc. according to the actual situation. The specific classification type is not limited in this application.

[0139] In a possible implementation of the embodiment of the present application, when it is determined that the target subject is undergoing cognitive classification for the first time, the implementation process of step S103 may specifically include the following steps AB:

[0140] Step A: Use the characteristic data of the target subjects to construct pseudo-cohort data.

[0141] In this implementation, in order to improve the accuracy of the cognitive classification of the target subject, this embodiment uses the target subject's cohort data (i.e., the process data and result data generated when the target subject was classified in the past) and cross-sectional data (i.e., the process data and result data generated when all subjects were classified at the same classification time) to comprehensively classify the target subject's cognitive status using machine learning methods, thereby obtaining a more accurate classification result. Therefore, when the target subject is cognitively classified for the first time, since it has no historical cohort data, it is first necessary to use the target subject's characteristic data to construct pseudo cohort data. The specific construction methods include the following two methods:

[0142] The first method for constructing pseudo-cohort data is: first, obtain historical cohort data generated when K (K is a positive integer greater than 1) subjects were previously cognitively classified as candidate cohort data; then, use existing or future similarity calculation methods to calculate the similarity between the target subject's feature data Fea_all and the feature data of all candidate cohort data, such as using Euclidean distance or cosine distance to calculate the similarity between the target subject's feature data Fea_all and the feature data of each candidate cohort data, and use the candidate cohort data corresponding to the similarity that meets the preset conditions as the pseudo-cohort data of the target subject.

[0143] Among them, the specific content of the preset conditions can be set according to actual conditions, and the embodiments of the present application do not limit this. For example, the preset conditions can be set to a similarity value greater than 0.8, and the candidate queue data whose feature data has a similarity greater than 0.8 with the feature data Fea_all of the target subject can be used as the pseudo queue data of the target subject.

[0144] The second method for constructing pseudo-cohort data is: first, obtain historical cohort data generated when L (L is a positive integer greater than 1) subjects were previously cognitively classified as candidate cohort data; then, use the trained pseudo-cohort data selection model to select candidate cohort data that meets preset matching conditions (the specific content can be set according to actual conditions, and the embodiment of the present application is not limited to this) from all candidate cohort data as pseudo-cohort data for the target subject.

[0145] Among them, the pseudo-cohort data selection model is obtained by training using a binary classification model. The input of the model is the characteristic data Fea_all of the target subject and the characteristic data of any candidate cohort data. The goal is to predict whether these two pieces of data belong to different test data (cohort data) of the same subject, and use the label_smoothing strategy to train the model. For the target subject who has only collected cross-sectional data once, the probability that the characteristic data Fea_all of this test and the characteristic data of a candidate cohort data of a certain subject belong to the same cohort data is calculated. If this is greater than a preset threshold (the specific content can be set according to actual conditions, and the embodiment of the present application does not limit this, for example, it can be set to 0.85, etc.), it is considered that the test data corresponding to the characteristic data Fea_all of this test and the characteristic data of a candidate cohort data of a certain subject belong to the same pseudo-cohort data.

[0146] Next, after constructing the pseudo-cohort data using the characteristic data of the target subjects, the characteristic data of the pseudo-cohort data can be further used to construct a first cognitive classification model based on the pre-constructed cross-sectional data classification model to execute the subsequent step B to achieve accurate classification of the target subjects.

[0147] Among them, an optional implementation method is that the construction process of the cross-sectional data classification model can specifically include the following steps (1)-(3):

[0148] Step (1): Obtain sample process data and sample result data generated by sample subjects when performing cognitive classification.

[0149] In this implementation, in order to construct a cross-sectional data classification model, a lot of preparatory work needs to be done in advance. First, a large amount of process data and result data generated by subjects using electronic scales during cognitive classification needs to be collected at the same time, and used as sample process data and sample result data corresponding to sample subjects to form model training data (i.e., cross-sectional training data). And the classification type of the cognitive status of these sample subjects is manually marked in advance, i.e., normal or abnormal (can be specific to language abnormality, writing abnormality, eye movement abnormality, etc.), to train the cross-sectional data classification model.

[0150] Step (2): Extract characteristic data of sample subjects from sample process data and sample result data.

[0151] After obtaining the sample process data and sample result data generated by the sample subjects during cognitive classification through step (1), they cannot be directly used to train and generate a cross-sectional data classification model. Instead, it is necessary to extract the characteristic data of the sample subjects from the sample process data and the sample result data. The specific extraction process can be found in the above steps S1021-S1025. It is only necessary to replace the data of the target subjects with the data of the sample subjects, which will not be repeated here. The extracted characteristic data of the sample subjects can then be used to train a cross-sectional data classification model.

[0152] Step (3): Train the initial cross-sectional data classification model based on the characteristic data of the sample subjects and the cognitive classification identification labels corresponding to the sample subjects to generate a cross-sectional data classification model.

[0153] In this embodiment, after the characteristic data of the sample subject is extracted through step (2), the initial cross-sectional data classification model can be further trained according to the characteristic data of the sample subject and the classification identification label results corresponding to the subject, thereby generating a cross-sectional data classification model.

[0154] Among them, Figure 2 As shown, the initial cross-sectional data classification model may include an input embedding layer InputEmbedding, an encoding layer Encoder, a decoding layer Decoder, and the like.

[0155] Specifically, the characteristic data of each sample subject can be used as input data (i.e. Figure 2 The “cross-sectional data” of the InputEmbedding layer is input into the input embedding layer, and the cognitive classification result (normal or abnormal (can be specific to language abnormality, writing abnormality, eye movement abnormality, etc.)) corresponding to each sample subject is used as the actual classification result. Figure 2 The initially constructed cross-sectional data classification model shown in the figure is trained. During the training process, the cross-sectional data classification model can predict and output the cognitive classification results of the sample subjects based on the input characteristic data of the sample subjects, and compare the predicted classification results with their actual classification results. The parameters of the cross-sectional data classification model are updated according to the comparison results, and the final cross-sectional data classification model is obtained through multiple rounds of training.

[0156] Furthermore, after completing Figure 2 After training the cross-sectional data classification model shown, the model structure can be adjusted to Figure 3 As shown, as the first cognitive classification model (or the second cognitive classification model mentioned later), and Figure 2 The model parameters of the Input Embedding layer and Encoder layer in are directly used as Figure 3 The Input Embedding2 layer and Encoder2 layer of the first cognitive classification model are shown, and the initial parameters of the Input Embedding1 layer and Encoder1 layer are also the same as those of the Input Embedding2 layer and Encoder2 layer, respectively.

[0157] Then, the model parameters of the Input Embedding1 layer and the Encoder1 layer are fine-tuned using the feature data of the pseudo queue data (the extraction method can refer to the above steps S1021-1025), and some parameters of the Input Embedding2, Encoder2 and Decoder layers are kept unchanged. That is, the feature data of each pseudo queue data can be used as input data (i.e. Figure 3 The “pseudo-cohort data” of the Input Embedding1 layer is input into the , and the cognitive classification result (normal or abnormal (can also be specific to language abnormality, handwriting abnormality, eye movement abnormality, etc.)) corresponding to the original subject of each pseudo-cohort data is used as the actual cognitive classification result. Figure 3 The first cognitive classification model shown is trained. During the training process, the first cognitive classification model can predict and output the cognitive status corresponding to the original subject based on the feature data of the input pseudo-cohort data, and compare the predicted classification result with the actual classification result. According to the comparison result, the parameters of the first cognitive classification model are updated, and the final first cognitive classification model is obtained through multiple rounds of training.

[0158] Step B: Input the characteristic data of the target subject into the first cognitive classification model to determine the cognitive classification result of the target subject.

[0159] In this implementation, based on the pre-constructed cross-sectional data classification model, the characteristic data of the pseudo-cohort data constructed in step A is used to train the following Figure 3 After the first cognitive classification model shown, the target subject's feature data Fea_all (including audio features, video features, eye movement features, and scale test features) can be further input into the first cognitive classification model to determine a more accurate cognitive classification result of the target subject. For example, the first cognitive classification model can be used to output a binary classification value of 0 or 1, where 1 indicates that the target subject's cognitive status classification result is normal; 0 indicates that the target subject's cognitive status is abnormal.

[0160] In another possible implementation of the embodiment of the present application, when it is determined that the target subject is performing cognitive classification for the Nth time (N is a positive integer greater than 1), that is, when the target subject is not performing cognitive classification for the first time, the implementation process of the above step S103 may specifically include: inputting the characteristic data of the target subject into a pre-constructed second cognitive classification model to determine the cognitive classification result of the target subject.

[0161] The second cognitive classification model is constructed based on the pre-constructed cross-sectional data classification model using the characteristic data of the target subject's previous N-1 historical cohort data. The cross-sectional data classification model is still constructed through the above steps (1)-(3). Figure 2 The classification model described.

[0162] It should be noted that the construction method of the second cognitive classification model is similar to that of the first cognitive classification model, and even the composition structure of the model is consistent. Figure 3 As shown in , only the specific model parameters and training data are different. Specifically, after completing Figure 2 After training the cross-sectional data classification model shown in the figure, the model structure is adjusted to Figure 3 As shown in the second cognitive classification model, Figure 2 The model parameters of the Input Embedding layer and Encoder layer in are directly used as Figure 3 The Input Embedding2 layer and Encoder2 layer of the second cognitive classification model are shown, and the initial parameters of the Input Embedding1 layer and Encoder1 layer are also the same as those of the Input Embedding2 layer and Encoder2 layer, respectively.

[0163] Then, the model parameters of the Input Embedding1 layer and the Encoder1 layer are fine-tuned using the feature data of the target subject's previous N-1 historical cohort data (the extraction method can refer to the above steps S1021-1025), and some parameters of the Input Embedding2, Encoder2, and Decoder layers are kept unchanged. That is, the feature data of each historical cohort data can be used as input data (i.e. Figure 3 The “queue data” of the Input Embedding1 layer is input into the , and the cognitive classification result (normal or abnormal (can also be specific to language abnormality, handwriting abnormality, eye movement abnormality, etc.)) corresponding to each target subject is used as the actual classification result. Figure 3The second cognitive classification model shown is trained. During the training process, the second cognitive classification model can predict the cognitive status corresponding to the output target subject based on the characteristic data of the historical cohort data of each input target subject, and compare the predicted classification result with the actual classification result. According to the comparison result, the parameters of the second cognitive classification model are updated, and the final second cognitive classification model is obtained through multiple rounds of training.

[0164] S104: When it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, a cognitive abnormality warning prompt message is sent to a user associated with the target subject.

[0165] In this embodiment, when it is determined through step S103 that the cognitive classification result of the target subject belongs to a preset abnormal classification, for example, when it is determined that the cognitive classification result corresponding to the target subject is language abnormality, it indicates that corresponding measures need to be taken as soon as possible to improve the language ability of the target subject so as to improve the living standard of the target subject. Specifically, it is further necessary to send warning information that the cognitive classification result of the target subject belongs to an abnormal classification to the associated users of the target subject (such as the family members of the target subject or other friends, guardians, etc. who have an associated relationship with the target subject), so that the associated users can adopt the corresponding solutions as soon as possible to improve the cognitive situation and living standard of the target subject as soon as possible.

[0166] Among them, the specific sending method of the early warning prompt information and the form of the prompt message content can be set according to the actual situation. For example, a text and / or picture early warning prompt SMS, MMS, instant messaging software message or push information can be sent to the mobile phone of the target subject's daughter to remind her that the target subject's cognitive classification result has fallen into the preset abnormal classification and that corresponding solutions and protection measures need to be taken in time; or, it can also be in the form of an automatic intelligent voice call to the target subject's daughter to remind the target subject that the cognitive classification result has fallen into the preset abnormal classification and that corresponding solutions and protection measures need to be taken in time; or, it can also be in the form of an automatic intelligent voice call to the target subject's daughter to remind her that the target subject's cognitive classification result has fallen into the preset abnormal classification and that corresponding solutions and protection measures need to be taken in time.

[0167] It should be noted that the specific values ​​of the preset abnormality categories can be set according to actual conditions, and the embodiments of the present application do not limit this. For example, the preset abnormality categories can be divided into language abnormalities, writing abnormalities, eye movement abnormalities, etc.

[0168] Furthermore, when the target subject is cognitively classified for the first time, and the above steps S101-S103 determine that the cognitive status of the target subject belongs to a preset abnormal classification (such as language abnormality, writing abnormality, eye movement abnormality, etc.), in order to improve the cognitive status of the target subject more targeted, targeted rehabilitation training program suggestions can be automatically provided to the target subject, so as to improve the cognitive status and living standards of the target subject through targeted rehabilitation training in a timely manner. An optional implementation method is to first input the characteristic data Fea_all (including audio features, video features, eye movement features, and scale test features) of the target subject into Figure 2 The Input Embedding layer of the cross-sectional data classification model shown in Figure 1 gives Figure 2 The low-dimensional feature vector Embedding Vector in the dotted box is defined as the first low-dimensional feature vector, and then input into Figure 4 The pre-built first rehabilitation training program recommendation model shown in the figure obtains the rehabilitation training program recommended to the target subject. Among them, the type of rehabilitation training program can be set according to the actual situation, and the embodiment of the present application does not limit this. For example, the type of rehabilitation training program can be set to language rehabilitation, nutritional rehabilitation, writing training, etc.

[0169] Among them, the network structure of the first rehabilitation training program recommendation model is as follows Figure 4 As shown, it includes an encoding layer Encoder and a decoding layer Decoder, and is constructed using the first low-dimensional feature vector corresponding to the cross-sectional data generated by P (P is a positive integer greater than 1) subjects who were previously collected during the most recent cognitive classification.

[0170] Specifically, the first low-dimensional feature vector (i.e. Figure 2 The low-dimensional feature vector Embedding Vector in the dotted box is used as input data (i.e. Figure 4 The “Embedding Vector” of the Encoder layer is input into the Encoder layer, and the rehabilitation training program corresponding to each sample subject (such as sports rehabilitation, nutritional rehabilitation or cognitive training, etc.) is used as the actual recommendation result. Figure 4 The first rehabilitation training program recommendation model shown is trained. During the training process, the first rehabilitation training program recommendation model can predict and output the type of targeted rehabilitation training program recommended for the sample subject based on the input sample subject's Embedding Vector, and compare the recommendation result with its actual recommendation result. According to the comparison result, the parameters of the first rehabilitation training program are updated, and the final first rehabilitation training program recommendation model is obtained through multiple rounds of training.

[0171] Alternatively, when the target subject is cognitively classified for the Nth time, and it is determined through the above steps S101-S103 that the cognitive classification result of the target subject belongs to the preset abnormal classification, in order to improve the cognitive condition of the target subject more specifically, a targeted rehabilitation training program suggestion can also be automatically provided to the target subject, so as to improve the cognitive condition and living standard of the target subject through targeted rehabilitation training in a timely manner. An optional implementation method is to first input the characteristic data Fea_all (including audio features, video features, eye movement features, and scale test features) of the target subject into Figure 3 The Input Embedding layer of the second cognitive classification model shown in Figure 3 The two low-dimensional feature vectors Embedding Vector1 and Embedding Vector2 in the dotted box are defined as the second low-dimensional feature vector and the third low-dimensional feature vector respectively, and then the two are input as follows Figure 5 The pre-built second rehabilitation training program recommendation model shown is used to obtain a rehabilitation training program recommended to the target subject. The type of rehabilitation training program can still be set according to actual conditions, and the embodiment of the present application does not limit this. For example, the type of rehabilitation training program can be set to language rehabilitation, nutritional rehabilitation, writing training, etc.

[0172] Among them, the network structure of the second rehabilitation training program recommendation model is as follows Figure 5 As shown, it includes two encoding layers Encoder1 and Encoder2 and a decoding layer Decoder, which is pre-built as Figure 4 The first rehabilitation training program recommendation model shown is constructed based on the second low-dimensional feature vector and the third low-dimensional feature vector generated when the target subject was previously classified N-1 times.

[0173] Specifically, after completing Figure 4 After training the first rehabilitation training program recommendation model shown in FIG. 1 , the model structure is adjusted to be as follows: Figure 5 As shown, it is recommended as the second rehabilitation training program model, and Figure 4 The model parameters of the Encoder layer in are directly used as Figure 3 The second rehabilitation training scheme shown recommends the Encoder2 layer of the model, and the initial parameters of the Encoder1 layer are also the same as those of the Encoder2 layer.

[0174] Then, the second low-dimensional feature vector generated by the target subject's previous N-1 cognitive classifications is used (i.e. Figure 3The model parameters of the Encoder1 layer are fine-tuned by using the Embedding Vector1 in the dotted box, and some parameters of the Encoder2 and Decoder layers remain unchanged. That is, the second low-dimensional feature vector generated by each cognitive classification performed by the target subject can be used as input data (i.e. Figure 5 The “Embedding Vector1” of the Encoder1 layer is input into the Encoder2 layer, and the rehabilitation training program corresponding to each target subject (such as language rehabilitation, nutritional rehabilitation, or writing training, etc.) is used as the actual classification result. Figure 5 The second rehabilitation training program recommendation model shown is trained. During the training process, the second rehabilitation training program recommendation model can predict and output the type of targeted rehabilitation training program recommended for the target subject based on the Embedding Vector1 generated by each cognitive classification of the input target subject, and compare the recommendation result with its actual recommendation result. According to the comparison result, the parameters of the second rehabilitation training program recommendation model are updated, and the final second rehabilitation training program recommendation model is obtained through multiple rounds of training.

[0175] In summary, the present embodiment provides a cognitive warning method, first, obtain the process data and result data generated by the target subject to be classified during cognitive classification, wherein the process data includes the voice data, video data, and eye movement data generated by the target subject during cognitive classification, and the result data is the electronic scale data obtained by the target subject after cognitive classification, then, extract the characteristic data of the target subject from the process data and the result data, and determine the cognitive classification result of the target subject according to the characteristic data of the target subject, and then, when it is judged that the cognitive classification result of the target subject belongs to the preset abnormal classification, send cognitive abnormality warning prompt information to the target associated user of the target subject. It can be seen that when the embodiment of the present application performs cognitive classification on the target subject, the voice data, video data, eye movement data and other information related to cognitive function generated by the target subject during cognitive classification are considered, so that the cognitive classification result of the target subject can be determined more accurately, and then when it is judged that there is a cognitive abnormal classification, the cognitive abnormality warning prompt information can be immediately sent to the associated user of the target subject, so as to timely improve the target subject's cognition and living standards and avoid the occurrence of dangerous situations.

[0176] Second embodiment

[0177] This embodiment will introduce a cognitive warning device. For related content, please refer to the above method embodiment.

[0178] See also Figure 6, is a schematic diagram of the composition of a cognitive warning device provided in this embodiment, the device 600 includes:

[0179] The first acquisition unit 601 is used to acquire process data and result data generated by the target subject to be classified during cognitive classification, wherein the process data includes voice data, video data, and eye movement data generated by the target subject during cognitive classification, and the result data is electronic scale data obtained by the target subject after cognitive classification;

[0180] A first extraction unit 602 is used to extract characteristic data of the target subject from the process data and result data;

[0181] A classification unit 603, used to determine a cognitive classification result of the target subject according to the characteristic data of the target subject;

[0182] The warning unit 604 is used to send cognitive abnormality warning prompt information to the target associated user of the target subject when it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification.

[0183] In an implementation of this embodiment, the first extracting unit 602 includes:

[0184] A first extraction subunit, configured to extract audio features of the speech data from the speech data;

[0185] A second extraction subunit, configured to extract video features of the video data from the video data;

[0186] A third extraction subunit, configured to extract eye movement features of the eye movement data from the eye movement data;

[0187] A fourth extraction subunit, configured to extract a scale test feature of the electronic scale data from the electronic scale data;

[0188] The splicing subunit is used to splice the audio features, the video features, the eye movement features and the scale test features to obtain the feature data of the target subject.

[0189] In an implementation of this embodiment, the first extraction subunit is specifically used for:

[0190] Acoustic features, linguistic features, duration features, and spectrogram image features are extracted from the speech data and concatenated to obtain audio features of the speech data.

[0191] In an implementation of this embodiment, the second extraction subunit is specifically used for:

[0192] Lip shape features, facial expressions and other features are extracted from the video data and spliced ​​to obtain video features of the video data.

[0193] In an implementation of this embodiment, the third extraction subunit is specifically used for:

[0194] The gaze duration of different areas, the jump frequency of different areas, and the movement trajectory characteristics of the eye gaze point are extracted from the eye movement data and spliced ​​to obtain the eye movement characteristics of the eye movement data.

[0195] In an implementation of this embodiment, the fourth extraction subunit is specifically used for:

[0196] The test scores of each item in the neuropsychology scale are extracted from the electronic scale data and concatenated to obtain the scale test features of the electronic scale data.

[0197] In one implementation of this embodiment, when the target subject is cognitively classified for the first time, the classification unit 603 includes:

[0198] A construction subunit, used to construct pseudo-cohort data using the characteristic data of the target subject;

[0199] A classification subunit, configured to input the characteristic data of the target subject into a first cognitive classification model to determine a cognitive classification result of the target subject;

[0200] Among them, the first cognitive classification model is constructed on the basis of a pre-constructed cross-sectional data classification model using the characteristic data of the pseudo-cohort data; the cross-sectional data classification model is constructed using the characteristic data generated by the most recent cognitive classification of M subjects collected in advance; and M is a positive integer greater than 1.

[0201] In one implementation of this embodiment, the construction subunit includes:

[0202] A first acquisition subunit is used to acquire historical cohort data generated when K subjects were previously cognitively classified as candidate cohort data; K is a positive integer greater than 1;

[0203] The calculation subunit is used to calculate the similarity between the characteristic data of the target subject and the characteristic data of all the candidate queue data, and use the candidate queue data corresponding to the similarity that meets the preset conditions as the pseudo queue data of the target subject.

[0204] In one implementation of this embodiment, the construction subunit includes:

[0205] The second acquisition subunit is used to acquire historical cohort data generated when L subjects perform cognitive classification as candidate cohort data; L is a positive integer greater than 1;

[0206] The selection subunit is used to select candidate cohort data that meets preset matching conditions from the candidate cohort data by using the pseudo cohort data selection model as the pseudo cohort data of the target subject.

[0207] In one implementation of this embodiment, when the target subject performs cognitive classification for the Nth time, where N is a positive integer greater than 1, the classification unit 603 is specifically configured to:

[0208] Inputting the characteristic data of the target subject into a pre-constructed second cognitive classification model to determine a cognitive classification result of the target subject;

[0209] Among them, the second cognitive classification model is constructed on the basis of a pre-constructed cross-sectional data classification model, using the characteristic data of the previous N-1 historical cohort data of the target subject; the cross-sectional data classification model is constructed using the characteristic data generated by the most recent cognitive classification of M subjects collected in advance.

[0210] In an implementation of this embodiment, the device further includes:

[0211] The second acquisition unit is used to acquire sample process data and sample result data generated by the sample subjects when performing cognitive classification;

[0212] A second extraction unit, used to extract characteristic data of the sample subject from the sample process data and the sample result data;

[0213] A training unit, used to train the initial cross-sectional data classification model according to the characteristic data of the sample subjects and the cognitive classification identification labels corresponding to the sample subjects, so as to generate the cross-sectional data classification model;

[0214] The initial cross-sectional data classification model includes an input embedding layer, an encoding layer, and a decoding layer.

[0215] In one implementation of this embodiment, when the target subject is subjected to cognitive classification for the first time and it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, the device further includes:

[0216] A first input unit, used to input the characteristic data of the target subject into an Input Embedding layer of the cross-sectional data classification model to obtain a first low-dimensional feature vector;

[0217] A second input unit is used to input the first low-dimensional feature vector into a pre-constructed first rehabilitation training program recommendation model to obtain a rehabilitation training program recommended to the target subject;

[0218] Among them, the first rehabilitation training program recommendation model is constructed using the first low-dimensional feature vector corresponding to the cross-sectional data generated by the most recent cognitive classification of P subjects collected in advance; P is a positive integer greater than 1.

[0219] In one implementation of this embodiment, when the target subject is subjected to cognitive classification for the Nth time and it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, the device further includes:

[0220] a third input unit, configured to input the feature data of the target subject into an Input Embedding layer of the second cognitive classification model to obtain a second low-dimensional feature vector and a third low-dimensional feature vector;

[0221] A fourth input unit, configured to input the second low-dimensional feature vector and the third low-dimensional feature vector into a pre-constructed second rehabilitation training program recommendation model to obtain a rehabilitation training program recommended to the target subject;

[0222] Among them, the second rehabilitation training program recommendation model is constructed on the basis of the pre-constructed first rehabilitation training program recommendation model, using the second low-dimensional feature vector generated by the target subject's previous N-1 cognitive classifications; the first rehabilitation training program recommendation model is constructed using the low-dimensional feature vectors generated by the most recent cognitive classification of P subjects collected in advance.

[0223] Furthermore, the embodiment of the present application also provides a cognitive warning device, including: a processor, a memory, and a system bus;

[0224] The processor and the memory are connected via the system bus;

[0225] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any one of the implementation methods of the above-mentioned cognitive early warning method.

[0226] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned cognitive warning method.

[0227] Furthermore, an embodiment of the present application also provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute any one of the implementation methods of the above-mentioned cognitive warning method.

[0228] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0229] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0230] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0231] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A cognitive early warning method, characterized in that: include: Acquire process data and result data generated by the target subject to be classified during cognitive classification, wherein the process data includes voice data, video data, and eye movement data generated by the target subject during cognitive classification, and the result data is electronic scale data obtained by the target subject after cognitive classification; Extracting characteristic data of the target subject from the process data and result data; Determining a cognitive classification result of the target subject according to the characteristic data of the target subject; When it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, a cognitive abnormality warning prompt message is sent to a user associated with the target subject; The target subject is cognitively classified for the first time; and determining the cognitive classification result of the target subject according to the characteristic data of the target subject includes: Using the characteristic data of the target subject, constructing pseudo-cohort data; Inputting the characteristic data of the target subject into a first cognitive classification model to determine a cognitive classification result of the target subject; Wherein, the first cognitive classification model is constructed based on the pre-constructed cross-sectional data classification model using the characteristic data of the pseudo-cohort data; the cross-sectional data classification model is constructed using the characteristic data generated by the most recent cognitive classification of M subjects collected in advance; M is a positive integer greater than 1; The step of constructing pseudo-cohort data by using the characteristic data of the target subject includes: Acquire historical cohort data generated when K subjects were previously cognitively classified as candidate cohort data; wherein K is a positive integer greater than 1; Calculating the similarity between the characteristic data of the target subject and the characteristic data of all the candidate cohort data, and using the candidate cohort data corresponding to the similarity that meets the preset conditions as the pseudo cohort data of the target subject; Alternatively, the use of the characteristic data of the target subject to construct pseudo-cohort data includes: Acquire historical cohort data generated when L subjects perform cognitive classification as candidate cohort data; L is a positive integer greater than 1; The pseudo-cohort data selection model is used to select candidate cohort data that meet preset matching conditions from the candidate cohort data as the pseudo-cohort data of the target subject.

2. The method according to claim 1, characterized in that The step of extracting characteristic data of the target subject from the process data and result data includes: extracting audio features of the speech data from the speech data; extracting video features of the video data from the video data; extracting eye movement features of the eye movement data from the eye movement data; extracting a scale test feature of the electronic scale data from the electronic scale data; The audio features, the video features, the eye movement features and the scale test features are spliced ​​to obtain feature data of the target subject.

3. The method according to claim 2, characterized in that The extracting the audio features of the speech data from the speech data comprises: Acoustic features, linguistic features, duration features, and spectrogram image features are extracted from the speech data and concatenated to obtain audio features of the speech data.

4. The method according to claim 2, characterized in that: The extracting the video features of the video data from the video data comprises: Lip shape features and expression features are extracted from the video data and spliced ​​to obtain video features of the video data.

5. The method according to claim 2, characterized in that: The step of extracting eye movement features of the eye movement data from the eye movement data comprises: The gaze duration of different areas, the jump frequency of different areas, and the movement trajectory characteristics of the eye gaze point are extracted from the eye movement data and spliced ​​to obtain the eye movement characteristics of the eye movement data.

6. The method according to claim 2, characterized in that The step of extracting the scale test features of the electronic scale data from the electronic scale data includes: The test scores of each item in the neuropsychology scale are extracted from the electronic scale data and concatenated to obtain the scale test features of the electronic scale data.

7. The method according to claim 1, characterized in that When the target subject is cognitively classified for the Nth time, wherein N is a positive integer greater than 1, determining the cognitive classification result of the target subject according to the characteristic data of the target subject includes: Inputting the characteristic data of the target subject into a pre-constructed second cognitive classification model to determine a cognitive classification result of the target subject; Among them, the second cognitive classification model is constructed on the basis of a pre-constructed cross-sectional data classification model, using the characteristic data of the previous N-1 historical cohort data of the target subject; the cross-sectional data classification model is constructed using the characteristic data generated by the most recent cognitive classification of M subjects collected in advance.

8. The method according to claim 7, characterized in that The cross-sectional data classification model is constructed as follows: Obtaining sample process data and sample result data generated by sample subjects when they perform cognitive classification; Extracting characteristic data of the sample subject from the sample process data and the sample result data; Training the initial cross-sectional data classification model according to the characteristic data of the sample subjects and the cognitive classification identification labels corresponding to the sample subjects to generate the cross-sectional data classification model; The initial cross-sectional data classification model includes an input embedding layer, an encoding layer, and a decoding layer.

9. The method according to claim 8, characterized in that When the target subject is subjected to cognitive classification for the first time, and it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, the method further includes: Inputting the characteristic data of the target subject into the Input Embedding layer of the cross-sectional data classification model to obtain a first low-dimensional feature vector; Inputting the first low-dimensional feature vector into a pre-constructed first rehabilitation training program recommendation model to obtain a rehabilitation training program recommended to the target subject; Among them, the first rehabilitation training program recommendation model is constructed using the first low-dimensional feature vector corresponding to the cross-sectional data generated by the most recent cognitive classification of P subjects collected in advance; P is a positive integer greater than 1.

10. The method according to claim 8, characterized in that When the target subject is subjected to cognitive classification for the Nth time, and it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification, the method further includes: Inputting the feature data of the target subject into the Input Embedding layer of the second cognitive classification model to obtain a second low-dimensional feature vector and a third low-dimensional feature vector; Inputting the second low-dimensional feature vector and the third low-dimensional feature vector into a pre-constructed second rehabilitation training program recommendation model to obtain a rehabilitation training program recommended to the target subject; Among them, the second rehabilitation training program recommendation model is constructed on the basis of the pre-constructed first rehabilitation training program recommendation model, using the second low-dimensional feature vector generated by the target subject's previous N-1 cognitive classifications; the first rehabilitation training program recommendation model is constructed using the low-dimensional feature vectors generated by the most recent cognitive classification of P subjects collected in advance.

11. A cognitive warning device, characterized in that: include: A first acquisition unit is used to acquire process data and result data generated by the target subject to be classified when performing cognitive classification, wherein the process data includes voice data, video data, and eye movement data generated by the target subject during the cognitive classification process, and the result data is electronic scale data obtained by the target subject after cognitive classification; A first extraction unit, used to extract characteristic data of the target subject from the process data and result data; A classification unit, used to determine a cognitive classification result of the target subject according to the characteristic data of the target subject; An early warning unit, configured to send a cognitive abnormality early warning prompt message to a target associated user of the target subject when it is determined that the cognitive classification result of the target subject belongs to a preset abnormal classification; The target subject is subjected to cognitive classification for the first time; the classification unit comprises: A construction subunit, used to construct pseudo-cohort data using the characteristic data of the target subject; A classification subunit, configured to input the characteristic data of the target subject into a first cognitive classification model to determine a cognitive classification result of the target subject; Wherein, the first cognitive classification model is constructed based on the pre-constructed cross-sectional data classification model using the characteristic data of the pseudo-cohort data; the cross-sectional data classification model is constructed using the characteristic data generated by the most recent cognitive classification of M subjects collected in advance; M is a positive integer greater than 1; The construction subunit comprises: A first acquisition subunit is used to acquire historical cohort data generated when K subjects were previously cognitively classified as candidate cohort data; K is a positive integer greater than 1; A calculation subunit, used to calculate the similarity between the characteristic data of the target subject and the characteristic data of all the candidate cohort data, and use the candidate cohort data corresponding to the similarity that meets the preset conditions as the pseudo cohort data of the target subject; Alternatively, the construction subunit includes: The second acquisition subunit is used to acquire historical cohort data generated when L subjects perform cognitive classification as candidate cohort data; L is a positive integer greater than 1; The selection subunit is used to select candidate cohort data that meets preset matching conditions from the candidate cohort data by using the pseudo cohort data selection model as the pseudo cohort data of the target subject.

12. A cognitive warning device, characterized in that: include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 10.

Images