Cognitive disorder screening method and system based on face analysis

Abstract

The invention discloses a cognitive impairment screening method and system based on face analysis. The cognitive impairment screening method comprises the following steps: selecting a video clip with a preset duration for a user to watch, wherein the video clip comprises a plurality of smile points; setting a corresponding time window according to the appearing time of each laughing point; based on spontaneous facial expressions after the user watches the video, facial motion features of the user in each time window are collected, and the facial motion features comprise values of a plurality of facial motion units of the user; inputting the facial motion features of the user in each time window into a preset model to output a cognitive impairment screening result; wherein the preset model carries out model training based on historical big data, so as to form a mapping relation from the facial movement features to the cognitive impairment screening result. Therefore, by utilizing the method, the cognitive impairment can be screened in a diversified scene, the dependence on professionals is reduced, a user does not need to be wounded, and the method is particularly suitable for early screening of the cognitive impairment.

Description

Technical Field

[0001] The present invention relates to a cognitive impairment screening method based on face analysis, and also relates to a corresponding cognitive impairment screening system, belonging to the technical field of healthcare informatics. Background Art

[0002] Cognitive impairment is a disease that mainly affects the elderly population, characterized by the decline of cognitive functions such as memory, thinking, and attention. Common causes include Alzheimer's disease and cerebrovascular diseases, etc. With the aggravation of the global aging problem, the prevalence of cognitive impairment has been increasing year by year, which has a negative impact on the quality of life of patients in their daily life.

[0003] Currently, the screening methods for cognitive impairment mainly include neuropsychological tests (such as MMSE, MoCA), body fluid biomarker analysis, and brain imaging techniques (such as magnetic resonance imaging MRI, PET scan), etc. Neuropsychological tests require the operation of professional assessors and the high cooperation of subjects, so there are relatively large subjective factors. Body fluid biomarker analysis, for example, through the detection of cerebrospinal fluid or blood samples, although it helps in the screening and identification of Alzheimer's disease-related cognitive impairment, it is invasive and requires venous puncture or lumbar puncture to obtain samples, with complex operations and certain risks. At the same time, there is currently a lack of stable biomarkers and reliable diagnostic standard intervals suitable for the early cognitive impairment screening of Chinese people. And although PET scan in brain imaging techniques can detect abnormal brain metabolism, the equipment is expensive, the examination is time-consuming, and patients need to receive radiation, and the operation and evaluation process requires the participation of professional medical personnel, and these factors have restricted its popularization and application.

[0004] Generally speaking, the existing cognitive impairment screening methods are restricted by technology, cost, and operation convenience in large-scale screening. Therefore, there is an urgent need for a non-invasive, low-cost, and highly sensitive screening tool to meet the needs of early cognitive impairment screening. Summary of the Invention

[0005] The primary technical problem to be solved by the present invention is to provide a cognitive impairment screening method based on face analysis.

[0006] Another technical problem to be solved by the present invention is to provide a cognitive impairment screening system based on face analysis.

[0007] To achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] According to the first aspect of the embodiments of the present invention, there is provided a cognitive impairment screening method based on face analysis, including the following steps:

[0009] Select a video clip with a preset duration for the user to watch; wherein, the video clip contains multiple laughing points;

[0010] According to the time when each of the jokes occurs, a corresponding time window is set;

[0011] Based on the spontaneous facial expressions of the user after watching the video, the facial movement features of the user in each time window are collected; wherein the facial movement features include values ​​of multiple facial action units of the user;

[0012] Inputting the facial movement features of the user in each time window into a preset model to output a cognitive impairment screening result corresponding to the user;

[0013] Among them, the preset model is trained based on historical big data to form a mapping relationship from facial movement features to cognitive impairment screening results.

[0014] Preferably, before selecting a video clip of a preset length for the user to watch, the method further includes the following steps:

[0015] Pre-identifying the user's emotional and psychological state to obtain the user's current emotional and psychological state;

[0016] Determine whether the user's current emotional and psychological state meets the screening requirements; if so, directly perform cognitive impairment screening; if not, perform emotional regulation on the user until the user's emotional and psychological state meets the screening requirements.

[0017] Preferably, the step of pre-identifying the user's emotional and psychological state to obtain the user's current emotional and psychological state includes the following sub-steps:

[0018] Inquiring the user about their current emotional state by voice questioning, and presenting expressions or animations corresponding to different emotional states;

[0019] Acquire the user's voice feedback result, the expression or animation selected by the user, and the user's facial expression and touch screen tactile information;

[0020] A comprehensive analysis is performed based on the user's voice feedback results, selected expressions or animations, facial expressions and touch screen tactile information to output the user's current emotional and psychological state.

[0021] Preferably, the user's current emotional and psychological state includes: a positive state, a neutral state, and a negative state; and before the cognitive impairment screening, when the user's current emotional and psychological state is a positive state, the screening requirements are met.

[0022] Preferably, the preset model is constructed through the following sub-steps:

[0023] Obtain the facial motion features of a large number of historical users at each time window when watching a preset video clip; wherein, the facial motion features are the values of multiple facial action units of each historical user;

[0024] Obtain the cognitive impairment screening results of each of the historical users;

[0025] At the same time window, label the corresponding facial motion features based on the cognitive impairment screening results of each historical user to form a training data set;

[0026] Based on the training data set, associate the facial motion features with the cognitive impairment screening results through a machine learning algorithm to construct a mapping relationship from the facial features to the cognitive impairment screening results, thereby forming the preset model.

[0027] Preferably, divide the training data set into a training set and a test set according to a preset ratio;

[0028] Based on the training set, associate the facial motion features with the cognitive impairment screening results through a machine learning algorithm to construct a preliminary model;

[0029] Based on the test set, test the preliminary model to evaluate the performance of the preliminary model by means of K-fold cross-validation;

[0030] According to the cross-validation results of the preliminary model, adjust the hyperparameters of the preliminary model to form the final preset model.

[0031] According to the second aspect of the embodiments of the present invention, there is provided a cognitive impairment screening system based on face analysis, including:

[0032] A video unit for pushing a video clip with a preset duration for the user to watch; wherein, the video clip contains multiple laughing points;

[0033] A setting unit, connected to the video unit, to set a corresponding time window according to the time when each laughing point appears;

[0034] An acquisition unit, connected to the setting unit, for acquiring the facial motion features of the user within each time window; wherein, the facial motion features are the values of multiple facial action units of the user;

[0035] An output unit, built-in with a preset model and connected to the acquisition unit, to output the cognitive impairment screening result corresponding to the user based on the facial motion features of the user within each time window;

[0036] Among them, the preset model is trained based on historical big data to form a mapping relationship from facial movement features to cognitive impairment screening results.

[0037] Preferably, the cognitive impairment screening system further includes:

[0038] An emotion recognition unit, configured to recognize the emotional and psychological state of the user to obtain the current emotional and psychological state of the user;

[0039] A judgment unit, connected to the emotion recognition unit, for judging whether the current emotional and psychological state of the user meets the screening requirements; if it meets, directly perform cognitive impairment screening, if it does not meet, adjust the emotion of the user until the emotional and psychological state of the user meets the screening requirements.

[0040] Preferably, the cognitive impairment screening system is an APP installed on a smart device or an accessible website set on a browser.

[0041] According to the third aspect of the embodiments of the present invention, there is provided a cognitive impairment screening system based on face analysis, including a processor and a memory. The processor reads a computer program in the memory and is configured to execute the above-mentioned cognitive impairment screening method.

[0042] Compared with the prior art, the present invention has the following technical effects:

[0043] (1) The embodiments of the present invention propose a cognitive impairment screening method based on automated facial expression analysis. This method captures the spontaneous emotional reactions of the subjects in a natural state, comprehensively showing the true psychological cognitive process of the individual. The present invention combines the ideas of psychodynamic theory, paying attention to the influence of deep motives and potential factors behind individual behaviors and thoughts. By automatically and real-time analyzing the facial expressions of users, it accurately reflects their psychological states and cognitive processes, providing an efficient and reliable new method for the early screening of cognitive impairment.

[0044] (2) Through automated facial expression analysis, the embodiments of the present invention avoid the subjective intervention of evaluators throughout the screening process and reduce the need for patient cooperation. The skit video content is vivid and interesting, which can stimulate the natural emotional reactions of users, and complete data collection and analysis through automated technology, thus overcoming the limitations of strong subjectivity and low patient cooperation in traditional evaluations. This scenario-based screening mode is close to daily life, avoids language and cultural dependencies, and improves the scientificity and universality of the screening.

[0045] (3) The embodiments of the present invention utilize portable devices to conduct detections in daily environments, reducing the dependence on dedicated devices and professional personnel. Moreover, the evaluation time is short, the operation process is simple and easy to understand, facilitating popularization. This enables rapid implementation of cognitive impairment screening among a wide range of people, thereby solving the limitations of existing screening methods in large-scale population screening.

[0046] (4) The facial expression analysis method used in the embodiments of the present invention is based on video capture technology. Portable devices are employed for data collection, and a screening model is established through machine learning. The detection process involved can be realized in various scenarios, including homes, communities, and hospitals. The entire operation process does not require high costs and does not involve invasive operations on the body, enabling preliminary screening to be applicable to a wider range of people, thereby enhancing the popularity of early screening for cognitive impairment. Description of the Drawings

[0047] Figure 1 It is a flowchart of a cognitive impairment screening method based on face analysis provided by the first embodiment of the present invention;

[0048] Figure 1A It is a flowchart of another cognitive impairment screening method based on face analysis provided by the first embodiment of the present invention;

[0049] Figure 2 It is a schematic diagram of constructing a preset model in the first embodiment of the present invention;

[0050] Figure 3 It is a structural diagram of a cognitive impairment screening system based on face analysis provided by the second embodiment of the present invention;

[0051] Figure 4 It is a structural diagram of a cognitive impairment screening system based on face analysis provided by the third embodiment of the present invention; Detailed Embodiments

[0052] The following elaborates in detail on the technical content of the present invention in conjunction with the drawings and specific embodiments.

[0053] Facial expression changes are an intuitive physiological information feature. Subjects (i.e., users) in different health states have different cognitive processes and emotional expression characteristics. By conducting facial action unit analysis on the facial structure in regions, not only can the intuitive external emotions of the subjects be captured, but also potential psychological activities and cognitive processes can be explored.

[0054] On this basis, the embodiments of the present invention provide a cognitive impairment screening method and system based on face analysis. When conducting cognitive impairment screening, the user watches a specially set skit clip in the daily life scenario. The video content is designed with humor and emotional resonance. Combined with situational induction, it can naturally stimulate the user's emotional response, facial expression changes, and emotional response with diversified content (such as sound, plot, etc.), ensuring the ecological validity of the experiment. Moreover, by using a camera to record the user's emotional dynamics and potential emotional cognitive processing in real time, and using automated analysis technology to extract the presence and evaluation values of each facial action unit in each frame, the instantaneous changes in the facial appearance can be captured.

[0055] In the embodiments of the present invention, the theoretical perspective of psychodynamics is particularly incorporated to analyze the relationship between the changes in facial expressions and the individual psychological mechanism from the perspective of dynamic psychological processes. The psychodynamic theory emphasizes the role of an individual's emotions, subconsciousness, and psychological conflicts in behavioral manifestations. By observing the user during the skit watching process, the sequential changes in facial expressions not only reflect the explicit emotional states (such as pleasure, confusion, or concentration), but also can reveal their potential emotion regulation ability, emotional conflict response pattern, and cognitive processing characteristics of external stimuli.

[0056] Next, the cognitive impairment screening method and system provided by the embodiments of the present invention will be described in detail:

[0057] First Embodiment

[0058] As Figure 1 shown, a cognitive impairment screening method based on face analysis provided by the first embodiment of the present invention specifically includes the following steps:

[0059] S1: Obtain the user's current emotional and psychological state.

[0060] Before screening the user for cognitive impairment, the user's initial emotional state will directly affect the screening result of cognitive impairment. Generally, in a positive and pleasant emotional state, the user is often more open and optimistic, and the reception and processing of information are also more active, thus ensuring the accuracy of the cognitive screening result. On the contrary, in a negative, anxious, or depressive emotional state, it may lead to the user's thinking mode becoming narrow and conservative, making the user more likely to focus on negative information and ignore positive information, thereby affecting attention and memory, and further affecting the screening result of cognitive impairment.

[0061] Therefore, in one embodiment of the present invention, before the user is screened for cognitive impairment, the user's emotional and psychological state needs to be identified in advance to obtain the user's current emotional and psychological state. Specifically, the emotional and psychological state includes a positive state, a neutral state, and a negative state. When the user's current emotional and psychological state is positive or neutral, the user is directly screened for cognitive impairment; when the user's current emotional and psychological state is negative, emotional regulation needs to be initiated until the user's emotional and psychological state improves to positive or neutral, and then the next step of cognitive impairment screening is performed.

[0062] In one embodiment of the present invention, the user's current emotional state is obtained in the following manner:

[0063] First, the user is asked about their current emotional state through voice questions, and expressions or animations corresponding to different emotional states are presented;

[0064] Secondly, obtaining the user's voice feedback results, the expression or animation selected by the user, and the user's facial expression and touch screen tactile information;

[0065] Finally, a comprehensive analysis is performed based on the user's voice feedback results, selected expressions or animations, facial expressions, and touch screen tactile information to output the user's current emotional and psychological state.

[0066] It is understood that, in another embodiment, Figure 1A As shown, step S1 may be omitted and the process directly proceeds to step S2 to screen the user for cognitive impairment.

[0067] S2: Select a video clip of a preset length for the user to watch.

[0068] When the user's emotional and psychological state meets the requirements for cognitive impairment screening, a video clip of a preset length is selected for the user to watch.

[0069] It is understandable that different diseases may have different reactions to humorous stimuli in the video due to different impairments in cognitive psychological processing. For example, people with cognitive impairment may have delayed reactions to the appearance of a joke in the video, or may not be able to correctly identify the location of the joke. Therefore, in the embodiment of the present invention, the same video clip is provided for different users to watch, so as to collect different reactions of different users when watching the video, thereby realizing the screening of cognitive impairment of users.

[0070] S3: Set the time window.

[0071] It can be understood that since the video clips in the embodiments of the present invention have multiple laugh points, and each laugh point can stimulate the user when it appears. Therefore, it is necessary to collect data during the time period when the laugh point appears, that is: at the event when each laugh point appears, set a time window for collecting the user's facial expression data (AU value) under this time window.

[0072] S4: Collect the facial movement characteristics of the user within each time window.

[0073] Specifically, during the process of the user watching the video clip, when the laugh point appears, the camera is used to capture the emotional reaction naturally shown by the user within the corresponding time window in real time, and the collected data is transmitted to the facial expression analysis tool for automated analysis in real time.

[0074] In an embodiment of the present invention, the FACS system encoding and openface automated analysis are adopted to realize the automated extraction and processing of data. As an open-source facial behavior analysis toolkit, openface can automatically detect the action units in facial expressions, simplifies the work of traditional manual annotation, and realizes automatic expression analysis in large-scale data. In the embodiments of the present invention, the openface automated analysis technology is applied to the screening of cognitive impairment diseases, provides a reliable solution for the early and efficient screening of cognitive impairment diseases, and also provides an example for expanding the application scope of this digital facial expression analysis tool.

[0075] It can be understood that after automated analysis, it is possible to extract whether the facial action unit (ActionUnit) of each frame exists and its evaluation value, so as to capture the instantaneous changes in the facial appearance, and further extract meaningful facial movement characteristics from the standardized data (for example: the intensity, frequency, and action duration of the facial movement unit, etc.).

[0076] In an embodiment of the present invention, 4 time windows are set, and these time windows correspond to the laugh point ranges of the skit. During the entire video process, key time segments are intercepted through time series pattern recognition, combined with individualized feature modeling, and the dynamic changes and potential emotional dynamics of the facial movement unit within the key time windows are mainly analyzed, such as the intensity and combination pattern of the facial action units. At the same time, the emotional fluctuation parameters are dynamically analyzed to capture the changes in the user's emotional reaction within the key time windows.

[0077] S5: Output the cognitive impairment screening result corresponding to the user.

[0078] After obtaining the facial motion features of the user within each time window based on step S4, input the facial motion features into a preset model. After model operation, the cognitive impairment screening result corresponding to the user is output. Among them, the preset model is trained based on historical big data to form a mapping relationship from facial motion features to cognitive impairment screening results.

[0079] Specifically, as shown in Figure 2 The preset model is constructed in the following way:

[0080] ① Obtain the facial motion features of a large number of historical users within each time window when watching a preset video clip. It can be understood that the facial motion features are the values of each facial action unit of the historical users when watching the video. Through the values of each facial action unit, the emotional dynamic changes and potential emotional motivation of the historical users within the time window can be reflected.

[0081] ② Obtain the cognitive impairment screening results of each historical user. Among them, the cognitive impairment screening results can be obtained through traditional evaluation methods, such as: obtained through evaluation scales or evaluation tasks, etc., or can also be obtained through methods such as body fluid biomarker analysis or brain imaging technology.

[0082] ③ At the same time window, label the corresponding facial motion features based on the cognitive impairment screening results of each historical user to form a training data set. In the embodiment of the present invention, modeling based on the data at the same time window can ensure the consistency of the data and improve the modeling accuracy.

[0083] ④ Based on the training data set, associate the facial motion features with the cognitive impairment screening results through a machine learning algorithm to construct a mapping relationship from facial features to cognitive impairment screening results, thereby forming a preset model.

[0084] Specifically, the model is trained in the following way:

[0085] First, divide the training data set into a training set and a test set according to a preset ratio;

[0086] Secondly, based on the training set, associate the facial motion features with the cognitive impairment screening results through a machine learning algorithm to construct a preliminary model;

[0087] Then, based on the test set, test the preliminary model to evaluate the performance of the preliminary model through the method of K-fold cross-validation; where K is determined based on a preset ratio;

[0088] Finally, according to the cross-validation results of the preliminary model, adjust the hyperparameters of the preliminary model to form the final preset model.

[0089] It can be understood that based on this preset model, after the facial movement features of the user are obtained through step S4, the cognitive impairment screening result corresponding to the facial movement features can be matched through model calculation, so as to efficiently, accurately and quickly screen a large number of people and provide support for the early diagnosis of cognitive impairment.

[0090] Second Embodiment

[0091] As Figure 3 shown, on the basis of the above first embodiment, the second embodiment of the present invention provides a cognitive impairment screening system based on face analysis, including an emotion recognition unit 1, a judgment unit 2, a video unit 3, a setting unit 4, a collection unit 5, and an output unit 6.

[0092] Specifically, the emotion recognition unit 1 is used to recognize the emotional and psychological state of the user to obtain the current emotional and psychological state of the user. Specifically, the emotion recognition unit 1 may include a voice interaction module, an interface interaction module, and a touch screen interaction module, which are respectively used to obtain the voice feedback result, the interface selection result, and the touch screen tactile information of the user, so as to perform comprehensive analysis through the built-in algorithm and finally output the current emotional and psychological state of the user.

[0093] The judgment unit 2 is connected to the emotion recognition unit 1 to judge whether the current emotional and psychological state of the user meets the screening requirements. Specifically, the emotional and psychological state includes a positive state, a neutral state, and a negative state. When the current emotional and psychological state of the user is positive or neutral, the user is directly screened for cognitive impairment. When the current emotional and psychological state of the user is negative, emotion regulation needs to be started until the emotional and psychological state of the user improves to positive or neutral, and then the next step of cognitive impairment screening is carried out.

[0094] The video unit 3 is connected to the judgment unit 2 to push a video clip with a preset duration for the user to watch when the user meets the screening requirements. It can be understood that there are multiple video clips preset in the video unit 3, and different video clips have different preset durations and numbers of laughing points, so as to push different video clips for cognitive impairment screening according to users with different disease types.

[0095] The setting unit 4 is connected to the video unit 3 to set a corresponding time window according to the time when each laughing point appears. The setting of the time window refers to the description in the above first embodiment and will not be elaborated here.

[0096] The collection unit 5 is connected to the setting unit 4 to collect the facial movement features of the user in each time window. Correspondingly, the output unit 6 is connected to the collection unit 5 and has a preset model built in to output the cognitive impairment screening result corresponding to the user based on the facial movement features of the user in each time window.

[0097] In an embodiment of the present invention, preferably, the cognitive impairment screening system is an APP installed on a smart device or an accessible website set on a browser. Thereby, it enables users to complete the screening of cognitive impairment in various scenarios (such as: at home, in the community, or in the hospital, etc.), greatly improving the convenience and popularity of cognitive screening. Moreover, the entire screening process does not require expensive costs and does not involve invasive operations on the body, and can be applied to a wider range of people for preliminary screening.

[0098] It can be understood that each module unit in the embodiment of the present invention is a modular structure corresponding to each step in the above first embodiment, but is not limited to this module combination form. In other embodiments, the functions of each module unit can also be adaptively adjusted to implement the cognitive impairment screening method described in the above first embodiment.

[0099] Third Embodiment

[0100] As Figure 4 shown, based on the above first embodiment, the third embodiment of the present invention provides a cognitive impairment screening system based on face analysis. The cognitive impairment screening system includes one or more processors 21 and a memory 22. Among them, the memory 22 is coupled to the processor 21 and is used to store one or more programs. When the one or more programs are executed by the one or more processors 21, the one or more processors 21 are enabled to implement the cognitive impairment screening method based on face analysis as described in the above embodiment.

[0101] Among them, the processor 21 is used to control the overall operation of the cognitive impairment screening system to complete all or part of the steps of the above cognitive impairment screening method based on face analysis. The processor 21 can be a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a digital signal processing (DSP) chip, etc. The memory 22 is used to store various types of data to support the operation of the cognitive impairment screening system. These data can include, for example, instructions for any application program or method operating on the cognitive impairment screening system, as well as data related to the application program. The memory 22 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, etc.

[0102] In an exemplary embodiment, the cognitive impairment screening system may be specifically implemented by a computer chip or an entity, or by a product with certain functions, for executing the above-mentioned cognitive impairment screening method based on face analysis and achieving the same technical effects as the above method. A typical embodiment is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0103] In another exemplary embodiment, the present invention further provides a computer-readable storage medium including program instructions, which, when executed by a processor, implement the steps of the cognitive impairment screening method based on face analysis in any of the above embodiments. For example, the computer-readable storage medium may be the above-mentioned memory including program instructions, and the above program instructions may be executed by the processor of the cognitive impairment screening system to complete the above-mentioned cognitive impairment screening method based on face analysis and achieve the same technical effects as the above method.

[0104] In summary, the cognitive impairment screening method and system provided by the embodiments of the present invention have the following beneficial effects:

[0105] (1) The embodiments of the present invention propose a cognitive impairment screening method based on automated facial expression analysis. This method captures the spontaneous emotional reactions of the subjects in a natural state, comprehensively demonstrating the true psychological cognitive process of the individual. The present invention combines the ideas of psychodynamic theory, paying attention to the influence of the deep motives and potential factors behind individual behaviors and thoughts. By automatically and real-time analyzing the facial expressions of users, it accurately reflects their psychological states and cognitive processes, providing an efficient and reliable new method for the early screening of cognitive impairment.

[0106] (2) Through automated facial expression analysis, the embodiments of the present invention avoid the subjective intervention of the evaluator throughout the screening process and reduce the need for patient cooperation. The skit video content is vivid and interesting, which can stimulate the natural emotional reactions of users, and complete data collection and analysis through automated technology, thus overcoming the limitations of strong subjectivity and low patient cooperation in traditional evaluations. This scenario-based screening mode is close to daily life, avoiding language and cultural dependencies, and improving the scientificity and universality of the screening.

[0107] (3) The embodiments of the present invention utilize portable devices to conduct detections in daily environments, reducing the dependence on dedicated devices and professionals. Moreover, the evaluation time is short, and the operation process is simple, easy to understand, and convenient to popularize, enabling the rapid implementation of cognitive impairment screening among a wide range of people, thereby solving the limitations of existing screening methods in large-scale population screening.

[0108] (4) The facial expression analysis method used in the embodiments of the present invention is based on video capture technology. Portable devices are used for data collection, and a screening model is established through machine learning. The involved detection process can be realized in various scenarios, including homes, communities, and hospitals. The entire operation process does not require high costs and does not involve invasive operations on the body, and can be applied to a wider range of people for preliminary screening, thereby improving the popularity of early screening for cognitive impairment.

[0109] It should be noted that the above-mentioned multiple embodiments are only examples. The technical solutions of each embodiment can be combined and are all within the protection scope of the present invention.

[0110] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality of" means two or more, unless otherwise specifically defined.

[0111] The above provides a detailed description of the cognitive impairment screening method and system based on face analysis provided by the present invention. For those of ordinary skill in the art, any obvious changes made without departing from the substantial content of the present invention will constitute an infringement of the patent right of the present invention and will bear corresponding legal responsibilities.

Claims

1. A method for screening cognitive impairment based on face analysis, characterized in that The steps include: Selecting a video clip of a preset length for the user to watch; wherein the video clip contains a plurality of laugh points; According to the time when each of the jokes occurs, a corresponding time window is set; Based on the spontaneous facial expressions of the user after watching the video, the facial movement features of the user in each time window are collected; wherein the facial movement features include values ​​of multiple facial action units of the user; Inputting the facial movement features of the user in each time window into a preset model to output a cognitive impairment screening result corresponding to the user; Among them, the preset model is trained based on historical big data to form a mapping relationship from facial movement features to cognitive impairment screening results.

2. The method for screening for cognitive impairment according to claim 1, characterized in that Before selecting a video clip of a preset length for the user to watch, the following steps are also included: Pre-identifying the user's emotional and psychological state to obtain the user's current emotional and psychological state; Determine whether the user's current emotional and psychological state meets the screening requirements; if so, directly perform cognitive impairment screening; if not, perform emotional regulation on the user until the user's emotional and psychological state meets the screening requirements.

3. The method for screening for cognitive impairment according to claim 2, characterized in that The step of pre-identifying the user's emotional and psychological state to obtain the user's current emotional and psychological state includes the following sub-steps: Inquiring the user about their current emotional state by voice questioning, and presenting expressions or animations corresponding to different emotional states; Acquire the user's voice feedback result, the expression or animation selected by the user, and the user's facial expression and touch screen tactile information; A comprehensive analysis is performed based on the user's voice feedback results, selected expressions or animations, facial expressions and touch screen tactile information to output the user's current emotional and psychological state.

4. The method for screening cognitive impairment according to claim 3, characterized in that: The user's current emotional and psychological state includes: a positive state, a neutral state, and a negative state; and before the cognitive impairment screening, when the user's current emotional and psychological state is a positive state, the screening requirements are met.

5. The method for screening for cognitive impairment according to claim 1, characterized in that The preset model is constructed through the following sub-steps: Acquire facial motion features of a large number of historical users in each time window when watching a preset video clip; wherein the facial motion features are values ​​of multiple facial action units of each historical user; Obtaining cognitive impairment screening results for each of the historical users; In the same time window, corresponding facial movement features are annotated based on the cognitive impairment screening results of each of the historical users to form a training data set; Based on the training data set, the facial movement features are associated with the cognitive impairment screening results through a machine learning algorithm to construct a mapping relationship from the facial features to the cognitive impairment screening results, thereby forming the preset model.

6. The method for screening cognitive impairment according to claim 5, characterized in that: Dividing the training data set into a training set and a test set according to a preset ratio; Based on the training set, the facial movement features are associated with the cognitive impairment screening results by a machine learning algorithm to construct a preliminary model; Based on the test set, the preliminary model is tested to evaluate the performance of the preliminary model by means of K-fold cross validation; According to the cross-validation result of the preliminary model, the hyperparameters of the preliminary model are adjusted to form a final preset model.

7. A cognitive impairment screening system based on face analysis, characterized in that include: A video unit, used to push a video clip of a preset length for the user to watch; wherein the video clip contains multiple laugh points; A setting unit connected to the video unit to set a corresponding time window according to the time when each of the laugh points appears; A collection unit connected to the setting unit, for collecting facial movement features of the user in each time window; wherein the facial movement features are values ​​of multiple facial action units of the user; an output unit, which has a preset model built in and is connected to the acquisition unit, so as to output a cognitive impairment screening result corresponding to the user based on the facial movement features of the user in each time window; Among them, the preset model is trained based on historical big data to form a mapping relationship from facial movement features to cognitive impairment screening results.

8. The cognitive impairment screening system according to claim 7, characterized in that Also includes: An emotion recognition unit, used to recognize the emotional psychological state of the user to obtain the current emotional psychological state of the user; A judgment unit is connected to the emotion recognition unit to judge whether the user's current emotional and psychological state meets the screening requirements; if so, cognitive impairment screening is directly performed; if not, the user is emotionally regulated until the user's emotional and psychological state meets the screening requirements.

9. The cognitive impairment screening system according to claim 7, wherein: The cognitive impairment screening system is an APP installed on a smart device or an accessible website set on a browser.

10. A cognitive impairment screening system based on face analysis, characterized in that It comprises a processor and a memory, wherein the processor reads a computer program in the memory to execute the cognitive impairment screening method according to any one of claims 1 to 6.

Images