An intelligent detection system for assessing cognitive ability of the elderly based on multimodality
Through an intelligent detection system based on multimodality, a variety of data of the elderly are comprehensively acquired and analyzed, and the residual attention neural network detection model with stacked structure is used for analysis and processing, which solves the problems of low detection efficiency and limited accuracy in the existing technology, and achieves more comprehensive and accurate cognitive ability detection.
Patent Information
- Application Number
- CN202510040680.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-10
- Publication Date
- 2025-04-04
AI Technical Summary
The existing cognitive ability detection methods for the elderly are inefficient and limited in accuracy, mainly due to the reliance on a single type of data for detection.
A multimodal-based intelligent detection system is adopted, through the external behavior feature acquisition module, the internal feature acquisition module, the cognitive game module and the cognitive ability detection module, the limb, facial, vocalization, EEG and heart rate data of the subjects are comprehensively obtained and analyzed, and the residual attention neural network detection model of the stacked structure is used for analysis and processing, and the cognitive ability detection results are obtained.
It has achieved a more comprehensive and accurate detection of the cognitive ability of the elderly, and can evaluate the cognitive ability of the subjects from multiple aspects and improve the detection efficiency and accuracy.
Smart Images
Figure CN119745342A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical detection equipment, and in particular to an intelligent detection system for evaluating the cognitive ability of the elderly based on multimodality. Background Art
[0002] Cognitive ability refers to the ability of the human brain to process, store and extract information, that is, people's ability to grasp the composition of things, the relationship between performance and other things, the driving force of development, the direction of development and the basic laws. It is the most important psychological condition for people to successfully complete activities. With the increase of age, the cognitive ability of the elderly will degenerate, mainly manifested in the decline of memory, thinking ability, judgment, etc., which seriously affects the quality of life and social function of the elderly, which requires the detection of the cognitive ability of the elderly.
[0003] Cognitive ability testing is to obtain relevant data of the target through testing or testing, and then determine the level of the target's cognitive ability after judging and identifying the data. Existing testing methods mainly require the tester to complete the corresponding tasks or evaluation scales under the instruction of the tester, and then the tester determines whether there is cognitive impairment. This method has low detection efficiency, and because it is often based on only a single type of data, the accuracy of the test is limited to a certain extent. Summary of the invention
[0004] In view of the existing deficiencies, the present invention provides an intelligent detection system for evaluating the cognitive ability of the elderly based on multimodality.
[0005] The technical solution adopted by the present invention to solve the technical problem is: an intelligent detection system for evaluating the cognitive ability of the elderly based on multimodality, comprising:
[0006] The external behavior feature acquisition module is used to obtain the test subject's external body, facial and vocal behavior data;
[0007] The internal feature acquisition module is used to obtain the EEG data and heart rate data of the test subject when taking cognitive game evaluation;
[0008] The cognitive game module is used to provide cognitive assessment games to the test subjects;
[0009] The cognitive ability detection module uses a residual attention neural network detection model with a stacked structure to analyze and process behavioral data, EEG data, and heart rate data to obtain cognitive data, and compares the cognitive data with the evaluation criteria to obtain the detection results;
[0010] The display storage module is used to display and store various detected data and detection results.
[0011] Preferably, the external behavior feature acquisition module includes:
[0012] The limb feature and facial feature acquisition module is used to sequentially acquire the limb data of the test subject when walking and the facial data after walking and when performing cognitive game evaluation;
[0013] An audio acquisition module is used to collect audio data of the test subject reading the test information;
[0014] The eye tracker is used to collect the eye movement data of the test subjects when they are taking cognitive game assessments.
[0015] Preferably, the residual attention neural network detection model uses a convolutional neural network to delineate regions of interest when processing limb data and facial data.
[0016] Preferably, the internal feature acquisition module comprises:
[0017] EEG instrument, used to collect EEG data of the test subjects when they are taking cognitive game assessments;
[0018] The heart rate monitor is used to collect the heart rate data of the test subjects when they are taking cognitive game assessments.
[0019] Preferably, the heart rate meter is any one of a mirror heart rate meter or a millimeter wave radar respiration meter.
[0020] Preferably, the residual attention neural network detection model is a stacked structure formed by stacking multiple residual convolution modules and attention modules, and each of the attention modules includes a mask branch unit and a trunk branch unit.
[0021] Preferably, a printing module for printing a test report is also included.
[0022] Preferably, the cognitive game module is a mechanical game module or an electronic game module.
[0023] Preferably, a GPU computing host is also included, and the cognitive ability detection module is integrated on the GPU computing host.
[0024] Preferably, the GPU computing host is installed in a box, and the display storage module is installed on the box.
[0025] The beneficial effects of the present invention are that the present invention can detect the cognitive ability of the test subject from multiple aspects, not only from the external limbs, face and vocal behavior, but also from the internal brain nerves and heart rate performance, and combined with the detection of cognitive games, it can detect the cognitive ability of the test subject more comprehensively and accurately. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1is a structural block diagram of an embodiment of the present invention;
[0027] Figure 2 is a schematic diagram of the structure of an embodiment of the present invention in a test space, wherein the dotted line portion represents a schematic diagram of the structure inside the box;
[0028] Names and serial numbers of components in the figure: 1-external behavior feature acquisition module 10-camera 11-microphone 12-eye tracker 2-internal feature acquisition module 20-EEG meter 21-millimeter wave radar respirometer 3-cognitive game module 4-cognitive ability detection module 5-display storage module 50-display screen 6-printing module 7-GPU computing host 8-cabinet 9-test space. DETAILED DESCRIPTION
[0029] In order to more clearly illustrate the purpose, technical solutions and advantages of the embodiments of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments for a clear and complete description. Obviously, the embodiments described are partial embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.
[0030] Embodiments of the present invention Figure 1 and Figure 2 As shown in the figure, a multi-modal intelligent detection system for assessing the cognitive ability of the elderly includes:
[0031] The external behavior feature acquisition module 1 is used to obtain the behavior data of the subject's external body, face and voice. The external behavior features are the states of the subject's body and face, as well as the voice displayed by reading text information. For example, if a camera 10 is used as an external behavior feature acquisition module 1 for acquiring limb and facial states, the camera 10 can use one to acquire limb and facial states in sequence according to the situation, or use multiple cameras to acquire limb and facial states accordingly, and use a microphone 11 as an external behavior feature acquisition module 1 for acquiring vocalization conditions; when one camera 10 is used, the camera 10 and the microphone 11 are integrated into a box 8, the microphone 11 is installed on the upper part of the front of the box 8, and the camera is movably installed on the top of the box, and the movable installation can adjust its angle to shoot at different angles; when multiple cameras 10 are set, a test space 9 is selected, such as a room is selected as the test space 9, and then multiple cameras 10 are arranged at different positions of the test space 9, and the subject walks in the test space 9, so that the limb states of the subject in different directions are acquired through the cameras 10 at different positions, and at the same time, one of the cameras 10 and the microphone 11 is integrated into a box 8, and the camera 10 is used to acquire the facial state of the subject. The external behavior feature acquisition module 1 includes: a limb feature and facial feature acquisition module, which is used to sequentially acquire the limb data of the testee when walking and the facial data after walking and when performing cognitive game evaluation, that is, firstly acquire the limb data of the testee when walking through the camera 10, and then use the camera 10 to acquire the facial data of the testee after walking and the facial data when performing cognitive game evaluation; an audio acquisition module, which is used to acquire the audio data of the testee reading the test information, and uses the microphone 11 as the audio acquisition module. The test information can select text information related to the date and climate of the day, and then use the microphone 11 to acquire audio data when the testee reads this information aloud; an eye tracker 12, which is used to acquire the eye movement data of the testee when performing cognitive game evaluation. The eye tracker 12 can be integrated with the microphone 11 and the camera 10 for acquiring facial data on a box 8, and the eye movement data of the testee can be acquired through the eye tracker 12 while the tester is performing cognitive game evaluation.In this way, in the limb data, there are gait characteristics from four gait perspectives: the front, back, right and left sides of the limbs; and due to the deterioration of cognitive abilities of the elderly, the swing amplitude, speed and angle of the limb movements when taking each step while walking will be different, such as human muscle strength, tendon length, step frequency and stride, body center of gravity, joint position and overall coordination ability. There are subtle differences; in facial data, facial features will show different facial muscle movements and expression patterns depending on whether the elderly understand or do not understand the things they are facing; similarly, audio data is the reading of test information, which will show different speaking speeds and pitches under the different cognitive abilities of the elderly; eye movement data is the eye movement and blinking frequency under different cognitive abilities.
[0032] The internal feature acquisition module 2 is used to obtain the EEG data and heart rate data of the test subject when performing cognitive game evaluation. The EEG data and heart rate data can be used to quantify the user's attention and cognitive state when playing cognitive games, thereby forming a more complete and comprehensive detection and evaluation with limb data, facial data, audio data, and eye movement data. At this time, the internal feature acquisition module 2 includes: an electroencephalometer 20, which is used to collect the EEG data of the test subject when performing cognitive game evaluation; a heart rate meter, which collects the heart rate data of the test subject when performing cognitive game evaluation; the electroencephalometer 20 and the heart rate meter are set in the same test space where the camera 10 and the microphone 11 are installed, which is convenient for testing, so that the changes in the test subject's own nerves and heart rate when performing cognitive game evaluation can be obtained. In order to facilitate the installation of the heart rate tester and the detection of the heart rate, the heart rate meter is any one of the mirror heart rate meter and the millimeter wave radar respiration meter 21, so that non-contact heart rate detection is formed. At this time, the heart rate meter, the electroencephalogram meter 20, the microphone 11, the eye tracker 12, and the camera 10 can be integrated in a box 8, so that they form a whole, which is convenient to use.
[0033] The cognitive game module 3 is used to provide the test subject with a cognitive assessment game, and simulate different environments through cognitive games to gain a more comprehensive understanding of the test subject's cognitive ability. The cognitive game module 3 is a mechanical game module or an electronic game module. The mechanical game module sets the test content into game cards or game turntables that the test subject can touch and manipulate, such as traditional Huarongdao, Tangram and other props; the electronic game module uses electronic games to set the corresponding test content.
[0034] The cognitive ability detection module 4 uses a residual attention neural network detection model with a stacked structure to analyze and process the behavioral data, EEG data and heart rate data to obtain cognitive data, and compares the cognitive data with the evaluation judgment criteria to obtain the detection results. The residual attention neural network detection model uses the corresponding data of the elderly with normal cognitive abilities, and then uses the existing method to construct it. At this time, the residual attention neural network detection model is a stacked structure composed of multiple residual convolution modules and attention modules. Through this structure, it is possible to more accurately extract relevant feature values that characterize the state of cognitive degeneration, capture and quantify subtle changes in cognitive features, and realize quantitative detection of multi-dimensional complex cognitive states; each attention module includes a mask branch unit and a trunk branch unit. The trunk branch unit maps the input features to different feature spaces respectively, and the mask branch unit controls the trunk branch unit according to the feature generation in the feature space, which is used to characterize the importance of each feature in the trunk branch unit, thereby highlighting the key features in the model, and accurately analyzing and processing complex big data to obtain more accurate detection results; the evaluation judgment standard adopts any one of the existing commonly used clinical cognitive impairment assessment scales, such as the Mini Mental State Examination (MMSE), the Montreal Cognitive Assessment (MoCA), the Alzheimer's Disease Assessment Scale-Cognitive Scale (ADAS-cog) and the Clinical Dementia Rating (CDR), which can be pre-written into the cognitive ability detection module. The residual attention neural network detection model uses a convolutional neural network to delineate regions of interest when processing limb data and facial data. In the limb data and facial data, that is, the regions of interest delineated in the limb images and facial images collected by the camera 10, such as the head, hands, knees, feet, lips, nose, face, etc., and then introduces the residual network to analyze the gait characteristics represented by different states contained in the coordinates, thereby reducing the amount of data processing calculations and improving the analysis and processing efficiency. The residual attention neural network detection model analyzes these data and summarizes them to form a neural network structure that has the ability to quantitatively analyze the state of cognitive degeneration of people's brains.
[0035] The display storage module 5 is used to display and store the various data and test results detected. The display storage module 5 includes a display screen 50 and a storage chip (not shown in the figure). The storage chip is used to store the detected limb data, facial data, audio data, eye movement data, EEG data and heart rate data, as well as the test results obtained after processing and analysis by the cognitive ability detection module 4. The display screen 50 displays these data and results. At the same time, the display screen 50 is also used to display the test game of the electronic game module.
[0036] Further improvements such as Figure 1 and Figure 2As shown in , it also includes a printing module 6 for printing the test report. The printing module 6 selects any existing printer, and the printer can be used to conveniently print out the test result in the form of a test report.
[0037] Further improvements such as Figure 2 As shown in, it also includes a GPU computing host 7, a cognitive ability detection module 4 is integrated on the GPU computing host 7, the GPU computing host 7 is installed in a box 8, and the display storage module 5 is installed on the box 8, so that the GPU computing host 7 and the electroencephalometer 20 can be installed in the box 8, and the measuring electrodes of the electroencephalometer 20 pass through the side of the box 8. If the heart rate meter is a mirror heart rate meter or a millimeter wave radar breathing meter 21, it is installed on the front of the box 8, and a display screen 50, a camera 10 for detecting facial status, and an eye tracker 12 are installed in sequence from bottom to top on the top of the box 8 or above the front of the box 8. The camera 10 and the eye tracker 12 can be arranged side by side at the same height, and the storage chip is integrated on the GPU computing host 7. If the heart rate meter is a contact type measuring The measuring instrument is installed in the box, and then the detection electrode is passed through the side wall of the box. A microphone 11 is installed on the upper edge of the box 8, and other cameras 10 for detecting limb status are installed at different positions of the test space 9. The electroencephalogram 20, heart rate meter, camera 10, eye tracker 12 and microphone 11 are all electrically connected to the GPU computing host 7, and the information they detect is transmitted to the cognitive ability detection module 4, and the data is processed by the cognitive ability detection module 4. When the cognitive game module 3 is an electronic game module, it is also integrated into the GPU computing host 7. When the cognitive game module 3 is a mechanical game module, it is installed on the front of the box 8 adjacent to the camera 10 and the eye tracker 12, and the printer is installed on the side or the lower part of the front of the box 8.
[0038] Although the present invention has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made on the basis of the present invention. Therefore, these modifications or improvements made on the basis of not departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.
Claims
1. An intelligent detection system for assessing the cognitive ability of the elderly based on multimodality, characterized in that: include: The external behavior feature acquisition module is used to obtain the test subject's external body, facial and vocal behavior data; The internal feature acquisition module is used to obtain the EEG data and heart rate data of the test subject when taking cognitive game evaluation; The cognitive game module is used to provide cognitive assessment games to the test subjects; The cognitive ability detection module uses a residual attention neural network detection model with a stacked structure to analyze and process behavioral data, EEG data, and heart rate data to obtain cognitive data, and compares the cognitive data with the evaluation criteria to obtain the detection results; The display storage module is used to display and store various detected data and detection results.
2. The multimodal intelligent detection system for assessing cognitive ability of the elderly according to claim 1 is characterized in that: The external behavior feature acquisition module includes: The limb feature and facial feature acquisition module is used to sequentially acquire the limb data of the test subject when walking and the facial data after walking and when performing cognitive game evaluation; An audio acquisition module is used to collect audio data of the test subject reading the test information; The eye tracker is used to collect the eye movement data of the test subjects when they are taking cognitive game assessments.
3. The multimodal intelligent detection system for assessing cognitive ability of the elderly according to claim 2 is characterized in that: The residual attention neural network detection model uses a convolutional neural network to delineate regions of interest when processing limb data and facial data.
4. The multimodal intelligent detection system for assessing cognitive ability of the elderly according to claim 1 is characterized in that: The internal feature acquisition module comprises: EEG instrument, used to collect EEG data of the test subjects when they are taking cognitive game assessments; The heart rate monitor is used to collect the heart rate data of the test subjects when they are taking cognitive game assessments.
5. The multimodal intelligent detection system for assessing cognitive ability of the elderly according to claim 4 is characterized in that: The heart rate measuring device is any one of a mirror heart rate measuring device and a millimeter wave radar respiration measuring device.
6. The multimodal intelligent detection system for assessing cognitive ability of the elderly according to claim 1 or 2, characterized in that: The residual attention neural network detection model is a stacked structure composed of multiple residual convolution modules and attention modules, and each of the attention modules includes a mask branch unit and a trunk branch unit.
7. The multimodal intelligent detection system for assessing cognitive ability of the elderly according to claim 1 is characterized in that: Also included is a printing module for printing a test report.
8. The multimodal intelligent detection system for assessing cognitive ability of the elderly according to claim 1 is characterized in that: The cognitive game module is a mechanical game module or an electronic game module.
9. The multimodal intelligent detection system for assessing cognitive ability of the elderly according to claim 1 is characterized in that: It also includes a GPU computing host, and the cognitive ability detection module is integrated on the GPU computing host.
10. The multimodal intelligent detection system for assessing cognitive ability of the elderly according to claim 9, characterized in that: The GPU computing host is installed in a box, and the display storage module is installed on the box.