Seat type cognitive function examination cabin based on visual and auditory evaluation and method thereof

Abstract

The invention discloses a seat-type cognitive function inspection cabin based on visual and auditory assessment and a method thereof, and relates to the technical field of cognitive function assess.The seat-type cognitive function inspection cabin comprises a cabin body, earphones and a display screen, the earphones are used for providing sound scene information of auditory assessment, and the display screen is used for providing picture scene information of visual assessment. A visual platform mechanism is arranged in front of the cabin body. The eye movement state of the user is tested by using the eye tracker, and the visual attention of the user is tested by simulating common scenes in life; the visual spatial layering sense of an object is created by means of the light and shadow direction, the visual size and the visual overlapping effect of the object on a picture, the position of an arm of a user is simulated on the picture by means of a sports bracelet, and the visual perceptibility is evaluated through the interaction function with the object on the picture; the performance of vision in memory is tested by means of cooperation with sound or visual task operation, and the method for testing the visual cognition function is more comprehensive.

Description

Technical Field

[0001] The present invention relates to the technical field of cognitive function assessment, and in particular to a seated cognitive function examination cabin based on visual and auditory assessment and a method thereof. Background Art

[0002] Cognitive function refers to a series of psychological abilities involved in the human brain in processing, storing and using information. It covers many aspects and plays a key role in our daily life, study, work and many other activities. Cognitive function is mainly reflected in: perception, memory, attention, speech comprehension and logical thinking ability. Vision and hearing are the main ways of human perception and play an extremely important role in many aspects of daily life, study and work.

[0003] With the advancement of medical technology in recent years, in addition to the widely used classic scales such as the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA), virtual reality (VR) and augmented reality (AR) technologies have provided new means for cognitive function assessment; However, in existing technologies, the information presented by VR and AR technologies is often rich and complex, which may bring a large cognitive load to the person being evaluated, making it difficult for them to process and understand a large amount of information in a short period of time, thus affecting the accurate evaluation of their cognitive functions. For example, in AR-assisted learning or evaluation scenarios, too many virtual information prompts can easily distract the person being evaluated; The prior art also involves methods for evaluating cognitive attention using signal stimulation, such as the patent publication number CN114052736A, "Cognitive function evaluation system and method and its application in cognitive function deficiency". This method improves the type of stimulation signal (tactile, olfactory, auditory, visual), the number of stimulation signals, the stimulation signal display time, the stimulation signal display order, and the stimulation signal change interval, and has a good effect on the visual and auditory attention evaluation. Strong stimulation effects can easily affect the perception of the person being evaluated. The above devices or methods are highly targeted and are more effective when the patient's initial etiology is understood. However, for patients who are being evaluated for the first time, the evaluation results of these methods may be biased.

[0004] In view of this, how to provide a cognitive function detection device that can comprehensively evaluate visual and auditory cognitive functions has become a technical problem that needs to be solved in this field. Summary of the invention

[0005] The purpose of the present invention is to provide a seated cognitive function examination cabin and method based on visual and auditory assessment to solve the problems raised by the above background technology.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a seated cognitive function inspection cabin based on visual and auditory assessment, comprising a cabin body, headphones and a display screen, wherein the headphones are used to provide sound scene information for auditory assessment, and the display screen is used to provide picture scene information for visual assessment. A visual platform mechanism is arranged in front of the cabin body, and the visual platform mechanism comprises a suspended platform and a stand, and the display screen is mounted on the top surface of the visual platform mechanism, and bracket two is fixedly connected to both sides of the suspended platform, bracket one is fixedly connected to one side of the cabin body, and the stand is fixedly connected to one side of bracket two, and a rotating shaft is fixedly connected to the top of the suspended platform, and a turntable is rotatably connected to the bottom end of the rotating shaft, and the turntable is fixedly connected to the base of the cabin body, and bracket three is fixedly connected to the top of the stand; It also includes an eye tracker and a sports bracelet, wherein the eye tracker is arranged on the third bracket and is used to track the movement of the eyes, and the sports bracelet is arranged on the second bracket and is used to sense and analyze the movement of the upper limbs; The steps of use are as follows: wear the eye tracker in front of the user's eyes, and mount it on the bridge of the nose and the ears through the frame; wear the earphones on the user's head, and cover the left and right ears respectively through two earmuffs; wear the two sports bracelets on the user's wrists, and fix them by wrapping the wrists with a ring belt; the user sits on the cockpit body, rotates the rotation axis ninety degrees around the turntable, so that the suspended platform is enclosed in front of the user, and completes the visual and auditory cognitive function detection through the interaction between the user and the eye tracker, earphones, display screen and sports bracelets.

[0007] Preferably, a support frame is fixedly connected to the base of the cabin body, a cover shell is connected to the top of the support frame, a spring is fixedly connected to the top surface of the cover shell, and a buffer platform is fixedly connected to the top of the spring.

[0008] The present invention also discloses a cognitive function assessment method based on visual and auditory assessment, which includes two parts: a visual cognitive assessment method and an auditory cognitive assessment method. The visual cognitive assessment method includes: using an eye tracker to track the eye movement of a user under a visual test of a display screen, thereby completing a test of visual attention; using a sports bracelet to monitor the interactive limb movement of the user under a visual test of the display screen, thereby completing a test of visual perception; using a display screen to display an image and broadcast the corresponding sound information, testing the associative memory effect between the two, or testing the storage and processing capabilities of image information, thereby completing a test of visual memory; The auditory cognitive assessment methods include: using questions and answers to assess the accuracy of the user's information acquisition under the headphone auditory test, thereby completing the test of auditory attention; using information expression to assess the user's ability to understand and analyze information content under the headphone auditory test, thereby completing the test of auditory perception; using headphones to broadcast sounds and display corresponding image information to test the associative memory effect between the two, or testing the ability to store and process sound information to complete the test of auditory memory.

[0009] Preferably, in the visual attention test, three tests of visual selective attention, visual sustained attention and visual allocated attention are performed respectively, wherein: target information is displayed in a specific area on the display screen, and environmental information is displayed in other areas, an eye tracker is used to test the time that the user's gaze is focused on the target information, the proportion of time the user gazes in the area of ​​the target information is analyzed, and the user's visual selective attention ability in complex visual scenes is tested; a delayed visual scene is displayed on the display screen, and the target information changes continuously in the delayed visual scene, an eye tracker is used to test the change of the user's gaze with the target information, the following synchronization rate of the user's gaze with the change of the target information is analyzed, and the user's visual sustained attention ability in the delayed visual scene is tested; multiple groups of target information are displayed in a specific area on the display screen, and environmental information is displayed in other areas, an eye tracker is used to measure the user's gaze proportion and gaze transfer frequency in different target information, the situation of the user simultaneously obtaining information in multiple areas is analyzed, and the user's visual allocated attention ability in multi-target scenes is tested.

[0010] Preferably, in the visual perception test, three tests are performed respectively, namely, background distinction perception, object recognition perception, and spatial hierarchy perception, wherein: target information integrated into the background environment is displayed on the display screen to form a protective color of the environmental information for the target information, the user's upper limb movement is detected by a sports bracelet, and the interactive function between the user and the image information on the display screen is realized, and the task instruction of the test is to search for target information in the background environment to test the user's background distinction perception ability in a specific visual scene; multi-target scene information is displayed on the display screen, the user's upper limb movement is detected by a sports bracelet, and the interactive function between the user and the image information on the display screen is realized, and the task instruction of the test is to identify the task target in the multi-target scene to test the user's object recognition perception ability in the multi-target scene; three-dimensional space scene information is displayed on the display screen, and the spatial position state of the object is visually created through light and shadow and object overlapping effects, the user's upper limb movement is detected by a sports bracelet, and the interactive function between the user and the image information on the display screen is realized, and the task instruction of the test is to grab the task target in three-dimensional space to test the user's spatial hierarchy perception ability in three-dimensional space scenes.

[0011] Preferably, in the visual memory test, two tests, visual associative memory and visual processing memory, are performed separately, wherein: image information is displayed on a display screen, and sound information is played in coordination with headphones, and the sound information corresponds to the content described in the image information, and the task instruction of the test is for the user to distinguish the corresponding image information according to the sound information, so as to test the user's visual associative memory ability; single-view image information of the target object is displayed on the display screen, and the task instruction of the test is for the user to distinguish the visual state of the target object after the view angle is changed, so as to test the user's visual processing memory ability.

[0012] Preferably, in the auditory attention test, three tests of auditory selective attention, auditory sustained attention and auditory divided attention are performed respectively, among which: sound scene information of multiple groups of mixed conversations is played through headphones, and the task instruction of the test is to capture target conversation information among multiple groups of conversation information, and the user's auditory selective attention ability is tested through question and answer; sound scene information of long conversations is played through headphones, and the task instruction of the test is to capture random conversation information among the long conversation information, and the user's auditory sustained attention ability is tested through question and answer; sound scene information of multiple sound sources is played through headphones, and the task instruction of the test is to identify various types of sound information among the sounds of multiple sound sources, and the user's auditory divided attention ability is tested through question and answer.

[0013] Preferably, in the auditory perception test, three tests are performed respectively, namely, sound recognition perception, sound scene analysis and information comprehension perception, wherein: a specific type of sound scene information is played through headphones, and the task instruction of the test is to identify the sound type in the sound scene, so as to test the user's sound recognition perception ability; the sound scene information generated by the interaction of different objects is played through headphones, and the task instruction of the test is to describe the scene where the sound occurs, so as to test the user's sound scene analysis ability; the command requirements for the target object in the scene are played through headphones, and the user uses a sports bracelet to interact with the image information on the display screen, and performs interactive actions in the image scene information on the display screen, so as to test the user's information comprehension perception ability.

[0014] Preferably, in the auditory memory test, two tests, auditory associative memory and auditory processing memory, are performed separately, wherein: sound information is played through headphones, and image information is displayed collaboratively using a display screen, and the image information corresponds to the content described by the sound information, and the test task instruction is for the user to distinguish the corresponding sound information based on the image information, so as to test the user's auditory associative memory ability; sound information is played through headphones, and the test task instruction is to retell or translate the content of the sound information, so as to test the user's auditory processing memory ability.

[0015] Compared with the prior art, the present invention has the following beneficial effects: 1. Visually, the present invention uses an eye tracker to test the user's eye movement state, and simulates common scenes in life, such as crossing the road, driving a car, etc., and also uses game entertainment methods such as hat tricks to increase the fun function to test the user's visual attention; with the help of the light and shadow direction, visual size and visual overlapping effect of the object on the screen, the visual spatial hierarchy of the object is created, and the position of the user's arm is simulated on the screen by using a sports bracelet, and the visual perception is evaluated by the interactive function with the object on the screen; with the help of coordination with sound or visual task operations to test the visual performance in memory, the test method of visual cognitive function is more comprehensive; 2. In terms of hearing, the present invention plays recorded conversations with information through headphones, such as noisy conversations, long conversations, command conversations and other types of conversations, and tests the user's auditory attention through question-and-answer methods; with the help of scene information analysis of the sound, the user's perception ability is reflected by the ability to repeat the information; the hearing performance in memory is tested by coordination with pictures or auditory task operations, and the test method for auditory cognitive function is more comprehensive. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The overall structure of the seated cognitive function examination cabin based on visual and auditory assessment of the present invention is schematically shown. Figure 1 ; Figure 2 The overall structure of the seated cognitive function examination cabin based on visual and auditory assessment of the present invention is schematically shown. Figure 2 ; Figure 3 It is a structural schematic diagram of the visual platform mechanism in the seated cognitive function examination cabin based on visual and auditory assessment of the present invention; Figure 4 It is a schematic structural diagram of the visual platform mechanism in the seated cognitive function inspection cabin based on visual and auditory assessment in the open state of the present invention; Figure 5 It is a schematic diagram of the structure of the earphone in the seated cognitive function examination cabin based on visual and auditory assessment of the present invention; Figure 6 It is a structural schematic diagram of a buffer table in a seated cognitive function inspection cabin based on visual and auditory assessment of the present invention; Figure 7 It is a schematic diagram of the structure of an eye tracker in a seated cognitive function examination cabin based on visual and auditory assessment of the present invention; Figure 8 It is an overall block diagram of the cognitive function assessment method based on visual and auditory assessment of the present invention; Fig. 9 This is a simplified diagram of a demonstration case of the first test in Example 3 of the cognitive function assessment method based on visual and auditory assessment of the present invention; Fig.10This is a simplified diagram of a demonstration case of the second test in Example 3 of the cognitive function assessment method based on visual and auditory assessment of the present invention; Fig.11 This is a simplified diagram of a demonstration case of the third test in Example 3 of the cognitive function assessment method based on visual and auditory assessment of the present invention; Fig.12 This is a simplified diagram of a demonstration case of the first test in Embodiment 4 of the cognitive function assessment method based on visual and auditory assessment of the present invention; Fig.13 This is a simplified diagram of a demonstration case of the second test in Embodiment 4 of the cognitive function assessment method based on visual and auditory assessment of the present invention; Fig.14 This is a simplified diagram of a demonstration case of the third test in Embodiment 4 of the cognitive function assessment method based on visual and auditory assessment of the present invention; Fig.15 This is a simplified diagram of a demonstration case of the second test in Embodiment 5 of the cognitive function assessment method based on visual and auditory assessment of the present invention; In the figure: 1. cockpit body; 11. bracket one; 2. eye tracker; 3. earphone; 4. display screen; 5. visual platform mechanism; 51. suspended platform; 52. rotating axis; 53. turntable; 54. buffer table; 55. cover; 56. support frame; 57. bracket two; 58. spring; 59. stand; 590. bracket three; 6. sports bracelet. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the implementation regulations described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Example 1: Reference Figure 1-Figure 7 As shown, a seated cognitive function inspection cabin based on visual and auditory assessment includes a cabin body 1, earphones 3 and a display screen 4. The earphones 3 are used to provide sound scene information for auditory assessment, and the display screen 4 is used to provide picture scene information for visual assessment. A visual platform mechanism 5 is arranged in front of the cabin body 1. The visual platform mechanism 5 includes a suspended platform 51 and a stand 59. The display screen 4 is installed on the top surface of the visual platform mechanism 5. Both sides of the suspended platform 51 are fixedly connected with brackets 57. One side of the cabin body 1 is fixedly connected with bracket 11. The stand 59 is fixedly connected to one side of the bracket 2 57. The top of the suspended platform 51 is fixedly connected with a rotating shaft 52. The bottom end of the rotating shaft 52 is rotatably connected with a turntable 53. The turntable 53 is fixedly connected to the base of the cabin body 1. The top of the stand 59 is fixedly connected with bracket 3 590. The seated cognitive function examination cabin also includes an eye tracker 2 and a sports bracelet 6. The eye tracker 2 is arranged on the third bracket 590 and is used to track the movement of the eyes. The sports bracelet 6 is arranged on the second bracket 57 and is used to sense and analyze the movement of the upper limbs. A support frame 56 is fixedly connected to the base of the cockpit body 1 , a cover shell 55 is connected to the top of the support frame 56 , a spring 58 is fixedly connected to the top surface of the cover shell 55 , and a buffer platform 54 is fixedly connected to the top of the spring 58 .

[0019] In this embodiment, the user sits on the cockpit body 1, takes the eye tracker 2 hung on the bracket 3 590, wears it in front of the eyes, takes the earphone 3 hung on the bracket 1 11, wears it on the head, rotates the suspended platform 51 by 90 degrees to surround the front of the cockpit body 1, and takes the sports bracelet 6 from the brackets 2 57 on both sides, wears it on the two wrists respectively, and senses the user's upper limb movement status; When conducting a visual inspection, image information is played through the display screen 4. Complex environment scenes, long-term continuous scenes, natural environment scenes, etc. can be displayed on the display screen 4. The eye tracker 2 and the sports bracelet 6 are used as detection tools for the identification results. The eye tracker 2 can be used to check the user's eye movement status in the attention test items, including gaze points, eye saccades, pupil size changes, etc., which are used as a basis for judging the user's visual attention ability. The sports bracelet 6 can be used to check the user's forelimb movements in the visual perception test items. The forelimb movement simulation is projected on the display screen 4 for display, completing the interaction function between the user and the image information on the display screen 4, which is used as a basis for judging the user's visual perception ability; when conducting an auditory test, sound information is played through the earphone 3, and the user's cognitive functions in auditory attention, perception ability, memory, etc. are checked through a question-and-answer method.

[0020] Example 2: Reference Figure 8 As shown, this embodiment provides a cognitive function assessment method based on visual and auditory assessment, including a visual cognitive assessment method and an auditory cognitive assessment method, wherein the visual cognitive assessment method includes: The eye tracker 2 is used to track the eye movement of the user under the visual test of the display screen 4, so as to complete the test of visual attention; the sports bracelet 6 is used to monitor the interactive body movement of the user under the visual test of the display screen 4, so as to complete the test of visual perception; the display screen 4 is used to display images and broadcast corresponding sound information, to test the associative memory effect between the two, or to test the storage and processing capabilities of image information, so as to complete the test of visual memory; The auditory cognition assessment method includes: using questions and answers to assess the accuracy of information acquisition by the user in the auditory test of the earphone 3, thereby completing the test of auditory attention; using information expression to assess the user's ability to understand and analyze information content in the auditory test of the earphone 3, thereby completing the test of auditory perception; Using earphone 3 to broadcast the sound and display the corresponding image information, test the associative memory effect between the two, or test the storage and processing ability of the sound information, and complete the test of auditory memory; Spatial hierarchy perception and information comprehension perception were conducted under a joint test, and visual practice memory and auditory associative memory were conducted under a joint test.

[0021] In this embodiment, the visual cognitive function is manifested in three aspects: visual attention, visual perception, and visual memory; Visual attention refers to the cognitive ability of an individual to selectively focus on certain visual stimuli and ignore other irrelevant stimuli during visual perception. It plays an extremely important role in many aspects of daily life, study and work. The key points of visual attention are the attention and neglect of information. How to pay attention to effective targets and ignore invalid objects in a complex visual environment; how to pay attention to a single target and ignore other targets in a continuous visual environment; how to pay attention to multiple effective targets, control the frequency of information intake of multiple effective targets, and ensure that information intake is balanced. All of the above are important manifestations of visual attention ability. Visual perception refers to receiving visual information through the eyes and processing it in the brain, so that individuals can understand and interpret these visual stimuli and form cognition of the surrounding environment, objects and events; The key points of visual perception are the capture, analysis and understanding of information. Images contain the distinction of targets integrated into the background, the identification of multiple types of target information, and the perception of the position of objects in the spatial hierarchy. How to distinguish, identify and perceive the position of information requires analysis and understanding of information. These processes are important manifestations of visual perception ability. Visual memory refers to the ability to store, retain and reproduce information seen through the visual organs, which plays an extremely important role in many aspects such as learning, life and work. Visual memory is the storage and reproduction of information. There are many ways to store visual information. One is to use the associative memory method to store visual information. Vision, hearing, smell and touch are several means for the human body to receive external information. The ability to obtain information from vision and hearing is the strongest. Visual information and auditory information can be stored by associating, so that users can more easily form memory reproduction when receiving auditory information corresponding to vision. The second is to use the task processing ability of vision to test the memory function of visual cognition.

[0022] Auditory attention refers to the ability to direct and focus mental activities on specific sound stimuli during listening; The key points of auditory attention are focusing and ignoring. How to focus on effective targets and ignore invalid objects in a noisy environment; how to focus on a single target in a continuous auditory environment; how to focus on multiple effective targets and effectively obtain effective information for each target are all important manifestations of auditory attention ability.

[0023] Auditory perception refers to the brain's ability to receive, analyze, understand and integrate the sounds heard. It is a complex cognitive function that plays a key role in the interaction with the sound environment. The key points of auditory perception also lie in the reception, integration, analysis and understanding of information. For example, recognizing sounds of a certain timbre, understanding the sound state of objects based on the sounds produced by the interaction of objects, and being able to understand and execute sound information instructions are all important manifestations of auditory perception ability.

[0024] Auditory memory refers to the brain's ability to store, retain and reproduce heard information; The key to auditory memory lies in the storage and reproduction of information. Similarly, sound information can be associated with visual information for memory, and users can also be given instructions such as retelling and translating sound information to test memory ability.

[0025] Embodiment three: Fig. 9 , Fig.10 and Fig.11 As shown, in the visual attention test, three tests are conducted, namely, visual selective attention, visual sustained attention and visual allocated attention, among which: Display target information in a specific area on the display screen 4, and display environmental information in other areas, use the eye tracker 2 to test the time the user's gaze is focused on the target information, analyze the proportion of the time the user gazes in the area of ​​the target information, and test the user's visual selection attention ability in complex visual scenes; A delayed visual scene is displayed on a display screen 4, and the target information changes continuously in the delayed visual scene. The eye tracker 2 is used to test the change of the user's sight line with the target information, and the following synchronization rate of the user's sight line with the change of the target information is analyzed to test the user's visual sustained attention ability in the delayed visual scene; Multiple sets of target information are displayed in a specific area on the display screen 4, and environmental information is displayed in other areas. The eye tracker 2 is used to measure the user's gaze proportion and gaze shift frequency in different target information, analyze the user's simultaneous acquisition of information in multiple areas, and test the user's visual attention allocation ability in multi-target scenes.

[0026] In this embodiment, in order to test visual attention, the following three test schemes are provided: The first set of test cases is as follows: a busy street scene is presented on the display screen 4, and pedestrians and vehicles are passing through the intersection in the scene. The user simulates crossing the road in the case, so he needs to focus his visual attention on the intersection and ignore the shops and pedestrians on both sides of the street. The eye tracker 2 checks the time the user's gaze stays at the intersection. When the gaze time reaches the set value, a single measurement is completed, and multiple tests are performed by switching to different scenes; The second set of test scenarios is as follows: three hats are displayed on the display screen 4, one of which has candy under it, and the other two hats have nothing under them. After the hats are turned upside down, the user cannot directly observe the position of the candy, but can only know which hat the candy is under at the beginning. The picture starts to change, and the positions of the three hats keep changing. When the position changes, the user is required to point out the position of the candy. During the test, the eye tracker 2 checks the synchronization rate of the user's sight line with the moving trajectory of the candy, and determines the test result through the candy position verification and the eye tracker 2 monitoring the sight line change process; The third group of test scenarios is as follows: a car driving picture is presented on the display screen 4, with the front road condition in the center of the picture, the left and right lower corners of the picture are the left and right rearview mirrors respectively, and the central rearview mirror is located directly above the picture. The user needs to observe the center of the picture, the left and right rearview mirrors and the central rearview mirror respectively within a period of time. Driving obstacles or vehicles are displayed in different picture areas from time to time. The eye tracker 2 is used to test whether the user's line of sight can switch with the display information in different picture areas, and the test result is determined by comprehensively considering the user's line of sight shifting frequency in different areas. The above three groups of cases are classic cases of visual selective attention, visual sustained attention and visual divided attention tests. The cases in the actual inspection process include but are not limited to these.

[0027] Embodiment 4: Fig.12 , Fig.13 and Fig.14 As shown, in the visual perception test, three tests are conducted, namely, background distinction perception, object recognition perception, and spatial hierarchy perception, among which: The target information integrated into the background environment is displayed on the display screen 4, forming a protective color of the environment information for the target information. The sports bracelet 6 is used to detect the upper limb movement of the user to realize the interaction function between the user and the image information on the display screen 4. The task instruction of the test is to search for the target information in the background environment and test the user's background distinction perception ability in a specific visual scene; The multi-target scene information is displayed on the display screen 4, and the upper limb movement of the user is detected by the sports bracelet 6 to realize the interaction function between the user and the image information on the display screen 4. The task instruction of the test is to identify the task target in the multi-target scene to test the user's object recognition perception ability in the multi-target scene; The three-dimensional space scene information is displayed on the display screen 4, and the spatial position state of the object is visually created through the light and shadow and object overlapping effects. The user's upper limb movement is tested using the sports bracelet 6 to achieve the interaction function between the user and the image information on the display screen 4. The task instruction of the test is to grab the task target in the three-dimensional space to test the user's spatial hierarchy perception ability in the three-dimensional space scene.

[0028] In this embodiment, in order to test visual perception, three groups of test schemes are provided here: The first set of test scenarios is as follows: a snowy scene is presented on the display screen 4, and in the corner of the scene there is a polar bear lying prone on the snow. The user needs to distinguish and identify the polar bear in the snow. The sports bracelet 6 detects the movement of the user's arm, and virtual hands are displayed at both lower corners of the display screen 4. The virtual hands in the picture are connected to the user's arm signal. After the user searches for the target in the background environment, the user uses the arm to operate the virtual hands on the display screen 4 to identify and point out the target object in the background picture, and the visual perception is determined according to the selection result; The second set of test scenarios includes: a variety of transportation tools such as bicycles, cars, trucks, tricycles, and buses are presented on the display screen 4. In a random order, the system prompts the user to "point out the car in the picture". The user uses the arm to operate the virtual hand on the display screen 4 to move, identify and point out the car in the picture. Then the system prompts the user to "point out the tricycle in the picture". The user uses the arm to operate the virtual hand on the display screen 4 to move, identify and point out the tricycle in the picture, and so on. This process is repeated until the system prompts the user to complete the operation. The visual perception is determined according to the selection result. The third group of test scenarios is as follows: balloons of different colors are displayed on the display screen 4, floating in the room at different heights and distributed in different positions in the room. The picture is shot from the doorway of the room. The balloons present different spatial hierarchical positions according to their volume, image overlap, and light and shadow direction. The system prompts the user to "pick up the red balloon in the picture". The user uses the arm to operate the virtual hand on the display screen 4 to move. The user needs to control not only the height of the arm movement, but also the length of the arm extended forward, so that the virtual hand moves in three-dimensional space. The visual perception is determined according to the selection result. The above three groups of cases are classic cases of background distinction perception, object recognition perception, and spatial hierarchy perception experiments, and the cases in the inspection process include but are not limited to these.

[0029] Embodiment five: Fig.15As shown, in the visual memory test, two tests, visual associative memory and visual processing memory, were conducted respectively, among which: The image information is displayed on the display screen 4, and the sound information is played in coordination with the earphone 3, and the sound information corresponds to the content of the image information. The task instruction of the test is that the user distinguishes the corresponding image information according to the sound information, so as to test the user's visual associative memory ability; The single-view image information of the target object is displayed on the display screen 4. The task instruction of the test is for the user to distinguish the visual state of the target object after the view angle is converted, so as to check the visual processing memory ability of the user.

[0030] In this embodiment, in order to test visual memory, two sets of test schemes are provided: The first set of test scenarios is as follows: a historical painting image is displayed on the display screen 4, and a background story about the image is played on the earphone 3. After each background story and image is played, the user can choose whether to watch it again according to his or her own understanding ability. Three different sets of historical painting images and background stories are played, and then a background story is played randomly, and the user is asked to select the historical painting image corresponding to the background story. The visual memory is determined based on the selection result. The second test scheme is as follows: a dice is displayed on the display screen 4, and the points on the three sides of the dice can be seen in the viewing angle. The system prompts the user to "please point out the image of the dice after it rolls to the right", and then several groups of different dice images are given on the screen, asking the user to select the image of the perspective transformation after the dice rolls to the right, and the visual memory is determined based on the selection result; The above two groups of cases are classic cases of visual associative memory and visual processing memory tests respectively. The cases in the actual examination process include but are not limited to these.

[0031] Embodiment 6: In the auditory attention test, three tests are conducted: auditory selective attention, auditory sustained attention and auditory divided attention, wherein: The sound scene information of multiple groups of conversations is played through earphone 3. The task instruction of the test is to capture the target conversation information among the multiple groups of conversation information, and the user's auditory selective attention ability is tested through question-answering method; The sound scene information of a long conversation is played through earphone 3. The task instruction of the test is to capture random conversation information in the long conversation information, and the user's auditory sustained attention ability is tested by question and answer; The sound scene information of multiple sound sources is played through earphone 3. The task instruction of the test is to identify various types of sound information in the sounds of multiple sound sources, and the user's auditory attention allocation ability is tested through question-and-answer method.

[0032] In this embodiment, in order to test auditory attention, three groups of test schemes are provided here: The first test scenario is as follows: a conversation at a party is played to the user through earphone 3, where two or three people discuss a topic together, forming a noisy scene with multiple groups of conversations discussing at the same time. Before the start of the conversation, the system prompts "Please pay attention to the topic of tomorrow's travel and answer related questions". The user needs to ignore other topic information in the noisy environment and pay attention to the topic information of tomorrow's travel. At the end of the conversation, the user is asked about the relevant content involved in the topic, and the auditory attention is determined based on the answer result. The second test scenario is as follows: a ten-minute private conversation is played to the user through earphone 3. The content of the conversation involves the topic of tomorrow's travel. Before the conversation starts, the system prompts "Please pay attention to the time and place arrangements of the trip in the topic". The user needs to continuously capture the time and place information in the topic and list them in order. At the end, the user simply repeats the conversation, and the auditory attention is judged based on the answer result; The third test scenario is as follows: a piece of music played by an instrument is played to the user through earphone 3. Before the music starts, the system prompts "Please identify which instruments are involved in the pure music being played". The user needs to identify the sound information of different instruments. After the music played by earphone 3 ends, the user answers the type of instrument heard, and the auditory attention is judged based on the answer result. The above three groups of cases are classic cases of auditory selective attention, auditory sustained attention and auditory divided attention tests, and the cases in the inspection process include but are not limited to these.

[0033] Embodiment 7: In the auditory perception test, three tests are performed, namely, sound recognition perception, sound scene analysis, and information comprehension perception, wherein: Playing a specific type of sound scene information through earphone 3, the task instruction of the test is to identify the sound type in the sound scene and test the user's sound recognition perception ability; The sound scene information generated by the interaction of different objects is played through earphone 3. The task instruction of the test is to describe the scene where the sound occurs, so as to test the user's sound scene analysis ability; The instructions for the target object in the scene are played through the earphone 3, and the user uses the sports bracelet 6 to interact with the image information on the display screen 4, and performs interactive actions in the image scene information on the display screen 4 to test the user's information understanding and perception ability.

[0034] In this embodiment, in order to test the auditory perception, three groups of test schemes are provided here: The first set of test scenarios includes: playing different common environmental sounds such as wind, rain, human voice, and animal calls through earphone 3, and the user says the corresponding type at the end of each playback, and the auditory attention is determined based on the answer result; The second test scenario includes: playing the sound of two glasses hitting each other through earphone 3, and the user needs to describe the scene of the two glasses hitting each other based on the sound; playing the sound of falling water from a waterfall hitting a rock, and the user needs to describe the scene of the waterfall hitting a rock based on the sound, and the auditory attention is judged based on the answer; The third test scenario is as follows: the system instruction message "Please pick up the red cup on the screen" is played through the earphone 3. The user needs to understand the voice information expressed in the sound and use the arm to operate the virtual hand on the display screen 4 to control the virtual hand to pick up the red cup displayed on the display screen 4. The auditory attention is determined according to the operation result. The above three groups of cases are classic cases of sound recognition perception, sound scene analysis and information understanding perception experiments, and the cases in the inspection process include but are not limited to these.

[0035] Embodiment 8: In the auditory memory test, two tests of auditory associative memory and auditory processing memory are performed respectively, wherein: By playing the sound information on the earphone 3, the image information is displayed on the display screen 4 in coordination, and the image information corresponds to the content described by the sound information. The task instruction of the test is that the user distinguishes the corresponding sound information according to the image information, so as to test the user's auditory associative memory ability; By playing the sound information on the earphone 3, the task instruction of the test is to retell or translate the content of the sound information, so as to check the auditory processing memory ability of the user.

[0036] In this embodiment, in order to test auditory memory, two sets of test schemes are provided: The first set of test scenarios is as follows: playing an article introducing a scenic spot through earphone 3, and displaying the background image of the corresponding article on display screen 4. After each article introduction and background image are played, the user can choose whether to watch it again according to his or her comprehension ability. Playing three sets of different scenic spot introduction articles and background images, and then randomly displaying a scenic spot picture, asking the user to select the scenic spot article corresponding to the background image, and judging the auditory memory based on the selection result; The second test scenario is as follows: a phone number is played through earphone 3, and the user is required to repeat the phone number after ten minutes. Depending on the user's ability, a dialect or foreign language is played, and the user is required to translate the content into Mandarin after five minutes. The auditory memory is determined based on the repetition or translation results. The above two groups of cases are classic cases of auditory associative memory and auditory processing memory tests respectively. The cases in the implementation inspection process include but are not limited to these.

[0037] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A seated cognitive function examination cabin based on visual and auditory assessment, comprising a cabin body (1), headphones (3) and a display screen (4), wherein the headphones (3) are used to provide sound scene information for auditory assessment, and the display screen (4) is used to provide picture scene information for visual assessment, characterized in that: A visual platform mechanism (5) is arranged in front of the cockpit body (1), the visual platform mechanism (5) comprises a suspended platform (51) and a stand (59), the display screen (4) is mounted on the top surface of the visual platform mechanism (5), both sides of the suspended platform (51) are fixedly connected to bracket 2 (57), one side of the cockpit body (1) is fixedly connected to bracket 1 (11), the stand (59) is fixedly connected to one side of bracket 2 (57), the top end of the suspended platform (51) is fixedly connected to a rotating shaft (52), the bottom end of the rotating shaft (52) is rotatably connected to a turntable (53), the turntable (53) is fixedly connected to the base of the cockpit body (1), and the top end of the stand (59) is fixedly connected to bracket 3 (590); It also includes an eye tracker (2) and a sports bracelet (6), wherein the eye tracker (2) is arranged on a third bracket (590), and the eye tracker (2) is used to track the movement of the eyes; the sports bracelet (6) is arranged on a second bracket (57), and the sports bracelet (6) is used to sense and analyze the movement of the upper limbs.

2. The seated cognitive function examination cabin based on visual and auditory assessment according to claim 1 is characterized in that: A support frame (56) is fixedly connected to the base of the cockpit body (1); a cover shell (55) is connected to the top of the support frame (56); a spring (58) is fixedly connected to the top surface of the cover shell (55); and a buffer platform (54) is fixedly connected to the top of the spring (58).

3. A method for evaluating cognitive function based on visual and auditory assessment, using the seated cognitive function examination cabin based on visual and auditory assessment according to claim 1 or 2, characterized in that: It includes two parts: visual cognitive assessment method and auditory cognitive assessment method; among them, the visual cognitive assessment method includes: Using an eye tracker (2) to track the eye movements of a user under a visual test on a display screen (4), thereby completing a test of visual attention; Using a sports bracelet (6) to monitor the user's body interactive movements under the visual test of the display screen (4), thereby completing the test of visual perception; Using the display screen (4) to display the image and broadcast the corresponding sound information, testing the associative memory effect between the two, or testing the storage and processing capabilities of the image information, completing the test of visual memory; Auditory cognitive assessment methods include: Use questions and answers to evaluate the accuracy of information acquisition by users in the hearing test with headphones (3), thereby completing the test of auditory attention; Using information expression to evaluate the user's ability to understand and analyze information content under the earphone (3) auditory test, thereby completing the test of auditory perception; The earphone (3) is used to broadcast the sound and display the corresponding image information, to test the associative memory effect between the two, or to test the storage and processing ability of the sound information, thereby completing the test of auditory memory.

4. The method for cognitive function assessment based on visual and auditory assessment according to claim 3, characterized in that: In the visual attention test, three tests were conducted: visual selective attention, visual sustained attention, and visual divided attention. Displaying target information in a specific area of ​​a display screen (4) and displaying environmental information in other areas, using an eye tracker (2) to test the time that a user's gaze is focused on the target information, analyzing the proportion of time that the user spends gazing at the target information area, and testing the user's visual selective attention ability in complex visual scenes; A delayed visual scene is displayed on a display screen (4), wherein target information changes continuously in the delayed visual scene, an eye tracker (2) is used to test the user's line of sight following the change of the target information, the synchronization rate of the user's line of sight following the change of the target information is analyzed, and the user's visual sustained attention ability in the delayed visual scene is tested; Multiple sets of target information are displayed in specific areas on a display screen (4), and environmental information is displayed in other areas. An eye tracker (2) is used to measure the user's gaze percentage and gaze shift frequency in different target information, analyze the user's simultaneous acquisition of information in multiple areas, and test the user's ability to visually allocate attention in a multi-target scene.

5. The method for cognitive function assessment based on visual and auditory assessment according to claim 3, characterized in that: In the visual perception test, a sports bracelet (6) is used to detect the upper limb movement of the user, so as to realize the interaction between the user and the image information on the display screen (4), and three tests are performed respectively, namely, background distinction perception, object recognition perception, and spatial hierarchy perception, among which: Displaying the target information integrated into the background environment on the display screen (4) to form a protective color of the environment information for the target information. The task instruction of the test is to search for the target information in the background environment to test the user's ability to distinguish and perceive the background in a specific visual scene; Displaying multi-target scene information on a display screen (4), the task instruction of the test is to identify the task target in the multi-target scene, and to test the user's object recognition perception ability in the multi-target scene; The three-dimensional space scene information is displayed on the display screen (4), and the spatial position state of the object is visually created through the light and shadow and object overlapping effects. The task instruction of the test is to grasp the task target in the three-dimensional space, and the user's spatial hierarchy perception ability in the three-dimensional space scene is tested.

6. The method for cognitive function assessment based on visual and auditory assessment according to claim 3, characterized in that: In the visual memory test, two tests were conducted: visual associative memory and visual processing memory. The image information is displayed on a display screen (4), and the sound information is played in coordination with the earphone (3), and the sound information is made to correspond to the content expressed by the image information. The task instruction of the test is that the user distinguishes the corresponding image information according to the sound information, so as to test the visual associative memory ability of the user; The single-view image information of the target object is displayed on a display screen (4), and the task instruction of the test is for the user to distinguish the visual state of the target object after the view angle is changed, so as to test the visual processing memory ability of the user.

7. The method for cognitive function assessment based on visual and auditory assessment according to claim 3, characterized in that: In the auditory attention test, three tests were conducted: auditory selective attention, auditory sustained attention, and auditory divided attention. The sound scene information of multiple groups of mixed conversations is played through the earphone (3). The task instruction of the test is to capture the target conversation information among the multiple groups of conversation information, and the user's auditory selective attention ability is tested through question-and-answer method; The sound scene information of a long conversation is played through headphones (3). The task instruction of the test is to capture random conversation information in the long conversation information, and the user's auditory sustained attention ability is tested through question-and-answer method; The sound scene information of multiple sound sources is played through headphones (3), and the task instruction of the test is to identify various types of sound information in the sounds of multiple sound sources, and the user's auditory attention allocation ability is tested through a question-and-answer method.

8. The method for cognitive function assessment based on visual and auditory assessment according to claim 3, characterized in that: In the auditory perception test, three tests are conducted: sound recognition perception, sound scene analysis, and information understanding perception, among which: Playing a specific type of sound scene information through headphones (3), the task instruction of the test is to identify the type of sound in the sound scene, and to test the user's sound recognition perception ability; The sound scene information generated by the interaction of different objects is played through headphones (3). The task instruction of the test is to describe the scene where the sound occurs, so as to test the user's sound scene analysis ability; The user uses the earphone (3) to play instructions for the target object in the scene, and uses the sports bracelet (6) to interact with the image information on the display screen (4), and performs interactive actions in the image scene information on the display screen (4), thereby testing the user's information understanding and perception ability.

9. The method for cognitive function assessment based on visual and auditory assessment according to claim 3, characterized in that: In the auditory memory test, two tests were conducted, namely auditory associative memory and auditory processing memory, where: The sound information is played through the earphone (3), and the image information is displayed in coordination with the display screen (4), and the image information corresponds to the content expressed by the sound information. The task instruction of the test is that the user distinguishes the corresponding sound information according to the image information, so as to test the user's auditory associative memory ability; By playing sound information on headphones (3), the test task instruction is to retell or translate the content of the sound information, so as to test the user's auditory processing and memory ability.

Images