Unilateral visual spatial perception parameter acquisition device based on picture description and reading test

Abstract

The present application discloses a unilateral visual spatial perception parameter acquisition device based on picture description and reading test, which relates to the field of human-computer interaction technology. The device includes: a picture description test device, a reading test device and a parameter acquisition device; the picture description test device is used to perform a picture description test, and obtain picture description parameters according to the eye focus point movement trajectory and the cumulative gaze time; the reading test device is used to perform a reading test, and obtain reading parameters according to eye gaze, gestures and voice input; the visual spatial perception parameter acquisition device is used to obtain the target unilateral visual spatial perception parameters according to the picture description parameters and the reading parameters. The present application provides a data basis for a comprehensive assessment of the unilateral visual spatial perception ability of a subject, accurately obtains visual spatial perception data, and uses this comprehensive data to accurately identify the bias or deficiency or unilateral neglect of visual spatial perception ability.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to a device for acquiring unilateral visual space perception parameters based on picture description and reading tests. Background Art

[0002] In cognitive science and human-computer interaction research, unilateral visual-spatial perception ability (including left-side visual-spatial perception ability and right-side visual-spatial perception ability) refers to the degree to which an individual pays attention to visual or spatial information on a specific side. Accurately obtaining unilateral visual-spatial perception parameters is crucial to understanding an individual's cognitive state, especially when it is necessary to assess whether there is unilateral neglect disorder. This disorder usually manifests as an individual's neglect of information on one side in visual or spatial processing. In order to achieve accurate assessment of unilateral visual-spatial perception ability and assist in the diagnosis of unilateral neglect, it is particularly important to develop equipment that can obtain unilateral visual-spatial perception parameters.

[0003] At present, the methods for obtaining unilateral visual-spatial perception parameters mainly rely on paper-and-pencil tests, computerized cognitive tasks, and eye tracking technology. Subjects are usually required to complete tasks such as image description, visual search, or reading, and researchers use these tasks to indirectly infer their unilateral visual-spatial perception ability. These methods include recording the subject's gaze time in a specific area, eye movement trajectory and other data to measure the distribution of their visual-spatial perception ability. However, these methods have certain limitations in capturing the dynamic changes and comprehensive analysis of unilateral visual-spatial perception ability.

[0004] Although the existing test methods can obtain data on unilateral visual spatial perception ability to a certain extent, there are still several major problems. Traditional evaluation methods are not only defective in evaluating visual spatial perception ability, but most importantly, they are defective in the time and space accuracy of the evaluation. This application is to solve this problem. For example, the space is divided using finite cells to accurately obtain the visual spatial perception parameters of the subject to be tested, and then the distribution of the unilateral visual spatial perception ability of the individual in different situations is comprehensively evaluated, as well as which spatial range of the visual field the subject to be tested has defects in. At the same time, eye tracking can calculate the gaze time of the subject to be tested in the finite cells, as well as the execution time to complete a certain image tracking task, so as to obtain the unilateral visual spatial perception parameters of the subject to be tested, and according to these parameters, the unilateral visual spatial perception ability and cognitive ability of the subject to be tested can be analyzed. The accuracy of traditional methods is limited, and it is difficult to achieve accurate quantification of visual spatial perception parameters. Secondly, these methods can usually only reflect the state of visual spatial perception ability in a single task, and cannot comprehensively evaluate the distribution of unilateral visual spatial perception ability of individuals in different situations. Finally, the existing methods are insufficient in data integration and real-time analysis, and it is difficult to comprehensively capture and analyze data of multiple dimensions. Therefore, how to accurately obtain visual spatial perception data becomes an urgent problem to be solved.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The purpose of this application is to provide a unilateral visual-spatial perception parameter acquisition device based on picture description and reading test, aiming to solve the technical problem of how to accurately acquire visual-spatial perception data.

[0007] To achieve the above objectives, the present application proposes a device for acquiring unilateral visual spatial perception parameters based on picture description and reading test, the device comprising:

[0008] The device includes a picture description test device, a reading test device and a parameter acquisition device;

[0009] The picture description test device is used to perform a picture description test and obtain picture description parameters according to the eyeball focus point movement trajectory and the accumulated fixation time;

[0010] The reading test device is used to perform a reading test and obtain reading parameters based on eye gaze, gestures and voice input;

[0011] The visual space perception parameter acquisition device is used to obtain the target unilateral visual space perception parameter according to the picture description parameter and the reading parameter.

[0012] In one embodiment, the picture description testing device includes a first display module, a recording module and a first parameter acquisition module;

[0013] The first display module is used to display pictures and provide prompts for describing the pictures;

[0014] The first display module is further used to define cells on the picture according to a preset size;

[0015] The recording module is used to record the cumulative gaze time, the movement trajectory of the eyeball focus point, and the number of grids crossed by the movement trajectory of the eyeball focus point on the picture;

[0016] The first parameter acquisition module is used to obtain a first unilateral visual space perception parameter according to the number of grids, and to obtain a second unilateral visual space perception parameter according to the accumulated fixation time.

[0017] In one embodiment, the picture description testing device further includes an image processing module, a second parameter acquisition module, a picture description comprehensive parameter acquisition module, and a picture description parameter acquisition module;

[0018] The image processing module is used to obtain the region of interest in the image according to the eye focus point movement trajectory, the accumulated fixation time and a preset fixation time threshold;

[0019] The second parameter acquisition module is used to obtain a third unilateral visual space perception parameter according to the region of interest and the accumulated fixation time;

[0020] The picture description comprehensive parameter acquisition module is used to obtain a picture description comprehensive parameter according to the first unilateral visual space perception parameter, the second unilateral visual space perception parameter and the third unilateral visual space perception parameter;

[0021] The picture description parameter acquisition module is used to obtain the picture description parameter according to the ratio of the picture description comprehensive parameter to the preset picture description comprehensive parameter.

[0022] In one embodiment, the picture includes a left area and a right area, the number of grids includes a first number of grids and a second number of grids, the cumulative gaze time includes a first cumulative gaze time and a second cumulative gaze time, and the recording module includes a trajectory recording unit, a grid recording unit, and a gaze time recording unit;

[0023] The trajectory recording unit is used to record the movement trajectory of the eyeball focus point;

[0024] The grid recording unit is used to record the number of the first grids and the number of the second grids that the eyeball focus point movement trajectory passes through on the left area and the right area;

[0025] The gaze time recording unit is configured to record the first cumulative gaze time in the left area and the second cumulative gaze time in the right area.

[0026] In one embodiment, the image processing module includes a heat map unit, an activity map unit, and a region division unit;

[0027] The heat map unit is used to obtain an eye tracking heat map according to the eye focus point movement trajectory and the accumulated fixation time;

[0028] The activity map unit is used to obtain an average activity map based on a plurality of the eye tracking heat maps;

[0029] The region division unit is used to select a region in the average activity map where the cumulative gaze time is greater than or equal to a preset gaze time threshold as a region of interest.

[0030] In one embodiment, the region of interest includes a left region of interest and a right region of interest, and the second parameter acquisition module includes a left ratio unit, a right ratio unit and a parameter acquisition unit;

[0031] The left ratio unit is used to calculate the left gaze time ratio of the cumulative gaze time in the left region of interest to the cumulative gaze time in all regions;

[0032] The right ratio unit is used to calculate the right gaze time ratio of the cumulative gaze time in the right region of interest to the cumulative gaze time in all regions;

[0033] The parameter acquisition unit is used to obtain a third unilateral visual space perception parameter according to the left gaze time ratio and the right gaze time ratio.

[0034] In one embodiment, the reading test device includes a second display module, a detection module, a locking module, a voice processing module and a third parameter acquisition module;

[0035] The second display module is used to display a preset number of characters within a preset range, the characters including described characters and undescribed characters, the undescribed characters are displayed in a first preset color, and the described characters are displayed in a second preset color;

[0036] The detection module is used to detect gaze and gestures to obtain detection results;

[0037] The locking module is used to lock the text according to the detection result to obtain a locking result;

[0038] The speech processing module is used to receive speech input describing the text;

[0039] The third parameter acquisition module is used to obtain correct item parameters, incorrect item parameters and ignored item parameters according to the voice input, the locking result, the preset duration, the preset number and the undescribed text.

[0040] In one embodiment, the reading test device further includes a fourth parameter acquisition module and a reading parameter acquisition module;

[0041] The fourth parameter acquisition module is used to obtain the comprehensive reading parameter according to the preset range, the undescribed text, the preset format, the correct item parameter, the incorrect item parameter and the ignored item parameter;

[0042] The reading parameter acquisition module is used to obtain the reading parameter according to the ratio of the reading comprehensive parameter to the preset reading comprehensive parameter.

[0043] In one embodiment, the third parameter acquisition module includes a correct item parameter acquisition unit, an incorrect item parameter acquisition unit, and an ignored item parameter acquisition unit;

[0044] The correct item parameter acquisition unit is used to adjust and record the correct item parameters and set the locked text as the described text when the voice input corresponds to the locked result;

[0045] The error item parameter acquisition unit is used to adjust and record the error item parameter when the voice input does not correspond to the locking result;

[0046] The ignored item parameter acquisition unit is used to stop displaying the text after a preset time period, and calculate the ratio of the number of the undescribed text to the preset number to obtain the ignored item parameter.

[0047] In one embodiment, the preset range includes a left range, a front range, and a right range, and the fourth parameter acquisition module includes a left range parameter acquisition unit, a right range parameter acquisition unit, and a reading comprehensive parameter acquisition unit;

[0048] The left range parameter acquisition unit is used to obtain the left range parameter according to the number of the undescribed characters in the left range and the front range, and record the left range parameter in a preset form;

[0049] The right range parameter acquisition unit is used to obtain the right range parameter according to the number of the undescribed characters in the right range and the front range, and record the right range parameter in the preset form;

[0050] The comprehensive reading parameter acquisition unit is used to obtain the comprehensive reading parameter according to the correct item parameter, the incorrect item parameter, the ignored item parameter, the left range parameter and the right range parameter.

[0051] This application proposes a unilateral visual spatial perception parameter acquisition device based on picture description and reading test, which has at least the following technical effects:

[0052] The device includes a picture description test device, a reading test device and a parameter acquisition device; the picture description test device is used to perform a picture description test, and obtains picture description parameters according to the eye focus point movement trajectory and the cumulative fixation time; the reading test device is used to perform a reading test, and obtains reading parameters according to eye fixation, gestures and voice input; the visual space perception parameter acquisition device is used to obtain the target unilateral visual space perception parameters according to the picture description parameters and the reading parameters. The device includes a picture description test device, a reading test device and a visual space perception parameter acquisition device, forming a comprehensive system to evaluate the unilateral visual space perception ability of the subject. The picture description test device generates picture description parameters by monitoring the eye focus point movement trajectory and the cumulative fixation time of the subject, and these parameters reveal the visual attention of the subject to different areas when observing the picture. The reading test device records the subject's eye fixation point, gesture operation and voice input, and obtains reading parameters, which reflect the subject's processing method of text and the distribution of visual space perception ability during the reading process. Finally, the visual space perception parameter acquisition device integrates the picture description parameters and the reading parameters to calculate the target unilateral visual space perception parameters. This process obtains the subject's unilateral visual space perception data. Overall, the device provides a data basis for comprehensive assessment of subjects' unilateral visual-spatial perception ability by collecting data from two dimensions: visual observation and reading behavior. It accurately obtains visual-spatial perception data. The use of this comprehensive data can accurately identify biases or deficiencies in visual-spatial perception ability, providing strong data support and scientific basis for cognitive science research, clinical diagnosis and personalized intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0054] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0055] Figure 1 A schematic diagram of the module structure provided for the first embodiment of the device for acquiring unilateral visual space perception parameters based on picture description and reading test of the present application;

[0056] Figure 2 A schematic diagram of a reading test provided in Embodiment 1 of the device for acquiring unilateral visual spatial perception parameters based on picture description and reading test of the present application;

[0057] Figure 3 A schematic diagram of the module structure provided for the second embodiment of the device for acquiring unilateral visual space perception parameters based on picture description and reading test of the present application;

[0058] Figure 4 This is a thermal diagram of eye tracking provided for the second embodiment of the device for acquiring unilateral visual spatial perception parameters based on picture description and reading test of the present application.

[0059] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0061] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0062] In the field of cognitive science and human-computer interaction, the measurement of unilateral visual-spatial perception ability is very important for understanding an individual's cognitive state and diagnosing unilateral neglect disorder. Currently, researchers mainly use paper-and-pencil tests, computer tasks, and eye tracking technology to evaluate unilateral visual-spatial perception ability, but these methods are limited in accuracy, comprehensiveness, and real-time analysis. Although they can provide certain data, these traditional methods are difficult to accurately quantify visual-spatial perception parameters, and usually only reflect the state of visual-spatial perception ability in specific tasks, which is not enough to fully capture and analyze the distribution of visual-spatial perception ability of individuals in different situations.

[0063] The main solution of the embodiment of the present application is: through the collaborative work of a picture description test device, a reading test device and a visual-spatial perception parameter acquisition device, a comprehensive assessment of the subject's unilateral visual-spatial perception ability. By monitoring the subject's eye movements, gestures and voice input during the process of observing pictures and reading, data on visual attention and text processing methods are collected, and then these parameters are integrated to calculate unilateral visual-spatial perception parameters. This multi-dimensional data collection device not only provides a data basis for a comprehensive assessment of the distribution of the subject's visual-spatial perception ability, but the collected visual-spatial perception data can be used to accurately identify the bias or deficiency of visual-spatial perception ability, providing strong data support and scientific basis for cognitive science research, clinical diagnosis and personalized intervention.

[0064] It should be noted that the execution subject of the embodiments of the present application may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a mixed reality head-mounted display device (HoloLens 2 or HoloLens 3), an augmented reality device (AR), a mixed reality device (MR), 3D glasses, etc. The following takes a mixed reality head-mounted display device as an example to illustrate this embodiment and the following embodiments.

[0065] Based on this, the embodiment of the present application provides a unilateral visual space perception parameter acquisition device based on picture description and reading test, referring to Figure 1 , Figure 1 This is a schematic diagram of the module structure of the first embodiment of the unilateral visual space perception parameter acquisition device based on picture description and reading test in this application.

[0066] In this embodiment, the unilateral visual space perception parameter acquisition device based on picture description and reading test includes a picture description test device 10, a reading test device 20 and a parameter acquisition device 30:

[0067] The picture description test device 10 is used to perform a picture description test and obtain picture description parameters according to the eye focus point movement trajectory and the accumulated fixation time;

[0068] It should be noted that in the picture description test, the subjects need to observe and describe a picture within a limited time. The purpose of the picture description test is to record the subjects' eye movement data, such as the focus of their gaze and the duration of their gaze, through the description process, so as to analyze the distribution and detail of their visual spatial perception ability of the picture. This test uses the subject's description content and eye movement trajectory to judge their understanding of the picture and the concentration of their visual spatial perception. The eye focus point movement trajectory refers to the movement trajectory of the subject's eye focus point when observing the picture during the test, which shows the movement path of the subject's gaze on the picture. These trajectories reflect the subject's visual focus and can reveal the distribution of their visual spatial perception ability. The detection and recording of the movement trajectory of the eye's focus point relies on advanced eye tracking technology: First, use an eye tracker or eye tracker. These devices are equipped with an infrared camera and a light source to detect the position of the eye by tracking the reflected light points of the eye; the camera of the eye tracker captures the image of the eye, and the built-in sensor measures the position of the reflection point of the eye, thereby monitoring and recording the movement of the eye in real time; the system analyzes the image of the eye through image processing technology, identifies the reflection points of the pupil and cornea, and converts the focus point of the eye into coordinate data on the screen; these data are recorded in chronological order to generate an eye movement trajectory diagram that shows the movement path and gaze point of the eye; the recorded data can be used to analyze in detail the distribution of the subject's visual spatial perception ability when viewing pictures or reading texts, such as determining which areas are more frequently gazed and which areas are ignored, thereby providing an in-depth understanding of the subject's visual behavior and visual spatial perception ability. Cumulative fixation time refers to the cumulative fixation time of the subject on a specific area during the test. By recording and accumulating the time the subject stays on a certain part of the picture, the degree of visual spatial perception of the area can be judged. The longer the fixation time, the higher the subject's attention to the area. The process of detecting and recording cumulative gaze time involves multiple steps: first, using eye trackers or eye trackers, these devices monitor the position and movement of the eyeballs in real time through infrared cameras and sensors; the system continuously tracks the position data of the eyeballs to capture changes in the line of sight; when the eyeballs stay stably in the same position for more than a set time (such as 200 milliseconds), it is defined as a "gaze" event; the system records the start and end time of each gaze event and calculates the duration of each gaze; then, the duration of all gaze events is accumulated to obtain the cumulative gaze time on a specific area or the entire picture, which reflects the subject's degree of attention to each area during the viewing process; the recording module stores these cumulative gaze time data in real time to ensure the accuracy and completeness of the data; finally, these data are used to analyze the subjects' visual spatial perception ability and cognitive processing patterns, providing detailed gaze time information.Picture description parameters are comprehensive parameters obtained from the picture description test, including eye focus movement trajectory, cumulative gaze time, description content and other data. Through these parameters, the subjects' visual-spatial perception ability distribution and picture comprehension can be evaluated, and their visual-spatial perception ability level can be quantitatively analyzed.

[0069] It can be understood that the picture description test device 10 is a device for obtaining visual-spatial perception parameters, which collects data about the subject's visual-spatial perception ability by asking the subject to observe a picture and describe it. Specifically, the device records the movement trajectory of the subject's eye focus, that is, the movement path of the subject's eyeball when observing the picture, which reflects the distribution of his visual focus. At the same time, the device also records the cumulative gaze time, that is, the cumulative gaze time of the subject on each part of the picture, showing the degree of his attention to these areas. These data ultimately generate picture description parameters, which will be used to further analyze and evaluate the subject's visual-spatial perception ability status. It should be noted that the device is only responsible for data acquisition, and the actual unilateral neglect disorder or unilateral visual-spatial perception ability assessment is performed by professionals based on these parameters.

[0070] The reading test device 20 is used to perform a reading test and obtain reading parameters based on eye gaze, gestures and voice input;

[0071] It should be noted that in the reading test, the subjects need to perform a series of reading tasks, such as reading text or answering questions related to the text content. The purpose of the test is to obtain data about the subjects' visual spatial perception ability and cognitive status by recording the subjects' behaviors and reactions during the reading process. Eye fixation refers to the time the subjects' eyes stay on a specific text or area during the reading process. By tracking and recording these fixation points, the subjects' visual attention patterns and information processing methods can be analyzed. Gestures refer to hand movements that subjects may use when taking reading tests, such as sliding, clicking, pointing to the left, pointing to the right, pointing up, pointing down or pointing to a specific area, etc. These gestures can reflect the subjects' interactive methods for reading content and their information processing behaviors. Voice input refers to the subjects' description of text, input of information or answering of questions by speaking. This input method can help analyze the subjects' understanding and expression abilities and provide additional interactive data. Reading parameters are comprehensive data obtained from the reading test, including eye fixation time, gestures and voice input content. These parameters are used to describe the subjects' visual spatial perception ability distribution, reading habits and information processing abilities during the reading process, providing a basis for further analysis of their cognitive status.

[0072] It is understandable that the reading test device is a device for obtaining visual-spatial perception parameters during the reading process. It requires the subject to complete a reading task and records the subject's behavioral data such as gaze, gestures, and voice input in real time. Specifically, gaze records the subject's visual stop point and gaze time on the text, reflecting the distribution of his visual attention when reading; gestures record the subject's interactive actions on the screen during the reading process, such as sliding, clicking, or pointing to a specific text area, etc. These actions can provide clues to how they process the content; voice input captures the subject's oral input content to help analyze his understanding and expression ability. By integrating these data, the device generates reading parameters, which provides key data support for further analysis of the subject's visual-spatial perception ability status.

[0073] In the specific implementation, three tests are taken and the average score is calculated based on the reading parameters. It is planned to use the correct score of ordinary test subjects and the variance of the score (SD value) as the reference value, and perform 2-score normalization to obtain the target unilateral visual space perception parameters. Assuming that the average correct score of ordinary test subjects in this evaluation scheme is A, and the average correct score of the current test subjects is B, the calculation formula is B / A*100%. Based on the normalized percentage system, mild unilateral spatial neglect occurs below the 95% confidence level (that is, the score is lower than 95%). Moderate unilateral spatial neglect occurs below the 68% confidence level (that is, the score is lower than 68%), and severe unilateral spatial neglect occurs below the 34.13% confidence level (the score is planned to be set below 35%). Among them, the above distinction calculation is based on the sample normal distribution.

[0074] As an example, a reading test device includes a second display module, a detection module, a locking module, a voice processing module and a third parameter acquisition module; the second display module is used to display a preset number of characters within a preset range, and the characters include described characters and undescribed characters, the undescribed characters are displayed in a first preset color, and the described characters are displayed in a second preset color; the detection module is used to detect eye gaze and gestures to obtain a detection result; the locking module is used to lock the characters according to the detection result to obtain a locking result; the voice processing module is used to receive voice input describing the characters; the third parameter acquisition module is used to obtain correct item parameters, incorrect item parameters and ignored item parameters according to the voice input, the locking result, a preset duration, the preset number and the undescribed characters.

[0075] The preset range refers to the area or position range where the text is displayed during the test. For example, on a screen, the preset range may be the entire screen area or a part of the screen area where the text is displayed, which is used to control the display position and area of ​​the text. The preset number refers to the total number of texts displayed in the test. For example, 20 texts can be displayed in the test, and half of them are generated on the left and right sides of the subject. The Chinese characters are placed in the form of a curved surface distribution in space, including described and undescribed texts (all undescribed texts are initially undescribed texts). Described texts refer to texts that the subject has answered by description (correct description) or input. These texts will be marked as described in the test. In this embodiment, they are displayed in blue to distinguish between the subject's input and undescribed text. Undescribed texts refer to texts that the subject has not described or input during the test. These texts are displayed in a specific color in the test (displayed in gray in this embodiment) to distinguish which texts have been described and which have not been described. The first preset color refers to the color used to mark undescribed texts. For example, undescribed texts can be displayed in red to facilitate visual distinction. The second preset color refers to the color used to mark described texts. For example, the described text can be displayed in green to distinguish which text has been described by the subject. The locking result refers to determining which text has been focused on or operated based on the gaze and gesture detection results, and marking these texts as locked. The locking result helps identify the subject's actual attention to the text. The preset time refers to the length of time set in the test, which is used to measure the time it takes for the subject to complete the description task. For example, the preset time can be 5 minutes, which is used to evaluate the subject's performance within the specified time. The correct item parameters refer to the number or content of the text correctly described by the subject based on the voice input and the locking results. These parameters are used to measure the subject's ability to accurately recognize and describe the text content. The error item parameters refer to the number or content of the text incorrectly described by the subject based on the voice input and the locking results. These parameters are used to identify errors or omissions in the subject's description. The ignored item parameters refer to the evaluation of which text the subject did not describe based on the information and detection results of the undescribed text. These parameters are used to identify the text parts that the subject ignored during the reading process.

[0076] Please refer to Figure 2 , Figure 2A reading test schematic diagram provided for the first embodiment of the unilateral visual spatial perception parameter acquisition device based on picture description and reading test of this application, which shows a subject, undescribed text, blue arrows and red text, wherein the subject is the main body in the figure, an individual participating in the reading test, whose task is to read the displayed text and describe it. Undescribed text (gray) represents text that has not been described or recognized by the subject in the test. These texts are displayed in gray to distinguish them from the described texts, thereby clearly indicating the progress of the subject in the reading process and the distribution of visual spatial perception ability. In the figure, blue arrows are used to indicate the direction of the subject's gaze movement or the area of ​​concern. These arrows may indicate the movement path of the subject's sight, helping to analyze the subject's visual scanning pattern and visual spatial perception ability transfer during the reading process. The figure also includes a specific display area, which is a text display area for the subject to perform reading and description tasks. The area displays text according to a preset range and quantity for the subject to observe and operate. The locking result is indicated by red text or other visual marks in the figure, that is, the text that the subject successfully locks through gaze and gestures, which reflects the subject's visual spatial perception ability for specific text.

[0077] In this figure, you can see how the reading test device works. It uses the second display module to display a certain number of words within a preset range. These words are divided into two categories: described and undescribed. Undescribed words are displayed in a first preset color (such as gray), while described words are displayed in a second preset color (such as blue). Such color distinction helps to clearly track the reading and description progress of the subject. The detection module is responsible for capturing the subject's gaze position and gesture movements in real time and generating detection results. These results reflect the subject's visual focus and interactive behavior. The locking module locks the words that the subject has focused on based on the detection results and generates locking results, which helps to identify the subject's actual attention to the words. The voice processing module records and processes the subject's voice description of these words and analyzes their oral content. The third parameter acquisition module integrates voice input, locking results, preset duration, preset number of words and undescribed words to calculate correct item parameters, incorrect item parameters and ignored item parameters. These parameters comprehensively reflect the distribution of the subject's reading visual spatial perception ability, content understanding and interactive effect, and provide data support for subsequent analysis. In general, Figure 2 It demonstrates how the reading test device can obtain the visual-spatial perception ability and behavioral parameters of the subjects in the reading test through the coordinated work of multiple modules, thereby evaluating the distribution and efficiency of their unilateral visual-spatial perception ability.

[0078] The reading test device is composed of multiple modules working together to obtain the visual spatial perception parameters and behavioral parameters of the subject during the reading process in detail. First, the second display module displays a certain number of words within a preset range. These words are divided into two categories: described and undescribed. The undescribed words are displayed in a first preset color (such as gray), and the described words are displayed in a second preset color (such as blue) to clearly distinguish the reading progress of the subject. The detection module captures the subject's gaze position and gesture movements in real time, generates detection results, and reflects its visual focus and interactive behavior. The locking module locks the words that the subject has focused on according to the detection results, generates locking results, and determines which words are successfully recognized and operated. Next, the voice processing module records and processes the subject's voice description of these words and analyzes their oral content. Finally, the third parameter acquisition module integrates voice input, locking results, preset duration (such as the time specified by the reading task), preset number of words, and undescribed words, and calculates correct item parameters (such as the number or proportion of accurately described words), error item parameters (such as the number or proportion of incorrectly described words) and ignored item parameters (such as the number or proportion of undescribed words). These parameters comprehensively reflect the subjects' reading visual-spatial perception ability distribution, content comprehension and interaction effects, providing data support for subsequent analysis.

[0079] As an example, the third parameter acquisition module includes a correct item parameter acquisition unit, an incorrect item parameter acquisition unit and an ignored item parameter acquisition unit; the correct item parameter acquisition unit is used to adjust and record the correct item parameters when the voice input corresponds to the locking result, and set the locked text to the described text; the incorrect item parameter acquisition unit is used to adjust and record the incorrect item parameters when the voice input does not correspond to the locking result; the ignored item parameter acquisition unit is used to stop displaying the text after a preset time period, and calculate the ratio of the number of undescribed texts to the preset number to obtain the ignored item parameters.

[0080] The third parameter acquisition module is composed of a correct item parameter acquisition unit, an incorrect item parameter acquisition unit and an ignored item parameter acquisition unit, and is specifically used to calculate and adjust various parameters in the reading test in detail. First, when the correct item parameter acquisition unit detects that the voice input content corresponds to the locked result, the item is regarded as a correct description, the count or proportion of the correct item parameter is increased, and the corresponding text is converted from an undescribed state to a described state. Secondly, the incorrect item parameter acquisition unit is responsible for identifying the item as an incorrect description when the voice input content does not match the locked result, increasing the count or proportion of the incorrect item parameter, and recording the subject's error. Finally, after the preset time period ends, the ignored item parameter acquisition unit stops the text display, calculates the number of undescribed texts, compares it with the total preset number, and obtains the ignored item parameter, which reflects the proportion of text that the subject has not processed or ignored within the specified time. Through the coordinated operation of these three units, the third parameter acquisition module can comprehensively record and analyze the reading behavior of the subject and provide accurate visual spatial perception ability distribution data.

[0081] As an example, the reading test device also includes a fourth parameter acquisition module and a reading parameter acquisition module; the fourth parameter acquisition module is used to obtain a comprehensive reading parameter according to the preset range, the undescribed text, the preset form, the correct item parameter, the incorrect item parameter and the ignored item parameter; the reading parameter acquisition module is used to obtain a reading parameter according to a ratio of the comprehensive reading parameter to a preset comprehensive reading parameter.

[0082] The preset format means that when calculating and comparing comprehensive reading parameters, in order to simplify data processing and comparison, the parameters are displayed in a specific format. In this embodiment, the preset format is the format of x / 10, that is, the values ​​of each parameter are standardized to a ratio between 0 and 1, which is usually achieved by dividing the parameter value by 10. This format makes the data within the range of 0 to 1, which is convenient for subsequent calculation and comparison. Comprehensive reading parameters refer to an overall parameter obtained by comprehensively calculating all relevant data within a preset range, including the number of undescribed texts, correct item parameters, incorrect item parameters, and ignored item parameters. These parameters comprehensively reflect the overall performance of the subject in the reading test, including their processing of texts, accuracy, and omissions. Preset comprehensive reading parameters refer to the comprehensive reading parameters obtained when the device performs a reading test on an ordinary subject. These parameters are calculated based on the actual test data of the ordinary subject and serve as a benchmark for normal reading performance for comparison with the test results of other subjects. Reading parameters refer to the results obtained by calculating the ratio of comprehensive reading parameters to preset comprehensive reading parameters. This parameter is used to quantify the reading performance of the subjects and compare it with the common benchmark to evaluate their visual-spatial perception ability, relative deviations in accuracy and omissions. The calculation of reading parameters enables the reading performance of different subjects to be standardized and effectively compared.

[0083] The reading test device also includes a fourth parameter acquisition module and a reading parameter acquisition module, which are also used to calculate and analyze the results of the reading test. The fourth parameter acquisition module is responsible for comprehensively calculating the reading comprehensive parameter. It uses all data within a preset range, including the number of undescribed words, correct item parameters, incorrect item parameters, and ignored item parameters, to generate an overall reading comprehensive parameter in a preset form (such as a ratio standardized to x / 10). This parameter reflects the overall performance of the subject during the reading process. Subsequently, the reading parameter acquisition module obtains a reading parameter by calculating the ratio of the calculated reading comprehensive parameter to the preset reading comprehensive parameter. This reading parameter is used to evaluate the distribution of the subject's reading ability and visual spatial perception ability, and quantify its performance with the normal benchmark to provide a detailed performance evaluation.

[0084] As an example, the preset range includes a left range, a front range and a right range, and the fourth parameter acquisition module includes a left range parameter acquisition unit, a right range parameter acquisition unit and a comprehensive reading parameter acquisition unit; the left range parameter acquisition unit is used to obtain the left range parameters according to the number of undescribed characters in the left range and the front range, and record the left range parameters in a preset form; the right range parameter acquisition unit is used to obtain the right range parameters according to the number of undescribed characters in the right range and the front range, and record the right range parameters in the preset form; the reading comprehensive parameter acquisition unit is used to obtain the reading comprehensive parameters according to the correct item parameters, the incorrect item parameters, the ignored item parameters, the left range parameters and the right range parameters.

[0085] The left range refers to the area in the test area located to the left of the subject's line of sight. In the reading test, this usually includes the left half of the screen or paper, and is used to record the situation where the subject does not describe the text in this area. The front range refers to the area directly in front of the subject's line of sight. This is usually the central part of the test area, including the subject's main visual attention area. The right range refers to the area in the test area located to the right of the subject's line of sight. In the reading test, this usually includes the right half of the screen or paper, and is used to record the situation where the subject does not describe the text in this area. The left range parameter refers to the parameter calculated based on the number of undescribed texts in the left range and the front range. It reflects the proportion of texts that the subject does not describe in the left range, and is recorded in a preset form (such as a standardized ratio) for analyzing the distribution of the subject's reading and visual spatial perception abilities in the left area. The right range parameter refers to the parameter calculated based on the number of undescribed texts in the right range and the front range. It reflects the proportion of texts that the subject does not describe in the right range, and is recorded in a preset form (such as a standardized ratio) for analyzing the distribution of the subject's reading and visual spatial perception abilities in the right area.

[0086] The fourth parameter acquisition module is composed of a left range parameter acquisition unit, a right range parameter acquisition unit and a reading comprehensive parameter acquisition unit, which is used to calculate and record the various parameters in the reading test in detail. First, the left range parameter acquisition unit is responsible for calculating and recording the number of undescribed characters in the left range, combining the undescribed text data in the front range, generating a left range parameter reflecting the undescribed situation of the subject in the left range, and recording it in a preset form (such as a standardized ratio). Similarly, the right range parameter acquisition unit calculates and records the number of undescribed characters in the right range, and also combines the data of the front range to obtain the right range parameter, and records it in a preset form. These two parameters reveal the distribution and omission of the subject's visual spatial perception ability in the left and right visual ranges, respectively. Finally, the reading comprehensive parameter acquisition unit combines the correct item parameters, the wrong item parameters, the ignored item parameters, the left range parameters and the right range parameters to calculate a comprehensive reading comprehensive parameter for a comprehensive evaluation of the subject's reading performance.

[0087] The visual space perception parameter acquisition device 30 is used to obtain the target unilateral visual space perception parameter according to the picture description parameter and the reading parameter.

[0088] It should be noted that the target unilateral visual-spatial perception parameters refer to the measurement indicators specific to unilateral visual-spatial perception ability calculated from the picture description parameters and reading parameters. These parameters can be used to evaluate the distribution of subjects' visual-spatial perception ability on a specific side in visual or cognitive tasks. For example, the target unilateral visual-spatial perception parameters may reveal the degree to which subjects pay attention to left or right information in visual tasks, helping to identify whether there is a bias or imbalance in their visual-spatial perception ability. These parameters provide an in-depth analysis of the subjects' visual-spatial perception ability levels on different sides by combining data from picture description and reading tests.

[0089] It is understandable that the visual-spatial perception parameter acquisition device 30 is intended to obtain the target unilateral visual-spatial perception parameters by analyzing the picture description parameters and the reading parameters. The device combines the visual-spatial perception data obtained in the picture description test (such as the movement trajectory and fixation time of the subject's line of sight on the picture) with the visual-spatial perception data obtained in the reading test (such as the fixation time, gesture operation and voice input of each area) to calculate the distribution of the subject's visual-spatial perception ability on a specific side. Specifically, the device integrates these data to generate a comprehensive parameter that reflects the subject's degree of attention to the left or right visual information. In this way, the device can provide accurate measurements of the subject's unilateral visual-spatial perception ability, and provide valuable data and information for analyzing and diagnosing problems related to visual-spatial perception ability.

[0090] This embodiment provides a unilateral visual spatial perception parameter acquisition device based on picture description and reading test, the device includes a picture description test device, a reading test device and a parameter acquisition device; the picture description test device is used to perform a picture description test, and obtains picture description parameters according to the eye focus point movement trajectory and the cumulative fixation time; the reading test device is used to perform a reading test, and obtains reading parameters according to eye fixation, gestures and voice input; the visual spatial perception parameter acquisition device is used to obtain the target unilateral visual spatial perception parameters according to the picture description parameters and the reading parameters. The device includes a picture description test device, a reading test device and a visual spatial perception parameter acquisition device, forming a comprehensive system to evaluate the unilateral visual spatial perception ability of the subject. The picture description test device generates picture description parameters by monitoring the eye focus point movement trajectory and the cumulative fixation time of the subject, and these parameters reveal the visual attention of the subject to different areas when observing the picture. The reading test device records the subject's eye fixation point, gesture operation and voice input, and obtains reading parameters, which reflect the way the subject handles text and the distribution of visual spatial perception ability during the reading process. Finally, the visual-spatial perception parameter acquisition device integrates the image description parameters and reading parameters to calculate the unilateral visual-spatial perception parameters of the target. This process obtains the unilateral visual-spatial perception data of the subject. Overall, the device provides a data basis for a comprehensive assessment of the subject's unilateral visual-spatial perception ability by collecting data from two dimensions: visual observation and reading behavior. It accurately acquires visual-spatial perception data. This comprehensive data can accurately identify the bias or deficiency of visual-spatial perception ability, providing strong data support and scientific basis for cognitive science research, clinical diagnosis, and personalized intervention.

[0091] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 3 , Figure 3 This is a schematic diagram of the module structure of the second embodiment of the unilateral visual space perception parameter acquisition device based on picture description and reading test of the present application. The picture description test device 10 in the unilateral visual space perception parameter acquisition device based on picture description and reading test includes a first display module 101, a recording module 102, a first parameter acquisition module 103, an image processing module 104, a second parameter acquisition module 105, a picture description comprehensive parameter acquisition module 106 and a picture description parameter acquisition module 107:

[0092] The first display module 101 is used to display a picture and give a prompt for describing the picture; and is also used to define cells on the picture according to a preset size;

[0093] It should be noted that the picture refers to the visual content displayed to the subject in the picture description test device. It can be any form of image, such as a photo, illustration or icon, which is used to test the subject's description ability and visual spatial perception ability. Prompt refers to the guidance information or instructions given to the subject when the picture is displayed, which is intended to help the subject understand the test task. For example, the prompt may include "please describe the content of this picture" or "point out the main object in the picture" to guide the subject on how to describe the picture. The preset size refers to the standard size used when demarcating cells on the picture. For example, the preset size can be 100x100 pixels for each grid. This standardized size ensures that the cells demarcated in different tests are consistent, so as to accurately record and analyze the subject's gaze and description behavior. Cells refer to rectangular areas divided on the picture according to preset sizes. These areas help the system perform structured analysis of the picture. By dividing the picture into multiple cells, the subject's gaze time and description of each area can be recorded in detail, and then the distribution of their visual spatial perception ability for different parts of the picture can be evaluated.

[0094] It is understandable that the first display module is responsible for performing multiple key functions in the picture description test device. First, it displays a picture to the subject for description, and the picture may include various visual elements, such as scenes, objects or characters. In order to guide the subject's description task, the module will provide a prompt, such as "Please describe the content of this picture", to help the subject understand the test requirements and make an accurate description. Secondly, the first display module demarcates multiple cells on the picture according to a preset standard size. For example, the size of each grid can be set to 100x100 pixels. The purpose of this is to divide the picture into several small areas so as to accurately record the subject's gaze time and description content of each area. This gridding process enables the system to systematically analyze the distribution and description details of the subject's visual spatial perception ability in each part of the picture, thereby providing a detailed visual spatial perception ability and visual processing analysis.

[0095] The recording module 102 is used to record the cumulative gaze time, the movement trajectory of the eyeball focus point, and the number of grids crossed by the movement trajectory of the eyeball focus point on the picture;

[0096] It should be noted that the number of grids refers to the total number of different grids passed by the eye focus movement trajectory in the rectangular area (grid) divided into preset sizes on the picture. Specifically, the recording module will track the path of the eye focus moving on the picture and calculate how many different cells these paths pass through. This data helps to analyze the visual scanning patterns of the subjects when viewing the pictures, as well as their degree of attention to different areas. By counting the number of grids passed, we can understand the breadth of the subjects' exploration of the picture and the distribution of their visual spatial perception ability.

[0097] It is understandable that the recording module is responsible for the key data recording task in the picture description test device. First, it records the cumulative fixation time, that is, the total time that the subject stays on a specific area or the entire picture. This is achieved by continuously tracking the movement of the eyeball. When the eyeball stays stably in the same position for more than a set time (such as 200 milliseconds), the system accumulates this period of time as the fixation time. Secondly, the recording module 102 tracks and records the movement trajectory of the eyeball focus, that is, the path that the eyeball moves on the picture, and these paths reflect the visual scanning pattern of the subject. Finally, the module also counts the number of different grids that the eyeball movement trajectory passes through on the picture. The grid refers to the area delineated on the picture according to the preset size, and the recording module calculates the total number of grids that the eyeball trajectory passes through to evaluate the subject's attention breadth and visual spatial perception ability distribution to each part of the picture. These recorded data provide a detailed quantitative basis for subsequent visual spatial perception ability analysis and cognitive research.

[0098] As an example, the picture includes a left area and a right area, the number of grids includes a first number of grids and a second number of grids, the cumulative gaze time includes a first cumulative gaze time and a second cumulative gaze time, and the recording module includes a trajectory recording unit, a grid recording unit and a gaze time recording unit; the trajectory recording unit is used to record the movement trajectory of the eye focus point; the grid recording unit is used to record the first number of grids and the second number of grids passed by the movement trajectory of the eye focus point on the left area and the right area; the gaze time recording unit is used to record the first cumulative gaze time in the left area and the second cumulative gaze time in the right area.

[0099] The left area and the right area refer to two predefined areas on the picture, where the left area covers the left part of the picture, and the right area covers the right part of the picture. This division is used to analyze the distribution of the visual spatial perception ability of the subject on different sides of the picture. The first number of grids refers to the total number of grids that the eyeball focus passes through in the left area. The grid is a rectangular area delineated on the picture according to a preset size. Therefore, the first number of grids is the number of different grids that the eyeball passes through when moving in the left area. The second number of grids refers to the total number of grids that the eyeball focus passes through in the right area, that is, the number of different grids that the eyeball passes through when moving in the right area. The first cumulative fixation time refers to the total time that the subject stays in the left area. The fixation time recording unit records the fixation events of the eyeball in the left area and adds up the duration of these events to obtain the cumulative fixation time in the left area. The second cumulative fixation time refers to the total fixation time of the subject in the right area, that is, records the fixation events of the eyeball in the right area and calculates the total duration of these events, thereby obtaining the cumulative fixation time in the right area.

[0100] The recording module is responsible for recording eye movement and gaze data to analyze the subject's visual spatial perception ability of the picture. The picture is divided into a left area and a right area to classify the area where the line of sight is concentrated. The trajectory recording unit tracks and records the movement trajectory of the eye focus point to capture the movement path of the subject's line of sight. The grid recording unit calculates and records the number of grids that the eye focus point passes through in the left area and the right area, where the first number of grids refers to the total number of grids passed in the left area, and the second number of grids refers to the total number of grids passed in the right area. These grids are rectangular areas delineated according to preset sizes. The gaze time recording unit is responsible for recording the subject's cumulative gaze time in the left area and the right area, where the first cumulative gaze time refers to the total gaze time on the left area, and the second cumulative gaze time refers to the total gaze time on the right area. Through these records, the recording module can analyze in detail the subject's visual attention and gaze behavior in different areas.

[0101] The first parameter acquisition module 103 is used to obtain a first unilateral visual space perception parameter according to the number of grids, and to obtain a second unilateral visual space perception parameter according to the accumulated fixation time;

[0102] It should be noted that the function of the first parameter acquisition module includes calculating two unilateral visual spatial perception parameters, which can be used to evaluate the visual spatial perception ability distribution and possible neglect disorders of the subject. The first unilateral visual spatial perception parameter is obtained by comparing the number of grids on the left and right sides. Specifically, the number of grids passed by the eyeball focus in the left and right areas is first calculated, that is, the first number of grids and the second number of grids. The number of grids on the left is compared with the number of grids on the right, and a unilateral visual spatial perception parameter can be preliminarily obtained. If the number of grids on the left is less than that on the right, the parameter is less than 1, which can be used to evaluate left neglect; conversely, if the number of grids on the left is more than that on the right, the parameter is greater than 1, which can be used to evaluate right neglect. This parameter helps to preliminarily evaluate the visual spatial perception ability distribution of the subject on the left and right sides and possible left and right neglect phenomena. The second unilateral visual spatial perception parameter is calculated based on the cumulative fixation time on the left and right sides. First, the fixation time of the left and right areas is recorded, that is, the first cumulative fixation time and the second cumulative fixation time. By comparing these two time values, a fixation time parameter can be obtained. If the fixation time on the left side accounts for less than that on the right side, the parameter is less than 1, which can be used to evaluate the weak visual spatial perception ability on the left side; conversely, if the parameter is greater than 1, it can be used to evaluate the weak visual spatial perception ability on the right side. This parameter is used to evaluate the subject's fixation intensity in the left and right areas, thereby refining the evaluation of the degree of neglect. In further analysis, the device can also divide the image into more refined areas, such as the upper left area, the lower left area, the upper right area, and the lower right area. Through this refinement, the visual spatial perception parameters of each small area can be calculated to obtain the unilateral visual spatial perception parameters of each refined area. These parameters can be used for a more detailed evaluation of unilateral visual spatial perception ability or neglect disorder. This refined analysis can provide more accurate visual spatial perception ability distribution information and help identify the subject's visual attention in more specific areas.

[0103] It can be understood that the function of the first parameter acquisition module is to calculate two key unilateral visual spatial perception parameters. First, it calculates the first unilateral visual spatial perception parameter based on the number of grids in the left and right areas. By comparing the number of grids that the eye focus passes through in the left and right areas of the picture, this parameter can preliminarily evaluate the distribution of left and right visual spatial perception abilities. For example, if the number of grids on the left is significantly less than that on the right, it may indicate the presence of left-side neglect. Secondly, the module also calculates the second unilateral visual spatial perception parameter based on the cumulative fixation time of the left and right areas. By comparing the fixation time of the left and right areas, this parameter can further refine the evaluation of the degree of neglect. For example, if the fixation time on the left is less than that on the right, it may indicate that the visual spatial perception ability on the left is weaker. These two parameters combined help to comprehensively analyze the distribution of the subjects' visual spatial perception abilities and potential neglect disorders.

[0104] The image processing module 104 is used to obtain the region of interest in the image according to the eye focus point movement trajectory, the accumulated fixation time and a preset fixation time threshold;

[0105] It should be noted that the preset fixation time threshold refers to a set time standard used to determine which areas are of particular interest to the user. For example, if the threshold is set to 2 seconds, then when the eyeball fixates on a certain area for more than 2 seconds, it will be considered an area of ​​interest. The area of ​​interest refers to the part of the picture that the user pays special attention to, calculated based on the movement trajectory of the eyeball focus point and the cumulative fixation time. These areas may contain key details or information that attract the user's attention, such as people, text or other important elements in the picture.

[0106] It is understandable that the function of the image processing module is to identify the region of interest in the image based on the trajectory of the eyeball's focus point, the accumulated gaze time, and the preset gaze time threshold. Specifically, the module determines which areas have attracted the user's special attention by analyzing the trajectory of the eyeball's movement on the image and the gaze time in different areas. When the gaze time in a certain area exceeds the preset gaze time threshold (for example, 2 seconds), the area will be marked as an area of ​​interest. These areas of interest may contain important information or details in the image and are where the user's visual spatial perception is concentrated.

[0107] As an example, the image processing module includes a heat map unit, an activity map unit and a region division unit; the heat map unit is used to obtain an eye tracking heat map according to the movement trajectory of the eye focus point and the cumulative gaze time; the activity map unit is used to obtain an average activity map according to multiple eye tracking heat maps; the region division unit is used to select an area in the average activity map where the cumulative gaze time is greater than or equal to a preset gaze time threshold as an area of ​​interest.

[0108] The eye tracking heat map refers to a visualization image generated based on the movement trajectory of the eyeball focus point and the accumulated fixation time, which shows the user's fixation intensity in different areas of the picture. Color is usually used to indicate the length of fixation time. The darker the color, the longer the fixation time, reflecting the hot spots of the user's visual attention. The eye movement data comes from the tracking range of the head display lens. Even if the subject swings his head in order to use the area with stronger visual spatial perception ability to complete the image test task of this embodiment, the tracking range of his viewpoint on the lens is real and there is an offset. For example, his perspective can only appear on the right lens, although he can get all the left and right test tasks right when doing the task. We can get his visual spatial perception parameters through the eye tracking heat map on the lens, and then evaluate his unilateral neglect based on these parameters, which is impossible for traditional evaluation. In traditional evaluation, they can also get it right by shaking their heads and squinting their eyes, so it is not accurate.

[0109] The average activity map is an average image calculated from multiple eye tracking heat maps, which comprehensively reflects the average gaze of users on different areas of the image in multiple tests. By averaging the cumulative gaze time of different eye tracking heat maps, an overall activity map can be obtained to further analyze the user's visual attention trends and key areas. Please refer to Figure 4 , Figure 4The eye tracking heat map provided in the second embodiment of the unilateral visual spatial perception parameter acquisition device based on the picture description and reading test of this application can be used to visualize the eye movement data of the subject when taking the picture description test. The eye tracking heat map records the eye movement and fixation point of the subject when viewing the picture, and analyzes the distribution of the subject's visual spatial perception ability of different areas of the picture. In the heat map, the gradient of color is usually used to indicate the intensity and duration of fixation. For example, warm colors (such as red and yellow) may indicate the area where the subject fixates for a longer time, while colder colors (such as blue and green) indicate the area where the fixation time is shorter. Such color coding helps researchers quickly identify the areas that the subject focuses on during the observation process, as well as the areas that may be ignored. The specific description of the figure may include the following aspects: (1) Distribution of fixation points: The figure shows the positions of the various points where the subject's line of sight stays when observing the picture. (2) Fixation duration: Areas of different colors indicate the time the subject stays at each fixation point. The warmer the color, the longer the fixation time. (3) Region of interest (ROI): Specific regions of interest can also be marked in the graph. These regions are selected by the researcher based on pre-set criteria (such as fixation time threshold) to further analyze whether the subject's visual spatial perception is concentrated in these key areas. (4) Eye movement trajectory: The graph may also include the subject's eye movement trajectory, that is, the path of eye movement, which helps to understand the order and pattern of the subject's observation of the picture. By analyzing this graph, the subject's visual spatial perception pattern in the picture description task can be evaluated, and then visual spatial perception data can be obtained. This data is very useful for diagnosing and studying cognitive problems such as unilateral spatial neglect disorder.

[0110] This application uses QR code (two-dimensional code) tracking technology and HoloLens 2 (or HoloLens 3, etc.) devices, specifically using advanced computer vision algorithms to enhance the user experience. Specifically, this technology combines a depth sensor and a high-resolution camera to accurately detect the position and direction of the QR code by scanning and analyzing the image of the QR code in real time. The QR code is fixed on a bracket that can be raised and lowered, and it is portable and can be used anytime, anywhere to form an assessment environment that is accurate to the individual's height. The data captured by the sensor and camera are sent to the processing unit, which identifies and tracks the position of the QR code through complex image recognition and spatial calculations. These data are then mapped to the three-dimensional coordinate system of the device, and the image is expanded with the QR code as the center point to form a test area, ensuring that the application can present virtual content at the exact location in the user's field of view (the test area and the user / testee's viewing area are two separate areas). Through this fine positioning and tracking, HoloLens 2 can achieve a seamless augmented reality experience, allowing virtual objects to perfectly connect with QR codes in the real environment, providing users / testees with an immersive interactive experience. The eye tracking technology used in this application is based on infrared LED lighting and infrared cameras. Twelve infrared LEDs are arranged around the device frame to illuminate the user's / subject's eyes. Two infrared cameras located at the bridge of the nose capture eye image data. Through image processing algorithms, the system / device can calculate eye movement data in real time to achieve accurate gaze point rendering. Gaze point rendering tracks the user's / subject's eye movements in real time, determines the specific location where the user's / subject's gaze is, and renders the corresponding virtual content at that location. Specifically, the infrared camera captures the image of the user's / subject's eye, and the infrared LED provides lighting to ensure a clear image. Through complex image processing algorithms, the system / device calculates the user's / subject's gaze point, that is, the specific location of the user's / subject's line of sight in the virtual environment.

[0111] The image processing module includes a heat map unit, an activity map unit, and a region division unit. The heat map unit first uses the eye focus point movement trajectory and the cumulative gaze time to generate an eye tracking heat map, which is a color-coded image that shows the user's gaze intensity in different areas of the picture, where the depth of the color indicates the length of the gaze time. Then, the activity map unit synthesizes multiple eye tracking heat maps into an average activity map. By averaging these heat maps, a comprehensive reflection of the user's overall gaze pattern in multiple observation stages is obtained. Finally, the region division unit screens out those areas whose cumulative gaze time is equal to or exceeds the preset gaze time threshold based on the average activity map. These areas are marked as areas of interest to highlight the parts that the user pays most attention to, thereby providing data support for further analysis.

[0112] The second parameter acquisition module 105 is used to obtain a third unilateral visual space perception parameter according to the region of interest and the accumulated fixation time;

[0113] It should be noted that the third unilateral visual-spatial perception parameter refers to an indicator calculated based on the region of interest and the cumulative gaze time, which is used to further evaluate the distribution of the subject's unilateral visual-spatial perception ability. Specifically, this parameter determines the strength of the visual-spatial perception ability of these areas on the left or right side by analyzing the subject's gaze time on the region of interest.

[0114] It can be understood that the function of the second parameter acquisition module is to calculate the third unilateral visual spatial perception parameter based on the region of interest and the cumulative gaze time. Specifically, the module calculates the cumulative gaze time of the user in the region of interest, and then obtains the third unilateral visual spatial perception parameter. This parameter evaluates the visual spatial perception ability strength of the left or right region by comparing the user's gaze time in the region of interest, helping to further understand the distribution of the subject's visual spatial perception ability and possible unilateral visual spatial perception ability bias or neglect.

[0115] As an example, the region of interest includes a left region of interest and a right region of interest, and the second parameter acquisition module includes a left ratio unit, a right ratio unit and a parameter acquisition unit; the left ratio unit is used to calculate the left gaze time ratio of the cumulative gaze time in the left region of interest to the cumulative gaze time of all regions; the right ratio unit is used to calculate the right gaze time ratio of the cumulative gaze time in the right region of interest to the cumulative gaze time of all regions; the parameter acquisition unit is used to obtain a third unilateral visual spatial perception parameter based on the left gaze time ratio and the right gaze time ratio.

[0116] The left fixation time ratio refers to the ratio of the cumulative fixation time in the left region of interest to the cumulative fixation time of the user in all regions of interest. Specifically, this ratio is calculated by calculating the user's fixation time in the left region of interest and dividing it by the total fixation time in all regions of interest, thereby reflecting the user's relative visual spatial perception concentration in the left region. The right fixation time ratio refers to the ratio of the cumulative fixation time in the right region of interest to the cumulative fixation time of the user in all regions of interest. It is calculated by calculating the user's fixation time in the right region of interest and dividing it by the total fixation time in all regions of interest, thereby obtaining the relative visual spatial perception concentration of the right region. These two ratios are used to compare the fixation time distribution of the left and right regions of interest, thereby obtaining the third unilateral visual spatial perception parameter, which can be used to evaluate the user's unilateral visual spatial perception ability strength and possible visual spatial perception bias or neglect.

[0117] For example, if the region of interest is mainly concentrated on the left side of the picture, then by calculating the subject's cumulative fixation time in these areas, the third unilateral visual spatial perception parameter can be obtained. This parameter can show the ratio between the fixation time in the left area and the fixation time in the right area. If the cumulative fixation time on the left side is significantly higher than that on the right side, the parameter value will be greater than 1, otherwise it will be less than 1. This parameter helps to understand in detail the distribution of the subject's visual spatial perception in the region of interest, thereby more accurately evaluating unilateral visual spatial perception ability or neglect.

[0118] The picture description comprehensive parameter acquisition module 106 is used to obtain a picture description comprehensive parameter according to the first unilateral visual space perception parameter, the second unilateral visual space perception parameter and the third unilateral visual space perception parameter;

[0119] It should be noted that the picture description comprehensive parameter is an indicator obtained by combining the first unilateral visual spatial perception parameter, the second unilateral visual spatial perception parameter and the third unilateral visual spatial perception parameter, which is used to comprehensively evaluate the unilateral visual spatial perception ability of the subjects in the picture description test. Specifically, the first unilateral visual spatial perception parameter obtains visual spatial perception data by comparing the number of grids in the left and right regions, which can be used to preliminarily evaluate the distribution of visual spatial perception ability; the second unilateral visual spatial perception parameter obtains visual spatial perception data based on the cumulative fixation time of the left and right regions, which can be used to evaluate the intensity of attention; the third unilateral visual spatial perception parameter further refines the distribution of visual spatial perception ability on the left and right sides by the ratio of the fixation time of the region of interest to the total fixation time of all regions. Integrating these parameters, the picture description comprehensive parameter provides a comprehensive perspective for measuring and analyzing the unilateral visual spatial perception ability characteristics and possible visual spatial perception ability bias of the subjects in the picture description process.

[0120] It can be understood that the function of the picture description comprehensive parameter acquisition module is to calculate the picture description comprehensive parameter based on the first unilateral visual spatial perception parameter, the second unilateral visual spatial perception parameter and the third unilateral visual spatial perception parameter. The module integrates these three unilateral visual spatial perception parameters to comprehensively evaluate the overall unilateral visual spatial perception ability performance of the subject in the picture description test.

[0121] The picture description parameter acquisition module 107 is used to obtain the picture description parameter according to the ratio of the picture description comprehensive parameter to the preset picture description comprehensive parameter.

[0122] It should be noted that the preset picture description comprehensive parameter refers to the standardized picture description comprehensive parameter obtained by the device when conducting a picture description test on ordinary subjects. This parameter is used as a baseline value to compare with the test results of other subjects, so as to evaluate whether their performance deviates from the normal range. The picture description parameter is an indicator calculated based on the ratio of the picture description comprehensive parameter to the preset picture description comprehensive parameter. This parameter is used to measure the degree of deviation of the subject's performance in the picture description test from the normal baseline, reflecting the abnormality of their visual-spatial perception ability distribution and attention. By comparing these two parameters, the visual-spatial perception ability characteristics of the subject during the picture description process can be evaluated, and possible visual-spatial perception ability problems or biases can be identified.

[0123] It is understandable that the function of the picture description parameter acquisition module is to obtain the picture description parameter by calculating the ratio of the picture description comprehensive parameter to the preset picture description comprehensive parameter. Among them, the picture description comprehensive parameter is an indicator obtained by comprehensively considering the visual spatial perception data of the subject in the picture description test, while the preset picture description comprehensive parameter is the standard benchmark value obtained when the same test is performed on ordinary subjects. By comparing the ratio of the two, the module generates a picture description parameter, which is used to measure the degree of deviation of the subject's test performance from the normal benchmark, and help identify possible visual spatial perception ability abnormalities or biases.

[0124] The picture description test device in the unilateral visual space perception parameter acquisition device based on picture description and reading test described in this embodiment includes a first display module, a recording module, a first parameter acquisition module, an image processing module, a second parameter acquisition module, a picture description comprehensive parameter acquisition module and a picture description parameter acquisition module, wherein the first display module is used to display a picture and give a prompt for describing the picture; the first display module is also used to delineate cells on the picture according to a preset size; the recording module is used to record the cumulative gaze time, the movement trajectory of the eyeball focus point and the number of grids crossed by the movement trajectory of the eyeball focus point on the picture; the first parameter acquisition module is used to obtain the first unilateral visual space perception parameter according to the number of grids. The image processing module is used to obtain the region of interest in the image according to the eyeball focus point movement trajectory, the cumulative fixation time and the preset fixation time threshold; the second parameter acquisition module is used to obtain the third unilateral visual spatial perception parameter according to the region of interest and the cumulative fixation time; the picture description comprehensive parameter acquisition module is used to obtain the picture description comprehensive parameter according to the first unilateral visual spatial perception parameter, the second unilateral visual spatial perception parameter and the third unilateral visual spatial perception parameter; the picture description parameter acquisition module is used to obtain the picture description parameter according to the ratio of the picture description comprehensive parameter to the preset picture description comprehensive parameter. In the unilateral visual spatial perception parameter acquisition device based on picture description and reading test, each component works together to achieve the accurate acquisition of unilateral visual spatial perception parameters or data. First, the first display module displays the picture and provides description prompts, and at the same time delineates cells on the picture according to a preset size, which not only facilitates the subject to describe the picture, but also ensures the detailed division of the picture area, providing a clear basis for the analysis of visual spatial perception ability. Subsequently, the recording module records the subject's cumulative gaze time, the trajectory of the eyeball focus point, and the number of grids it crosses on the picture. These data provide key basic information for subsequent analysis and can be used to understand the distribution of the subject's visual spatial perception ability. Next, the first parameter acquisition module calculates the first unilateral visual spatial perception parameter based on the number of grids, and calculates the second unilateral visual spatial perception parameter based on the cumulative gaze time. This parameter can be used to preliminarily evaluate the distribution of the subject's left and right visual spatial perception abilities and the intensity of attention. The image processing module determines the area of ​​interest in the picture based on the trajectory of the eyeball focus point, the cumulative gaze time, and the preset gaze time threshold, so that the subject's attention to a specific area can be highlighted, which is convenient for further analysis. The second parameter acquisition module uses the area of ​​interest and the cumulative gaze time to calculate the third unilateral visual spatial perception parameter, providing more detailed information on the distribution of left and right visual spatial perception abilities.Subsequently, the picture description comprehensive parameter acquisition module integrates the first unilateral visual spatial perception parameter, the second unilateral visual spatial perception parameter and the third unilateral visual spatial perception parameter to calculate the picture description comprehensive parameter, which can be used to comprehensively evaluate the overall visual spatial perception performance of the subject. Finally, the picture description parameter acquisition module compares the ratio of the picture description comprehensive parameter to the preset picture description comprehensive parameter to obtain the picture description parameter, which can be used to quantify the degree of deviation of the subject's visual spatial perception performance and help identify possible visual spatial perception abnormalities or biases. The coordinated work of this series of steps realizes the comprehensive measurement and scientific analysis of the subject's visual spatial perception data, providing strong data support for further evaluation and intervention.

[0125] It should be noted that the above examples are only used to understand the present application, and do not constitute a limitation on the unilateral visual spatial perception parameter acquisition device based on picture description and reading test of the present application. More forms of simple transformations based on this technical concept, or direct / indirect application in other related technical fields are all within the scope of protection of the present application.

Claims

1. A device for acquiring unilateral visual spatial perception parameters based on picture description and reading test, characterized in that: The device includes a picture description test device, a reading test device and a parameter acquisition device; The picture description test device is used to perform a picture description test and obtain picture description parameters according to the eyeball focus point movement trajectory and the accumulated fixation time; The reading test device is used to perform a reading test and obtain reading parameters based on eye gaze, gestures and voice input; The visual space perception parameter acquisition device is used to obtain the target unilateral visual space perception parameter according to the picture description parameter and the reading parameter; The picture description testing device includes a first display module, a recording module and a first parameter acquisition module; The first display module is used to display a picture and provide a prompt for describing the picture, wherein the picture includes a left area and a right area; The first display module is further used to define cells on the picture according to a preset size; The recording module is used to record the accumulated gaze time, the movement trajectory of the eyeball focus point, and the number of grids crossed by the movement trajectory of the eyeball focus point on the picture; The first parameter acquisition module obtains a first unilateral visual space perception parameter by comparing the number of grids crossed on the left area and the right area, and obtains a second unilateral visual space perception parameter according to the accumulated fixation time on the left and right sides; The picture description testing device also includes an image processing module, a second parameter acquisition module, a picture description comprehensive parameter acquisition module and a picture description parameter acquisition module; The image processing module is used to obtain the region of interest in the image according to the eye focus point movement trajectory, the accumulated fixation time and a preset fixation time threshold; The second parameter acquisition module is used to obtain a third unilateral visual space perception parameter according to the region of interest and the accumulated fixation time; The picture description comprehensive parameter acquisition module is used to obtain a picture description comprehensive parameter according to the first unilateral visual space perception parameter, the second unilateral visual space perception parameter and the third unilateral visual space perception parameter; The picture description parameter acquisition module is used to obtain the picture description parameter according to the ratio of the picture description comprehensive parameter to a preset picture description comprehensive parameter.

2. The device according to claim 1, characterized in that The image processing module includes a heat map unit, an activity map unit and a region division unit; The heat map unit is used to obtain an eye tracking heat map according to the eye focus point movement trajectory and the accumulated fixation time; The activity map unit is used to obtain an average activity map based on a plurality of the eye tracking heat maps; The region division unit is used to select a region in the average activity map where the cumulative gaze time is greater than or equal to a preset gaze time threshold as the region of interest.

3. The device according to claim 1, characterized in that The region of interest includes a left region of interest and a right region of interest, and the second parameter acquisition module includes a left ratio unit, a right ratio unit and a parameter acquisition unit; The left ratio unit is used to calculate the left gaze time ratio of the cumulative gaze time in the left region of interest to the cumulative gaze time in all regions; The right side ratio unit is used to calculate the right side gaze time ratio of the cumulative gaze time in the right side region of interest to the cumulative gaze time in all regions; The parameter acquisition unit is used to obtain the third unilateral visual space perception parameter according to the left gaze time ratio and the right gaze time ratio.

4. The device according to claim 1, characterized in that The reading test device includes a second display module, a detection module, a locking module, a voice processing module and a third parameter acquisition module; The second display module is used to display a preset number of characters within a preset range, the characters including described characters and undescribed characters, the undescribed characters are displayed in a first preset color, and the described characters are displayed in a second preset color; The detection module is used to detect the gaze and the gesture to obtain a detection result; The locking module is used to lock the text according to the detection result to obtain a locking result; The voice processing module is used to receive the voice input describing the text; The third parameter acquisition module is used to obtain correct item parameters, incorrect item parameters and ignored item parameters according to the voice input, the locking result, the preset duration, the preset number and the undescribed text.

5. The device according to claim 4, characterized in that The reading test device also includes a fourth parameter acquisition module and a reading parameter acquisition module; The fourth parameter acquisition module is used to obtain the comprehensive reading parameter according to the preset range, the undescribed text, the preset format, the correct item parameter, the incorrect item parameter and the ignored item parameter; The reading parameter acquisition module is used to obtain the reading parameter according to the ratio of the reading comprehensive parameter to a preset reading comprehensive parameter.

6. The device according to claim 4, characterized in that The third parameter acquisition module includes a correct item parameter acquisition unit, an incorrect item parameter acquisition unit and an ignored item parameter acquisition unit; The correct item parameter acquisition unit is used to adjust and record the correct item parameters and set the locked text as the described text when the voice input corresponds to the locked result; The error item parameter acquisition unit is used to adjust and record the error item parameter when the voice input does not correspond to the locking result; The ignored item parameter acquisition unit is used to stop displaying the text after a preset time period, and calculate the ratio of the number of the undescribed text to the preset number to obtain the ignored item parameter.

7. The device according to claim 5, characterized in that The preset range includes a left range, a front range and a right range, and the fourth parameter acquisition module includes a left range parameter acquisition unit, a right range parameter acquisition unit and a reading comprehensive parameter acquisition unit; The left range parameter acquisition unit is used to obtain the left range parameter according to the number of the undescribed characters in the left range and the front range, and record the left range parameter in a preset form; The right range parameter acquisition unit is used to obtain the right range parameter according to the number of the undescribed characters in the right range and the front range, and record the right range parameter in the preset form; The comprehensive reading parameter acquisition unit is used to obtain the comprehensive reading parameter according to the correct item parameter, the incorrect item parameter, the ignored item parameter, the left range parameter and the right range parameter.

Images