System, program and storage medium for analyzing characteristics of developmental disorder

Abstract

To provide a system, a program and a storage medium for analyzing characteristics of a developmental disorder, which allow how children with a developmental disorder use their eyes to be grasped one by one.SOLUTION: A characteristic analysis system for a developmental disorder analyzes characteristics of a developmental disorder of a subject by: displaying the analysis image data selected from the analysis image data for analyzing how the eyes are used; storing the displayed target data and the line-of-sight data on the subject acquired by line-of-sight detection means at each of predetermined time intervals; performing coordinate-conversion of the stored target data and line-of-sight data into XY-axis coordinates; and analyzing how the eyes are used based on the time-series movement of the XY-axis coordinates of the line-of-sight data with respect to the XY-axis coordinates of the target data and / or the change in the time-series deviation amount between the XY-axis coordinates of the target data and the XY-axis coordinates of the line-of-sight data.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a system, program, and storage medium for analyzing characteristics of developmental disorders, which analyze the characteristics of developmental disorders by analyzing the way children with developmental disorders use their eyes. [Background technology]

[0002] Developmental disorders include ASD (Autism Spectrum Disorder), ADHD (Attention Deficit Hyperactivity Disorder), and LD (Specific Learning Disorder, Learning Disability). It is said that these developmental disorders often involve some kind of problem with eye use. Furthermore, developmental disorders often manifest as multiple overlapping disorders, and the characteristics of each developmental disorder vary from child to child. Therefore, it is important to analyze the eye use of each child with a developmental disorder.

[0003] The invention of Patent Document 1 discloses a training support device comprising: a display unit; an imaging unit that images a subject; a gaze detection unit that detects the gaze direction of the subject from the image captured by the imaging unit; a viewpoint detection unit that detects the subject's viewpoint in the display area of ​​the display unit based on the gaze direction; an output control unit that causes the display unit to display a diagnostic image showing the cause of a specified event and the event, and an explanatory image showing the causal relationship between the cause and the event; and an evaluation unit that calculates an evaluation value of the subject based on the viewpoint detected by the viewpoint detection unit when the diagnostic image is displayed.

[0004] The invention of Patent Document 2 discloses an autism diagnostic support device that detects characteristics of autism based on the way a subject looks at another subject, and is characterized by comprising: a gaze detection unit that detects the gaze direction of the subject looking at the subject; a camera that captures an image of the subject; a pupil position detection unit that measures the pupil coordinates of the subject; and a data analysis unit that uses the gaze direction and the pupil coordinates to calculate the relationship between the gaze direction of the subject and the pupil position of the subject, and outputs the relationship together with the image of the subject.

[0005] The invention of Patent Document 3 discloses a diagnostic support device that includes a housing, a first imaging unit that images a subject and outputs a first image, a second imaging unit that images a target person facing the subject across the housing and outputs a second image, a direction detection unit that detects the subject's gaze direction as a first gaze direction represented in a first coordinate system based on the first image, a position detection unit that detects the target's eye position as a first position represented in a second coordinate system based on the second image, a conversion unit that performs a process of converting the first gaze direction to a second gaze direction represented in the second coordinate system or a process of converting the first position to a second position represented in the first coordinate system, and a determination unit that determines whether the subject is gazing at the target's eyes based on the second gaze direction and the first position converted by the conversion unit, or based on the first gaze direction and the second position. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent No. 6330638 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-183873 [Patent Document 3] Japanese Patent Application Laid-Open No. 2011-206542 Summary of the Invention [Problem to be solved by the invention]

[0007] The invention of Patent Document 1 is a training support device for people with developmental disorders, the invention of Patent Document 2 is a diagnostic technology that uses the coordinates of an infant's gaze point and the coordinates of the mother's pupils, and the invention of Patent Document 3 is an invention that focuses on whether or not people with developmental disorders look into the eyes of the person they are facing, which is a characteristic of people with developmental disorders, but there was a problem in that it was not possible to analyze how people with developmental disorders use their eyes.

[0008] The present invention was conceived in light of these problems, and aims to provide a system, program, and storage medium for analyzing the characteristics of developmental disorders, which analyzes the eye usage challenges of each child with developmental disorders, who often have multiple disorders including ASD, ADHD, and LD, and analyzes the characteristics of their developmental disorders. [Means for solving the problem]

[0009] The system for analyzing characteristics of developmental disorders described in claim 1 is a system for analyzing the characteristics of a developmental disorder of a subject, comprising a display screen, gaze detection means for detecting the gaze of the subject, and an analytical information terminal, wherein a control unit of the analytical information terminal causes an image display unit to display analytical image data selected from a plurality of analytical image data stored in a memory unit, each of which sets a task for analyzing eye usage, and causes the memory unit to store, at predetermined time intervals, target data to be gazed at in the task instructed to the subject in the displayed analytical image data, and the subject's gaze data acquired by the gaze detection means, causes a coordinate conversion unit to convert the stored target data and gaze data into XY-axis coordinates and store them in the memory, causes an eye usage analysis unit to analyze the movement over time of the XY-axis coordinates of the gaze data relative to the XY-axis coordinates of the target data, and / or changes in the amount of deviation over time between the XY-axis coordinates of the target data and the XY-axis coordinates of the gaze data, and causes the image display unit to display the analysis results of the eye usage analysis unit.

[0010] The system for analyzing characteristics of developmental disorders described in claim 2 is characterized in that, in claim 1, the image data for analysis is content images or video data for analyzing the characteristics of pursuit eye movements, content images or video data for analyzing the characteristics of saccadic eye movements, content images or video data for analyzing the characteristics of coordinated movements, content images or video data for analyzing the characteristics of attention ability, or content images or video data for analyzing the characteristics of spatial cognitive ability.

[0011] The developmental disorder characteristic analysis system of claim 3 is characterized in that, in claim 1, the control unit further causes the frame image generation unit to generate a plurality of frame images at predetermined time intervals using the target data and the gaze data stored in the memory unit at predetermined time intervals and arrange the frame images in chronological order, causes the gaze path analysis unit to generate a gaze path analysis image such as a trace that tracks the gaze movement of the subject, a gaze path that shows the order in which the subject's gaze fixed, or a heat map that shows the gaze fixation time of the subject, within the time series range up to the arbitrarily specified frame image, and causes the image display unit to display in picture-in-picture on the display screen the frame images arranged in chronological order, a scroll function to the right or left that scrolls the frame images back and forth in chronological order, and a selection function that can select the frame image that specifies the range from the instruction to the subject in the chronological order of the gaze path analysis image.

[0012] The developmental disorder characteristic analysis system of claim 4 is in claim 2 or 3, wherein the display screen is a touch panel type, and comprises an imaging means for imaging the subject, an audio acquisition means for acquiring the audio of the subject, and a manual input means for manual input by the subject, and wherein the control unit reflects touch information of a touch on the display screen or manual input information by the manual input means on the display screen of the image data for analysis on the image display unit, causes the image captured by the imaging means to be displayed in picture-in-picture on the display screen displaying the analysis results by the eye usage analysis unit or on the display screen displaying the gaze path analysis image by the gaze path analysis unit, and causes an audio output unit to output the audio acquired by the audio acquisition means.

[0013] The program described in claim 5 is a program for causing the computer of the analytical information terminal to function as each of the components of the developmental disorder characteristics analysis system described in claim 4.

[0014] The recording medium described in claim 6 is a computer-readable storage medium that stores a program for causing the computer of the analytical information terminal to function as each part of the developmental disorder characteristic analysis system described in claim 4. [Effects of the Invention]

[0015] The invention of claim 1, 2, 4, 5 or 6 has the effect of analyzing various eye usage patterns in children with developmental disabilities, who often have multiple disorders among ASD, ADHD and LD, such as pursuit eye movement, saccadic eye movement, coordination, gaze ability and spatial cognitive ability, by acquiring target data to which the subject should direct their gaze in a task instructed to the subject and the subject's gaze data at predetermined time intervals, with the coordinates of each on the X-axis and Y-axis, and analyzing the various eye usage patterns in time series between the target data and the gaze data, thereby enabling analysis of the characteristics of developmental disabilities that are influenced by eye usage patterns.

[0016] The invention described in claim 3, 4, 5 or 6 makes it possible to call up a time series timing that you want to check when analyzing how your eyes are used, such as a trace that tracks the movement of the subject's gaze, a gaze path that shows the order in which the subject's gaze fixed, or a heat map that shows the duration of time the subject's gaze is fixed, and to check the subject's gaze path in addition to analyzing how your eyes are used.This has the effect of capturing all the gaze information emitted by the subject, allowing you to analyze how each subject uses their eyes, and analyzing the characteristics of developmental disorders that are influenced by how your eyes are used.

[0017] The invention described in claim 4, 5 or 6 can add the subject's movements such as changes in facial expression, the subject's voice, and the speed of the subject's finger reactions in addition to the analysis of eye usage and gaze path, and can comprehensively capture the subject's gaze and body movements to analyze the eye usage of each subject, thereby achieving the effect of being able to analyze the characteristics of developmental disorders that are influenced by eye usage. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is an explanatory diagram of the configuration of a developmental disorder characteristic analysis system of the present invention. [Figure 2] FIG. 2 is an explanatory diagram of the configuration of the analytical information terminal of the developmental disorder characteristic analysis system of the present invention. [Figure 3] 1 is a flow diagram of the developmental disorder characteristic analysis system of the present invention. [Figure 4] An explanatory diagram of a content image or video of image data for analysis of pursuit eye movements, where (a) is an explanatory diagram of the starting position when the object of gaze is a car, and (b) is an explanatory diagram of the object of gaze when the car is moving. [Figure 5] In the case of pursuit eye movement as shown in FIG. 4, this is an explanatory diagram in which the time-series movement of the XY-axis coordinates of the gaze data relative to the XY-axis coordinates of the target data is graphed so as to be readable. [Figure 6] An explanatory diagram of content images or videos of image data for analysis of saccadic eye movements, where (a) is an explanatory diagram for setting the display order in advance, (b) is an explanatory diagram for when the target data for the first gaze target is displayed, and (c) is an explanatory diagram for when the target data for the second gaze target is displayed at a position separated from the first. [Figure 7] In the case of saccadic eye movement as shown in Figure 6, these explanatory diagrams are graphs that allow the time-series movement of the XY-axis coordinates of the gaze data relative to the XY-axis coordinates of the target data to be read. (a) is an explanatory diagram in the center of Figure 6 where only the position of "1" is displayed, and (b) is an explanatory diagram in the upper right corner of Figure 6 where only the position of "2" is displayed. [Figure 8] An explanatory diagram of the content image or video of image data for analysis of eye-hand coordination, where (a) is an explanatory diagram of a slot, (b) is an explanatory diagram of when the rotation of the leftmost column is stopped, and (c) is an explanatory diagram of when the same alphabet is lined up in a horizontal row as instructed. [Figure 9]When analyzing eye-hand coordination as shown in Figure 8, these explanatory diagrams are graphed so that the time-series movement of the XY-axis coordinates of the gaze data relative to the XY-axis coordinates of the target data can be read. (a) is an explanatory diagram when the top left-hand square is the target data, and (b) is an explanatory diagram when the top center square is the target data. [Figure 10] An explanatory diagram of a content image or video of image data for analysis of gaze ability, where (a) is an explanatory diagram when the face is the target data, and (b) is an explanatory diagram when the eyes are the target data. [Figure 11] In the case of the gaze ability shown in Figure 10, this is an explanatory diagram in which the gaze target is a face, and the graph is used to read the time-series movement of the XY-axis coordinates of the gaze data relative to the XY-axis coordinates of the target data. [Figure 12] FIG. 10 is an explanatory diagram of a content image or video of analytical image data for spatial cognitive ability, in which lines between a circle and a square are crossed. [Figure 13] In the case of spatial cognitive ability as shown in Figure 12, this is an explanatory diagram that graphs the gaze data relative to the XY-axis coordinates of the target data and the time-series movements of the XY-axis coordinates of the path traced by the subject's finger so that they can be read. [Figure 14] FIG. 10 is an explanatory diagram of a content image or video of image data for analysis of visual attention ability, in the case of a color palette. [Figure 15] In the case of the gaze ability shown in FIG. 14, this is an explanatory diagram in which the time-series movement of the XY-axis coordinates of the gaze data relative to the XY-axis coordinates of the target data is graphed so as to be readable. [Figure 16] FIG. 10 is an explanatory diagram of an example of a frame image and a scan path analysis image, illustrating tracing. [Figure 17] FIG. 10 is an explanatory diagram of an example of a frame image and a gaze path analysis image, illustrating a gaze path. [Figure 18] FIG. 10 is an explanatory diagram of an example of a frame image and a scan path analysis image, and is an explanatory diagram of a heat map. DETAILED DESCRIPTION OF THE INVENTION

[0019] When multiple disorders such as ASD, ADHD, and LD appear together, it can cause imbalances in communication skills, attention, etc., and these imbalances differ from person to person, and are the characteristics of each individual's developmental disorder. Furthermore, it is said that these developmental disorders often involve some kind of problem with how the eyes are used.

[0020] For example, the developmental disorders of children can manifest themselves in the following ways: first, there is a problem with the use of the eyes, in that the pursuit eye movement, which moves the eyes to follow the object of gaze, is not sufficiently developed.

[0021] Secondly, after reading and understanding what is written on the blackboard, they have difficulty copying it down into their notebooks because they are unable to smoothly move their gaze from the distant blackboard to their notebooks in front of them. This means that they have not adequately developed saccadic eye movements (also known as saccadic eye movements), which involve quickly moving the eyes from one point to another.

[0022] Thirdly, there are issues with how they use their eyes, such as not being able to color, or having difficulty harmonizing what they see with their eyes and how they move their hands toward a certain goal, such as when letters are unbalanced and go outside the frame, and they have not been able to fully develop coordination between their eyes and their hands.

[0023] Fourth, they rarely make eye contact with others, which makes it difficult to communicate with others using their gaze as a cue, and they tend to avoid eye contact. This also means that they have not adequately developed the ability to focus their attention on others, which is an issue with how they use their eyes.

[0024] Fifth, because it is difficult to process the information received by the eyes and grasp the overall image of space, it is difficult to accurately remember and recall the shapes of letters, and there are issues with how people use their eyes in that their spatial cognitive ability, which is the ability to recognize the position, shape, size, relative positions, direction, height, width, distance, etc. of objects in three-dimensional space, is not sufficiently developed, which leads to problems with reading and writing, solving geometric problems in arithmetic, playing ball games, and tidying up and organizing.

[0025] Therefore, the system 1 for analyzing characteristics of developmental disorders of the present invention is an invention that analyzes the use of the five eyes of children with developmental disorders to analyze the characteristics of developmental disorders. Also, since it is said that the development of a child's nervous system reaches about 80% by the age of 4 to 5 and 100% by the age of 12, the system 1 for analyzing characteristics of developmental disorders of the present invention is primarily targeted at children with developmental disorders.

[0026] As shown in Fig. 1, the developmental disorder characteristics analysis system 1 of the present invention is a system for analyzing the characteristics of developmental disorders based on an analysis of how a subject uses their eyes, and includes a display screen 3, gaze detection means 4 that detects the subject's gaze, and an analytical information terminal 2. It also includes imaging means 5 that images the subject, voice acquisition means 6 that acquires the subject's voice, and manual input means 7 that the subject manually inputs. The display screen 3, gaze detection means 4, analytical information terminal 2, imaging means 5, voice acquisition means 6, and manual input means 7 are communicatively connected via wiring 9.

[0027] The display screen 3 may be, for example, the terminal screen of the analytical information terminal 2, and may be any screen capable of displaying images or videos. When the subject is asked to trace or press the display screen 3 with a finger, a touch panel type display screen 3 is used, which allows input by touching the display screen 3.

[0028] The gaze detection means 4 is a device that includes a near-infrared light source and a camera and is capable of detecting the gaze position of the subject, and is, for example, an eye tracker (for example, manufactured by Tobii).

[0029] The analytical information terminal 2 is an information terminal used by the subject or the person analyzing the characteristics of developmental disorders, and is equipped with, for example, a CPU (Central Processing Unit) that controls the developmental disorder characteristics analysis system of the present invention. The CPU reads and executes programs to cause each unit to perform its respective function.

[0030] The imaging means 5 is, for example, a camera, and is fixed at a position where the subject's face can be captured within the field of view of the camera. The camera may be a built-in camera in the analytical information terminal 2 or an external camera.

[0031] The voice acquisition means 6 includes, for example, a microphone, which may be a general microphone such as a condenser microphone. The microphone may be a built-in microphone in the analytical information terminal 2 or an external microphone.

[0032] The manual input means 7 may be any manual input means, for example, a push button switch, which allows the subject to manually input ON information.

[0033] As shown in FIG. 2, the analytical information terminal 2 comprises a control unit 10, an eye usage analysis unit 11, a coordinate conversion unit 12, a memory unit 13, an image display unit 14, a transmission / reception unit 15, a frame image generation unit 16, and a gaze path analysis unit 17, and the control unit 10 causes the image display unit 14 to display analytical image data selected from a plurality of analytical image data in which tasks for analyzing eye usage are set and stored in the memory unit 13, and stores in the memory unit 13 target data to be gazed at in the task instructed to the subject in the displayed analytical image data, and a front The gaze data of the subject acquired by the gaze detection means 4 is stored at predetermined time intervals, the coordinate conversion unit 12 converts the stored target data and gaze data into XY-axis coordinates and stores them in the memory unit 13, the eye usage analysis unit 11 analyzes the time-series movement of the XY-axis coordinates of the gaze data relative to the XY-axis coordinates of the target data and / or changes in the amount of deviation between the XY-axis coordinates of the target data and the XY-axis coordinates of the gaze data, and displays the analysis results of the eye usage analysis unit 11 on the image display unit 14. Furthermore, the analytical information terminal 2 is equipped with an audio output unit 18, as shown in Fig. 2, and the control unit 10 controls the audio output unit 18 to output audio.

[0034] The image data for analysis may be, for example, content images or video data for analyzing the characteristics of pursuit eye movements as shown in FIG. 4, content images or video data for analyzing the characteristics of saccadic eye movements as shown in FIG. 6, content images or video data for analyzing the characteristics of coordinated movements as shown in FIG. 8, content images or video data for analyzing the characteristics of attention ability as shown in FIG. 10 or FIG. 14, or content images or video data for analyzing the characteristics of spatial cognition ability as shown in FIG. 12.

[0035] The control unit 10 further causes the frame image generation unit 16 to generate a plurality of frame images arranged in chronological order at predetermined time intervals based on the target data and the gaze data for each predetermined time interval stored in the memory unit 13, causes the gaze path analysis unit 17 to generate a gaze path analysis image 81 such as a trace that tracks the movement of the subject's gaze, a gaze path that shows the order in which the subject's gaze fixed, or a heat map that shows the subject's gaze fixation time, within the time series range from the instruction to the subject to the frame image arbitrarily specified, and causes the image display unit 14 to display in picture-in-picture on the display screen 3 the frame images arranged in chronological order, a scroll function to the right or left that scrolls the frame images back and forth in chronological order, and a selection function that can select the frame image that specifies the range from the instruction to the subject in the time series of the gaze path analysis image 81.

[0036] The display screen 3 is a touch panel type and is equipped with an imaging means 5 for imaging the subject, an audio acquisition means 6 for acquiring the subject's audio, and a manual input means 7 for manual input by the subject, and the control unit 10 causes the image display unit 14 to reflect touch information from touching the display screen 3 or manual input information from the manual input means 7 on the display screen 3 of the image data for analysis, causes the image captured by the imaging means 5 to be displayed in picture-in-picture on the display screen 3 displaying the analysis results from the eye usage analysis unit 11 or on the display screen 3 displaying the gaze path analysis image 81 from the gaze path analysis unit 17, and causes an audio output unit 18 to output the audio acquired by the audio acquisition means 6.

[0037] Next, the analysis image data will be described. The analysis image data is stored in the memory unit 13 and displayed on the display screen 3, and a task to be performed by the subject is displayed on the analysis image data. The analysis image data can be used to analyze eye usage for pursuit eye movement, coordination, saccade eye movement, gaze ability, or spatial cognitive ability. The analysis image data of the present invention is not limited to the examples described in the present invention, and may be any analysis image data that can analyze eye usage for pursuit eye movement, coordination, saccade eye movement, gaze ability, or spatial cognitive ability.

[0038] Next, the target data will be described. The target data indicates the task to be instructed to the subject using the analysis image data displayed on the display screen 3, i.e., the range of the object to which the subject should direct their gaze, and is set according to the method of analyzing eye usage for pursuit eye movement, coordination, saccadic eye movement, gaze ability, or spatial cognitive ability. The time series is analyzed to determine when and how long the subject's gaze looks within the range of the target data, and how much it deviates from the range of the target data over time.

[0039] The gaze data will now be described. The gaze data indicates the position of the latitude line that the subject looks at in the display screen 3 on which the image data for analysis is displayed in chronological order after the subject is given a task instructed in the image data for analysis displayed on the display screen 3, and is acquired by the gaze detection means 4 and stored in the memory unit 13 at predetermined time intervals.

[0040] Next, the flow of analysis of eye usage by the control unit 10 of the analytical information terminal 2 will be described with reference to a developmental disorder characteristics analysis system flow diagram 100 shown in Fig. 3. First, there is a step 101 for storing analytical image data. In the step 101 for storing analytical image data, a plurality of analytical image data sets each having a task set for analyzing eye usage for pursuit eye movement, coordination, saccadic eye movement, gaze ability, or spatial cognitive ability are stored in the memory unit 13.

[0041] Next is the analysis image data selection step 102. In the analysis image data selection step 102, the analysis image data that is determined to be used by the subject is selected from the plurality of analysis image data stored in the storage unit 13.

[0042] Next is the analysis image data display step 103. In the analysis image data display step 103, the control unit 10 causes the image display unit 14 to display the selected analysis image data on the display screen 3.

[0043] Next is the subject's task engagement step 104. The task engagement step 104 first includes a task instruction step 105, in which the image display unit 14 displays a task for the subject to engage in on the screen of the image data for analysis displayed on the display screen 3. The subject looks at the task and engages with it. Then, in order to observe the situation when the subject engages with the task, a target data and gaze data acquisition step 106 and a subject movement information acquisition step 107 are executed simultaneously with the task engagement step 104.

[0044] In the target data and gaze data acquisition step 106, the control unit 10 causes the transceiver unit 16 to receive the gaze data of the subject from the gaze detection means 4 at predetermined time intervals and store it in the memory unit 13, and simultaneously causes the target data to be stored in the memory unit 13 at predetermined time intervals. At the same time, in the subject operation information acquisition step 107, the control unit 10 causes the transceiver unit 16 to receive one or more pieces of information selected from subject touch input information, subject image capture, subject voice, and subject manual input information at predetermined time intervals, store the subject image capture and the subject voice in the memory unit 13, and store the subject touch input information and the subject manual input information in the memory unit 13 as needed for analysis.

[0045] The control unit 10 causes the image display unit 14 to reflect touch information on the display screen 3 or manual input information by the manual input means 7 on the display screen 3 of the image data for analysis, or causes the image display unit 14 to display a picture-in-picture image of the image captured by the imaging means 5 on the display screen 3 displaying the analysis results by the eye usage analysis unit 11 or on the display screen 3 displaying the gaze path analysis image 81 by the gaze path analysis unit 17, and causes the audio output unit 18 to output the audio captured by the audio acquisition means 6 from a speaker provided in the information terminal 2 for analysis.

[0046] Next is an XY-axis coordinate conversion step 108. In the XY-axis coordinate conversion step 108, the control unit 10 causes the coordinate conversion unit 12 to convert the target data and the line-of-sight data stored in the memory unit 13 into XY-axis coordinates and store them in the memory unit 13.

[0047] Next are an eye usage analysis step 109 and a gaze path analysis step 110. The analysis of the characteristics of developmental disorders based on the analysis of the subject's eye usage is performed based on output information, namely, the analysis results in the eye usage analysis step 109 and the gaze path analysis step 110, which are generated based on input information acquired from the target data and gaze data acquisition step 106 and the subject's movement information acquisition step 107.

[0048] In the eye usage analysis step 109, the control unit 10 causes the eye usage analysis unit 11 to analyze the time-series movement of the XY-axis coordinates of the gaze data relative to the XY-axis coordinates of the target data, and / or, although not shown, the change in the amount of deviation between the XY-axis coordinates of the target data and the XY-axis coordinates of the gaze data over time, as shown in, for example, Figures 5, 7, 9, 11, 13, or 15. For example, if the target data is a point, the amount of deviation is the distance between the position of the point and the position of the gaze data, and for example, if the target data is a shape with an outline, the amount of deviation is the distance between the position of the gaze data and the position of the closest outline.

[0049] In the scan path analysis step 110, the control unit 10 causes the frame image generation unit 16 to generate the target data and the gaze data for each predetermined time interval stored in the storage unit 13 into a plurality of frame images 80 for each predetermined time interval, and arrange the frame images 80 in time series, and causes the scan path analysis unit 17 to perform a trace that plots and tracks the movement of the subject's gaze in time series for each predetermined time interval, for example, as shown in FIG. 16, within a time series range from the timing when instructions are displayed to the subject on the display screen 3 to the frame image 80 that is arbitrarily instructed, and to perform a trace that plots and tracks the movement of the subject's gaze in time series for each predetermined time interval, for example, as shown in FIG. 17, and to perform a trace that plots and tracks the order in which the subject's gaze stops for a predetermined time or more, for example, as "1", "2", "3", etc. and a heat map that displays the subject's gaze fixation time as, for example, the size of the area of ​​a figure, as shown in FIG. 18, and the image display unit 14 displays, in a picture-in-picture format, the frame images 80 arranged in chronological order, as shown in, for example, FIGS. 16 to 18, a rightward or leftward scroll function (not shown) that scrolls the frame images 80 back and forth in chronological order, and a selection function, for example, a slider 82 that can select the frame image that specifies the range from the instructions to the subject in the chronological order of the gaze path analysis image 81.

[0050] When an arbitrary frame image 80 is specified with the slider 82, the control unit 10 causes the image display unit 14 to display an image identical to the specified frame image 80 enlarged on the display screen 3, and when the scroll function is used, causes the frame image 80 and the images on the display screen 3 to scroll back and forth in chronological order.

[0051] Next is an analysis result display step 111. In the analysis result display step 111, the control unit 10 causes the image display unit 14 to display the analysis result of the eye usage analysis unit 11 in the eye usage analysis step 109 or the analysis result of the gaze trajectory analysis unit 17 in the gaze trajectory analysis step 110. At this time, if necessary for the analysis, the control unit 10 causes the image display unit 14 to display the subject's touch input information, the subject's captured screen, and the subject's manual input information in picture-in-picture format at the same time in time on the display screen 3 on which the analysis result of the eye usage analysis unit 11 is displayed or on the display screen 3 on which the analysis result of the gaze trajectory analysis unit 17 is displayed, and if necessary for the analysis, the control unit 10 causes the audio output unit 18 to output the subject's voice from a speaker provided in the analytical information terminal 2 at the same time in time.

[0052] Next is a developmental disorder characteristics analysis step 112. In the developmental disorder characteristics analysis step 112, the subject's touch input information, the subject's captured screen, the subject's manual input information, and the subject's voice are checked as necessary for the analysis, and the analysis result of the gaze path analysis unit 17 is checked as necessary for the analysis, while the subject's eye usage is analyzed based on the analysis result of the eye usage analysis unit 11, and the characteristics of the subject's developmental disorder are analyzed. In the developmental disorder characteristics analysis step 112, a pattern of the association between the classification of eye usage and the characteristics of developmental disorders is created in advance, and the characteristics of developmental disorders are analyzed based on this pattern.

[0053] Furthermore, in the developmental disorder characteristic analysis step 112, the input information acquired from the target data and gaze data acquisition step 106 and the subject's movement information acquisition step 107 can be input to AI to have the AI ​​analyze how the subject uses their eyes and analyze the characteristics of the developmental disorder.

[0054] Next, an embodiment of the developmental disorder characteristic analysis system 1 of the present invention will be described. The developmental disorder characteristic analysis system 1 of the present invention is an invention that analyzes the characteristics of developmental disorders based on the analysis image data that analyzes eye usage for pursuit eye movements, coordination, saccade eye movements, gaze ability, or spatial cognitive ability. The analysis image data in the developmental disorder characteristic analysis system 1 of the present invention is not limited to the examples and may be any analysis image data that can analyze eye usage for pursuit eye movements, coordination, saccade eye movements, gaze ability, or spatial cognitive ability. The following embodiment will mainly describe steps in which the information processing of the control unit 10 in the developmental disorder characteristic analysis system flow diagram 100 differs depending on the analysis image data for pursuit eye movements, coordination, saccade eye movements, gaze ability, or spatial cognitive ability.

[0055] First, an example of pursuit eye movement is described. In the analysis image data display step 103, the analysis image data is displayed as a content image or video in which an arbitrarily set gaze target moves along a linear or curved trajectory. The gaze target may be any gaze target familiar to children with developmental disabilities, such as an animal, vehicle, food, or character. For example, as shown in FIG. 4, a car is displayed as the gaze target. Then, in the task instruction step 105, an instruction to follow the moving car that appears on the display screen 3 with one's eyes is displayed. Then, in the target data and gaze data acquisition step 106, the target data is the area surrounded by the outline of the car. For example, in the case of FIG. 4(a), target data 21a of the car and gaze data 20a of the subject are stored. In the case of FIG. 4(b), target data 21b of the car moving along trajectory T and gaze data 20b of the subject are stored.

[0056] In the eye usage analysis step 109, the control unit 10 causes the eye usage analysis unit 11 to generate, for example, as shown in FIG. 5, an X-axis coordinate 22 of the target vehicle's movement trajectory and a Y-axis coordinate 23 of the target vehicle's movement trajectory, and generate time-series plots of gaze X-axis coordinates 24 and gaze Y-axis coordinates 25 at predetermined time intervals. Then, the gaze data 20a shown in FIG. 4(a) is generated as a black square (gaze data 20a, gaze X-axis coordinate 24) and a black circle (gaze data 20a, gaze Y-axis coordinate 25) in FIG. 5, and the gaze data 20b shown in FIG. 4(b) is generated as a black square (gaze data 20b, gaze X-axis coordinate 24) and a black circle (gaze data 20b, gaze Y-axis coordinate 25) in FIG. 5. Then, for example, the coordinate deviation between the gaze data 20a and the target data 21a of the vehicle at the same time is displayed as a deviation amount 26 in the X-axis direction and a deviation amount 27 in the Y-axis direction. Then, although not shown, a diagram showing only the amount of deviation over time may be generated. In the analysis result display step 111, the control unit 10 causes the image display unit 14 to display on the display screen 3 the diagram generated in the eye usage analysis step 109 or the diagram generated in the scan path analysis step 110.

[0057] In the developmental disorder characteristic analysis step 112, for example, as shown in FIG. 5, the characteristics of the developmental disorder are analyzed based on the time-series movement of the XY-axis coordinates of the gaze data 20 relative to the XY-axis coordinates of target data 21 of a car, which is the target data, and / or, although not shown, based on the change in the amount of deviation over time between the XY-axis coordinates of the target data 21 and the XY-axis coordinates of the gaze data 20, and further, if necessary for the analysis, the subject's image capture screen and subject's voice information are confirmed, and further, if necessary for the analysis, the scan path analysis image 81 is confirmed, and the characteristics of the developmental disorder are analyzed based on the pattern of association between the classification of eye usage in pursuit eye movement and the characteristics of the developmental disorder, which has been created in advance.

[0058] Next, an example of saccadic eye movement is described. In the analysis image data display step 103, the analysis image data is displayed as a content image or video in which an arbitrarily set gaze target suddenly appears in sequence at a distance from a certain position. The gaze target may be any gaze target familiar to children with developmental disabilities, such as numbers, animals, vehicles, food, or characters. For example, as shown in FIG. 6, a number is displayed as the gaze target. Then, in the task instruction step 105, an instruction to gaze intently at the number that appears on the display screen 3 is displayed. Then, in the target data and gaze data acquisition step 106, the target data is the area surrounded by a circle. For example, in the case of FIG. 6(b), target data 31 of the circle surrounding the number "1" and the subject's gaze data 35a are stored. In the case of FIG. 6(c), target data 32 of the circle surrounding the number "2" that appears at a distance from the "1" and almost replaces the "1" are stored.

[0059] In the eye usage analysis step 109, the control unit 10 causes the eye usage analysis unit 11 to generate the ranges of target data 31, 32 for "1" or "2" on XY coordinates, and plot the subject's gaze data 35a, 35b on XY coordinates at predetermined time intervals. Although not shown, a diagram showing only the amount of deviation over time may also be generated. In the analysis result display step 111, the control unit 10 causes the image display unit 14 to display the diagram generated in the eye usage analysis step 109 or the diagram generated in the gaze path analysis step 110 on the display screen 3.

[0060] In the developmental disorder characteristic analysis process 112, as shown in Figures 7(a) and 7(b), for example, the characteristics of the developmental disorder are analyzed based on the movement of XY-axis coordinates over time, such as the time-series trajectory and deviation amount of gaze data 35a relative to target data 31 and the time-series trajectory and deviation amount of gaze data 35b relative to target data 32, and / or, although not shown, based on changes in the time-series deviation amount between the XY-axis coordinates of target data 31, 32 of the gaze target "1" or "2" which is the target data, and the XY-axis coordinates of gaze data 35a, 35b, and further, if necessary for the analysis, the subject's image capture screen and subject's voice information are confirmed, and further, if necessary for the analysis, the gaze path analysis image 81 is confirmed, and the characteristics of the developmental disorder are analyzed based on the previously created pattern of association between the classification of eye usage in saccadic eye movements and the characteristics of the developmental disorder.

[0061] Next, an example of eye-hand coordination is described. In the analysis image data display step 103, the analysis image data is displayed as a content image or video in which an arbitrarily set gaze target moves regularly and the moving gaze target can be stopped by manually pressing the manual input means 7 at a predetermined position. For example, as shown in FIG. 8, there is analysis image data for playing slots. The slots are, for example, three columns and three rows, with each column rotating downward 41, as shown in FIG. 8(a). However, the slots as analysis image data may have any number of columns or rows, and the rotation direction may be upward, downward, leftward, or rightward. Then, in the task instruction step 105, for example, instructions are displayed in which the rotating slots are stopped by manually pressing the manual input means 7, starting from the left end, and the alphabet "A"s appearing on the display screen 3 are lined up in a horizontal line in the center box W1. When all the "A"s are lined up, they appear in a horizontal line, as shown in FIG. 8(c). In the target data and gaze data acquisition step 106, if the subject gazes intently at frame W2, one frame upstream from frame W1, and presses the manual input device 7 with their hand at the exact moment they determine that the position they should focus on to align the slots horizontally in frame W1 is the best position, the target data corresponds to the range of each column in frame W2. Therefore, for example, in the case of FIG. 8(a), target data 43a and gaze data 45a are stored, and in the case of FIG. 8(b), target data 43b and gaze data 45b are stored. When the subject presses the manual input device 7 with their hand while all columns are rotating, the control unit 10 causes the transmitter / receiver 15 to receive the press information and causes the image display unit 14 to stop the rotation of the leftmost column of the slots displayed on the display screen 3. Each time the subject presses the manual input device 7 with their hand, the rotation of the rightmost column stops.

[0062] In the eye usage analysis step 109, the control unit 10 causes the eye usage analysis unit 11 to generate ranges of target data 43a, 43b in XY coordinates, as shown in, for example, FIG. 9(a) or (b), and to plot the subject's gaze data 45a, 45b in chronological order at predetermined time intervals on the XY coordinates. The control unit 10 then stores, in chronological order, the gaze positions at which the subject manually pressed the manual input means 7 to stop the rotation of each row of slots. Although not shown, a diagram may be generated that displays only the deviation between the target data and the gaze data over time. In the analysis result display step 111, the control unit 10 causes the image display unit 14 to display, on the display screen 3, the diagram generated in the eye usage analysis step 109 or the diagram generated in the gaze path analysis step 110.

[0063] In the developmental disorder characteristic analysis step 112, as shown in Figures 9(a) and 9(b), for example, the characteristics of developmental disorders are analyzed based on the movement of XY-axis coordinates over time, such as the time-series trajectory and deviation amount of the gaze data 45a relative to the target data 43a and the time-series trajectory and deviation amount of the gaze data 45b relative to the target data 43b, and / or, although not shown, based on changes in the time-series deviation amount between the XY-axis coordinates of the target data 43a and 43b and the XY-axis coordinates of the gaze data 45a and 45b, and further based on the subject's manual input information on when the subject pressed the manual input means 7, and further, if necessary for the analysis, the subject's captured image screen and subject's voice information are confirmed, and further, if necessary for the analysis, the gaze path analysis image 81 is confirmed, and the characteristics of developmental disorders are analyzed based on the previously created patterns of association between the classification of eye usage in eye-hand coordination actions and the characteristics of developmental disorders.

[0064] Next, an example of gaze ability will be described. In the analysis image data display step 103, the analysis image data is displayed as a content image or video of a person's face, which is easy to analyze for gaze aversion, as shown in FIG. 10(a), for example. Then, in the task instruction step 105, in the case of FIG. 10(a), an instruction to stare intently at the face displayed on the display screen 3 is displayed, and in the case of FIG. 10(b), an instruction to stare intently at the eyes is displayed. Then, in the target data and gaze data acquisition step 106, in the case of FIG. 10(a), the target data corresponds to the area surrounded by the outline of the person's face, and the target data 50a and the subject's gaze data 55a are stored. In the case of FIG. 10(b), the target data corresponds to the area surrounding the eyes, and the target data 50b and the subject's gaze data 55b are stored.

[0065] In the eye usage analysis step 109, when the target data is, for example, the outline of a face, the control unit 10 generates an X-axis and Y-axis diagram, as shown in FIG. 11, with arrows added to indicate the range of the target data 50a and the time series of the subject's gaze data 55a. While the range of the facial target data 50a does not move, the subject's gaze data 55a moves over time. The control unit 10 then generates, in the eye usage analysis unit 11, a diagram, as shown in FIG. 11, of the time series movement of the XY-axis coordinates of the gaze data 55a relative to the XY-axis coordinates of the range of the target data 50a and / or, although not shown, a diagram of the change in the amount of deviation over time between the XY-axis coordinates of the range of the target data 50a and the XY-axis coordinates of the gaze data 55a. In the analysis result display step 111, the control unit 10 causes the image display unit 14 to display on the display screen 3 the diagram generated in the eye usage analysis step 109 or the diagram generated in the scan path analysis step 110.

[0066] In the developmental disorder characteristics analysis step 112, based on the diagram generated by the eye usage analysis unit 11, the gaze path analysis image 81 is checked if necessary for the analysis, and further, while checking the subject's image screen and subject's voice information if necessary for the analysis, the characteristics of the developmental disorder are analyzed based on the pattern of association between the classification of eye usage in gaze ability and the characteristics of the developmental disorder, which has been created in advance.

[0067] Next, an example of spatial cognitive ability will be described. In the analysis image data display step 103, the analysis image data is displayed on the display screen, and the subject is asked to perform a manual task while looking at target data. For example, as shown in FIG. 12, a figure in which a circle and a square are partially crossed is displayed. Then, in the task instruction step 105, an instruction to touch and trace the circle on the display screen 3 with a finger is displayed on the display screen 3. Then, in the target data and gaze data acquisition step 106, the target data is the line of the circle. For example, in the case of FIG. 12, the target data 73 of the line of the circle and the subject's gaze data 75 are stored in chronological order at predetermined time intervals. Then, as shown in FIG. 13, the control unit 10 causes the image display unit 14 to display a trajectory 76 traced by the subject's finger on the display screen 3, and, if necessary, records it in the memory unit 13.

[0068] In the eye usage analysis step 109, the control unit 10 causes the eye usage analysis unit 11 to generate a line of target data 73 on an X-axis and Y-axis diagram, and to add, for example, arrows so that the time series of the subject's gaze data 75 can be seen, as shown in Fig. 13, and generate a trajectory 76 traced by the subject's finger, which is subject touch input information. In the analysis result display step 111, the control unit 10 causes the image display unit 14 to display on the display screen 3 the diagram generated in the eye usage analysis step 109 or the diagram generated in the gaze path analysis step 110.

[0069] In the developmental disorder characteristics analysis step 112, the gaze path analysis image 81 is checked as needed for the analysis based on the diagram generated by the eye usage analysis unit 11 and the subject's touch input information, which is the path 76 traced by the subject's finger, and the subject's captured image and subject's voice information are checked as needed for the analysis, and the characteristics of the developmental disorder are analyzed based on the pattern of association between the classification of eye usage in spatial cognitive ability and the characteristics of the developmental disorder, which has been created in advance.

[0070] Next, an example of visual attention ability will be described. In the analysis image data display step 103, the analysis image data, which simultaneously displays multiple types of images and separates images of the same type, is displayed on the display screen 3. For example, as shown in Figure 14, there is a color palette example with different color sections, such as "yellow," "blue," "red," and "light blue." In the task instruction step 105, the display screen 3 displays instructions instructing the subject to maintain their gaze for at least 0.5 seconds on each of the light blue sections labeled "Search Color" 60 in the upper left corner. In the target data and gaze data acquisition step 106, the target data is the range of the "light blue" section. For example, in the case of Figure 14, the target data are target data 63a, 63b, and 63c indicating the sections. The target data 63a, 63b, and 63c and the subject's gaze data 65 are stored in chronological order at predetermined time intervals. In addition, when the control unit 10 detects that the subject has remained in the section of the target data 63a for a time period exceeding the threshold value of 0.5 seconds in the gaze data 65, it can also control the control unit 10 to erase the border line of the section of the target data 63a, as shown in Figure 15(b).

[0071] In the eye usage analysis step 109, the control unit 10 generates, in the eye usage analysis unit 11, a diagram of the X and Y axes, for example, with arrows to indicate the time series of the target data 63a, 63b, and 63c segments and the subject's gaze data 65, as shown in Figures 15(a) and 15(b). When the subject holds their gaze for a threshold of 0.5 seconds or more, the diagram is generated to indicate the timing at which the frame of the target data 63a segment is erased. In the analysis result display step 111, the control unit 10 causes the image display unit 14 to display on the display screen 3 the diagram generated in the eye usage analysis step 109 or the diagram generated in the gaze path analysis step 110.

[0072] In the developmental disorder characteristics analysis step 112, based on the diagram generated by the eye usage analysis unit 11, the gaze path analysis image 81 is checked if necessary for the analysis, and further, while checking the subject's image screen and subject's voice information if necessary for the analysis, the characteristics of the developmental disorder are analyzed based on the pattern of association between the classification of eye usage in gaze ability and the characteristics of the developmental disorder, which has been created in advance.

[0073] Next, the analytical information terminal 2 is provided with a program for causing the computer to function as each of the means and units included in the system 1 for analyzing characteristics of developmental disorders.

[0074] Next, the analytical information terminal 2 includes a computer-readable storage medium that stores a program for causing the computer to function as each of the means and units included in the system 1 for analyzing characteristics of developmental disorders. [Explanation of symbols]

[0075] 1. Developmental Disorder Characteristics Analysis System 2. Analysis information terminal 3 Display screen 4. Gaze detection means 5. Imaging Method 6. Audio acquisition method 7 Manual input methods 9 Wiring 10 Control Unit 11 Eye Usage Analysis Section 12 Coordinate conversion section 13 Storage section 14 Image display section 15 Transmitter / Receiver 16 Frame image generation unit 17 Gaze trajectory analysis section 18 Audio output section 20 Gaze Data 21 Target Data 22 X-axis coordinate of the movement trajectory of the object to be followed 23 Y-axis coordinate of the movement trajectory of the object to be followed 24 Line of sight X-axis coordinate 25 Line of sight Y-axis coordinate 26 X-axis deviation 27 Y-axis deviation 31 Target Data 32 Target Data 35 Gaze Data 41 Rotation direction 45 Gaze Data 50 Target Data 55 Gaze Data 63 Target Data 65 Gaze Data 73 Target Data 75 gaze data 76 Trajectory 80 frames 81 Gaze trajectory analysis image 82 Slider 100 Developmental Disorder Characteristics Analysis System Flowchart 101 Memory process 102 Selection process 103 Display process 104 Issue-solving process 105 Assignment instruction process 106 Acquisition process 107 Acquisition process 108 Conversion Process 109 Analysis process 110 Analysis process 111 Analysis result display process 112 Developmental Disorders Characteristics Analysis Process

Claims

1. A developmental disorder characteristic analysis system for analyzing characteristics of a developmental disorder of a subject, comprising: a display screen, a gaze detection means for detecting the gaze of the subject, and an analytical information terminal; a control unit of the analytical information terminal, displaying, on the image display unit, image data for analysis selected from a plurality of image data for analysis, each of which sets a task for analyzing eye usage and which is stored in the storage unit; storing, in the storage unit, target data to be gazed at in the task instructed to the subject in the displayed image data for analysis and gaze data of the subject acquired by the gaze detection means at predetermined time intervals; a coordinate conversion unit converting the stored target data and the line of sight data into XY coordinates and storing the converted data in the storage unit; causing the eye usage analysis unit to analyze the time-series movement of the XY-axis coordinates of the gaze data relative to the XY-axis coordinates of the target data, and / or the change in the amount of deviation between the XY-axis coordinates of the target data and the XY-axis coordinates of the gaze data, A system for analyzing characteristics of developmental disorders, characterized in that the analysis results of the eye usage analysis unit are displayed on an image display unit.

2. The system for analyzing characteristics of developmental disorders described in claim 1, characterized in that the analysis image data is content images or video data for analyzing the characteristics of pursuit eye movements, content images or video data for analyzing the characteristics of saccadic eye movements, content images or video data for analyzing the characteristics of coordinated movements, content images or video data for analyzing the characteristics of attention ability, or content images or video data for analyzing the characteristics of spatial cognitive ability.

3. The control unit further causing a frame image generating unit to generate a plurality of frame images at predetermined time intervals from the target data and the line of sight data stored in the storage unit, and arranging the frame images in chronological order; a gaze path analysis unit that generates a gaze path analysis image, such as a trace that tracks the movement of the subject's gaze, a gaze path that displays the order in which the subject's gaze fixates, or a heat map that displays the subject's gaze fixation time, within a time series range up to the arbitrarily specified frame image; The developmental disorder characteristic analysis system of claim 1, characterized in that the image display unit displays the frame images arranged in chronological order, a right or left scrolling function for scrolling the frame images back and forth in chronological order, and a selection function for selecting the frame images that specify a range from the instructions to the subject in the chronological order of the gaze path analysis image in a picture-in-picture format on the display screen.

4. The display screen is a touch panel type, and the device comprises an imaging means for imaging the subject, a voice acquisition means for acquiring the voice of the subject, and a manual input means for manual input by the subject, The control unit the image display unit reflects touch information on the display screen or manual input information by the manual input means on the display screen of the image data for analysis, and displays a picture-in-picture of the image captured by the imaging means on the display screen displaying the analysis result by the eye usage analysis unit or on the display screen displaying the gaze trajectory analysis image by the gaze trajectory analysis unit; 4. The system for analyzing characteristics of developmental disorders according to claim 2, wherein an audio output unit outputs the audio acquired by the audio acquisition means.

5. A program for causing the computer of the analytical information terminal to function as each component of the developmental disorder characteristics analysis system described in claim 4.

6. A computer-readable storage medium storing a program for causing the computer of the analytical information terminal to function as each component of the developmental disorder characteristics analysis system described in claim 4.

Images