A visual detection system based on children's attention

Abstract

The present invention relates to a visual detection system based on children's attention, and specifically to the field of computer vision, comprising an eye tracking module, an eye movement analysis module, an attention analysis module, a holographic linkage module and an evaluation feedback module, wherein the eye tracking module uses an eye tracker to capture children's eye movement data and constructs an eye movement three-dimensional arc surface coordinate system, the eye movement analysis module maps the gaze point coordinates in the eye movement data set to the eye movement three-dimensional arc surface coordinate system, and obtains the gaze point features, the attention analysis module associates the eye movement data set with the children's attention data, and constructs a convolutional neural network model to identify the children's attention features, the holographic linkage module overlaps and maps the heat holographic image and the physiological index holographic image on the holographic projection of the dynamic visualized eye movement three-dimensional arc surface coordinate system, and the evaluation feedback module provides a visual inspector interaction function through the dynamic holographic projection of the eye movement three-dimensional arc surface coordinate system, and provides real-time feedback on the children's visual inspection situation.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and more particularly to a visual detection system based on children's attention. Background Art

[0002] At present, there are about 23 million patients with attention deficit hyperactivity disorder aged 4-18 in my country. As a neurodevelopmental disorder, early identification and early intervention are very important for patients.

[0003] Current clinical diagnosis has high professional requirements. The condition is complicated and there are many comorbidities, which are difficult to identify and confirm during diagnosis. The basis for diagnosis mostly comes from the subjective judgment or past experience of parents, teachers and doctors. There is a lack of authoritative objective evaluation methods. The indicators of clinical diagnosis are too single, and misdiagnosis and missed diagnosis are inevitable.

[0004] Eye trackers can accurately obtain children's personalized eye movement data, and dynamic holographic projection can provide children's real-time performance in visual tasks. The development of convolutional neural networks can accurately classify and identify children's performance in different visual tasks, more accurately assess children's visual attention levels, and reduce misdiagnosis and missed diagnosis. Summary of the invention

[0005] In view of the technical problems existing in the prior art, the present invention provides a visual detection system based on children's attention, which solves the problems raised in the above background technology through an eye movement analysis module, an attention analysis module and a holographic linkage module.

[0006] The technical solution of the present invention to solve the above technical problems is as follows: comprising an eye tracking module, an eye movement analysis module, an attention analysis module, a holographic linkage module and an evaluation feedback module;

[0007] Eye tracking module: Use an eye tracker to capture children's eye movement data to form an eye movement dataset, and construct an eye movement three-dimensional arc coordinate system. Calibrate and capture the children's gaze point coordinates and gaze duration in the visual inspection cabin, check whether there are errors and inconsistent information in the eye movement dataset, and use minimum-maximum normalization to reduce the eye movement dataset to a specified range and map it to the same scale;

[0008] Eye movement analysis module: maps the fixation point coordinates in the eye movement data set to the eye movement three-dimensional arc coordinate system, and uses text descriptions and colors to mark the fixation duration and number of fixations of the fixation point coordinates, connects the fixation points in the eye movement three-dimensional arc coordinate system with arcs, and converts them into the movement trajectory of the fixation points, and obtains the fixation point features according to the eye movement data set;

[0009] Attention analysis module: calls the eye movement data set in the eye tracking module to reflect the children's attention level and preference for visual stimuli, the children's attention maintenance ability, and the children's attention allocation to different stimuli, and builds a convolutional neural network model to identify children's attention characteristics;

[0010] Holographic linkage module: Call the convolutional neural network model of the attention analysis module to identify the attention characteristics of children for data analysis, which is used to identify the distribution and duration of children's attention to different stimuli, holographically project the eye movement three-dimensional arc coordinate system through dynamic visualization, and overlap and map the thermal holographic image and the physiological index holographic image;

[0011] Evaluation and feedback module: Use the database to record the actual feedback received by children, obtain direct feedback from children's guardians through feedback forms, and obtain historical cases related to children's attention through the hospital system, which are used to adaptively adjust the parameters of the eye movement three-dimensional arc coordinate system and the content and form of feedback, and provide visual inspectors with interactive functions through dynamic holographic projection of the eye movement three-dimensional arc coordinate system;

[0012] In a preferred embodiment, the eye tracking module uses an eye tracker to capture the child's eye movement data to form an eye movement data set, and constructs an eye movement three-dimensional arc coordinate system, sets eye movement calibration points and adaptively adjusts different light environments of different gaze points, and automatically jumps to the next calibration point when the eye tracker detects that the child's gaze point coincides with the calibration point, and is used to calibrate and capture the child's gaze point coordinates and gaze duration in the visual inspection cabin, and plays an attention assessment animation through the eye tracker to obtain the number of gaze points and the scanning speed, and the obtained child's gaze point coordinates, gaze duration, number of gaze points and scanning speed are stored in the eye movement data set via wireless Bluetooth and transmitted to the eye movement three-dimensional arc coordinate system.

[0013] Furthermore, the eye movement dataset is checked for errors and inconsistent information, and preliminary data items are prepared. The preliminary data items are used to fill missing values, and the eye movement dataset is reduced to a specified range using minimum-maximum normalization and mapped to the same scale. The minimum-maximum normalization formula is:

[0014]

[0015] Where X represents the data of the normalized eye movement dataset, y represents the data of the eye movement dataset, and y min Represents the minimum value of the data in the original eye movement dataset, y max Represents the maximum value of the data in the original eye tracking dataset.

[0016] In a preferred embodiment, the eye movement analysis module maps the fixation point coordinates in the eye movement data set to the eye movement three-dimensional arc surface coordinate system, and uses text descriptions and colors to mark the fixation duration and the number of fixation points of the fixation point coordinates, connects the fixation points in the eye movement three-dimensional arc surface coordinate system through arcs, marks the scanning speed from the starting fixation point to the next fixation point and the scanning direction of the fixation point above the arc, uses the eye movement three-dimensional arc surface coordinate system, and converts the arcs connecting the fixation points in the eye movement three-dimensional arc surface coordinate system into the motion trajectory of the fixation point, displays the motion trajectory of the fixation point in the form of animation, and obtains the acceleration and dwell time of the fixation point according to the motion trajectory, wherein the specific calculation formula of the acceleration is:

[0017]

[0018] Where a represents the acceleration of the fixation point, V1 represents the scanning speed of the starting fixation point, V2 represents the scanning speed of the next fixation point, t1 represents the dwell time of the starting fixation point, and t2 represents the dwell time of the next fixation point.

[0019] Furthermore, the line of sight jump feature is obtained according to the scanning speed and scanning direction of the gaze point in the eye movement dataset, the gaze point continuous movement feature is obtained according to the movement trajectory and scanning speed of the gaze point in the eye movement dataset, and the gaze point stay feature is obtained according to the gaze duration and the coordinate position of the gaze point.

[0020] In a preferred embodiment, the attention analysis module calls the eye movement data set in the eye tracking module, wherein the distribution position of the gaze point coordinates in the eye movement three-dimensional arc coordinate system and the gaze duration reflect the child's attention degree and preference for visual stimulation, and judge that the child's attention to visual stimulation is relatively short-term, reflecting the child's attention maintenance ability, and the number of gaze points reflects the child's attention distribution to different stimuli. Frequent gaze point switching indicates that the child's attention is distracted between multiple stimuli, and less gaze point switching indicates that the child continues to pay attention to one stimulus among multiple stimuli. The scanning time reflects the speed of the child's observation and processing of stimulus information, wherein a faster scanning speed indicates that the child's attention is more distracted, and a slower scanning speed indicates that the child is thinking deeply about the stimulus and focusing on one stimulus among multiple stimuli. The eye movement data set and the child's attention data are associated through the above.

[0021] Furthermore, a convolutional neural network model is constructed, including a convolutional layer, a pooling layer, and a fully connected layer, for identifying children's attention features. The specific steps are: inputting the eye movement data set and the associated children's attention data through the convolutional layer, and performing a convolution operation on the input eye movement data set and the associated children's attention data by applying a convolution kernel to extract different data features and capture the local features and spatial structure of the eye movement three-dimensional arc coordinate system. The specific formula of the convolutional layer is:

[0022]

[0023] Where Q represents the output of extracting different data features, i and j represent the coordinate positions of the output data in the eye movement three-dimensional arc coordinate system, k represents the number of channels for outputting different data features, M and N represent the size of the convolution kernel, L represents the number of channels of the input eye movement data set and the associated children's attention data, w represents the convolution kernel, P represents the input eye movement data set and the associated children's attention data, and the pooling layer is used to reduce the dimension of the data set features mapped to the eye movement three-dimensional arc coordinate system and the feature map of its spatial structure, and the maximum pooling operation is used to select the largest eigenvalue in each pooling window as the output, which is used to retain the most significant features of the feature map and reduce the dimension of the feature map, and the average pooling operation is used to retain the feature information, which is used to reduce the number of model parameters, and the fully connected layer is used to integrate the features extracted and output by the convolution layer and the pooling layer, and output the final classification result. The specific formula of the fully connected layer is:

[0024]

[0025] Where R i Represents the output of the final classification result, S i,j represents the weight matrix, T represents the dimension of the features extracted and output by the input convolution layer and the pooling layer, and D j represents the features extracted and output by the input convolution layer and pooling layer, b i Represents the bias vector, f() represents the activation function. In the fully connected layer, each neuron is connected to the neurons in the convolutional layer and the pooling layer, and the combination and classification of features are realized by learning weights.

[0026] In a preferred embodiment, the holographic linkage module calls the convolutional neural network model of the attention analysis module to identify the attention features of the child for data analysis, so as to identify the distribution and duration of the child's attention to different stimuli, and use a dynamic gaze heat map to display the distribution of the child's gaze points in the holographic image, so as to construct a thermal holographic image, and use an eye tracker to obtain the change in the child's pupil size and a visual detection cabin to obtain the child's electroencephalogram, so as to construct a physiological indicator holographic image, which reflects the child's physiological response to different stimuli combined with the distribution and duration of attention, holographically projects the eye movement three-dimensional arc coordinate system through dynamic visualization, and overlaps and maps the thermal holographic image and the physiological indicator holographic image, and annotates the eye movement data set and its associated child attention data through text description.

[0027] In a preferred embodiment, the assessment and feedback module uses a database to record the child's actual reception of feedback, including changes in reactions, behaviors, and EEG performance, obtains direct feedback from the child's guardian through a feedback form, including their feelings and opinions on the content and form of feedback provided by the system, compares changes in eye movement data sets in different time periods and their associated child attention data through holographic projection, analyzes whether the child's performance improves after receiving feedback, and obtains historical cases related to child attention through a hospital system for adaptive adjustment of the eye movement three-dimensional arc coordinate system parameters and feedback content and form.

[0028] Furthermore, the dynamic holographic projection of the eye-movement three-dimensional arc coordinate system provides interactive functions for visual inspectors, including controlling the holographic projection through gestures, touch and language commands, using gestures to zoom in and out the image to view children's attention-related data, and using language commands to select different results, so that doctors and operators can more intuitively understand children's attention and visual ability by observing the holographic projection.

[0029] In a preferred embodiment, the method specifically includes the following steps:

[0030] S101: Use an eye tracker to capture the eye movement data of children to form an eye movement dataset, and construct a three-dimensional arc coordinate system for eye movement. Use minimum-maximum normalization to reduce the eye movement dataset to a specified range and map it to the same scale.

[0031] S102: Mapping the fixation point coordinates in the eye movement data set to the eye movement three-dimensional arc surface coordinate system, obtaining fixation point related features, converting the fixation point into a motion trajectory of the fixation point through an arc connecting the fixation points in the eye movement three-dimensional arc surface coordinate system, and obtaining the fixation point acceleration and dwell time according to the motion trajectory;

[0032] S103: Correlate the eye movement dataset with the children’s attention data, and build a convolutional neural network model, using the convolutional layer, pooling layer, and fully connected layer in the model to identify the children’s attention features;

[0033] S104: calling the convolutional neural network model of the attention analysis module to perform data analysis on the attention features of the child identified, and overlappingly mapping the heat holographic image and the physiological index holographic image on the holographic projection of the dynamic visualization eye movement three-dimensional arc coordinate system;

[0034] S105: using a database to record the actual reception of feedback by the child, obtaining direct feedback from the child's guardian through a feedback form, and providing a visual inspector interaction function through a dynamic holographic projection of an eye-movement three-dimensional arc coordinate system;

[0035] The beneficial effects of the present invention are: by capturing the eye movement data of children through an eye tracker to form an eye movement data set, and constructing a three-dimensional arc coordinate system for eye movement, the personalized eye movement data of children can be accurately obtained, which helps doctors and educational experts better understand the specific needs of each child; through the dynamic holographic projection of the three-dimensional arc coordinate system for eye movement, the real-time performance of children in visual tasks is provided, which helps doctors evaluate the treatment effect, adjust the treatment plan in time, and improve the treatment effect; visual data is provided for doctors, educational experts and parents, and professional exchanges and cooperation are promoted; the convolutional neural network can be used to accurately extract features from the image of the three-dimensional arc coordinate system for eye movement, and the performance of children in different visual tasks can be efficiently classified and identified, which helps the system to more accurately evaluate the visual attention level of children. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of the method of the present invention;

[0037] Figure 2 This is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0039] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0040] In the description of the present application, the term "for example" is used to mean "used as an example, illustration or description". Any embodiment described as "for example" in the present application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details to obscure the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0041] Example 1

[0042] This embodiment provides Figure 1 A visual detection system based on children's attention is shown, which specifically includes the following steps:

[0043] S101: Use an eye tracker to capture the eye movement data of children to form an eye movement dataset, and construct a three-dimensional arc coordinate system for eye movement. Use minimum-maximum normalization to reduce the eye movement dataset to a specified range and map it to the same scale.

[0044] S102: Mapping the fixation point coordinates in the eye movement data set to the eye movement three-dimensional arc surface coordinate system, obtaining fixation point related features, converting the fixation point into a motion trajectory of the fixation point through an arc connecting the fixation points in the eye movement three-dimensional arc surface coordinate system, and obtaining the fixation point acceleration and dwell time according to the motion trajectory;

[0045] S103: Correlate the eye movement dataset with the children’s attention data, and build a convolutional neural network model, using the convolutional layer, pooling layer, and fully connected layer in the model to identify the children’s attention features;

[0046] S104: calling the convolutional neural network model of the attention analysis module to perform data analysis on the attention features of the child identified, and overlappingly mapping the heat holographic image and the physiological index holographic image on the holographic projection of the dynamic visualization eye movement three-dimensional arc coordinate system;

[0047] S105: Using a database to record the actual feedback received by the child, obtaining direct feedback from the child's guardian through a feedback form, and providing visual inspector interaction functions through dynamic holographic projection of an eye-movement three-dimensional arc coordinate system.

[0048] Example 2

[0049] This embodiment provides Figure 2 A visual detection system based on children's attention is shown, which specifically includes: an eye tracking module, an eye movement analysis module, an attention analysis module, a holographic linkage module and an evaluation feedback module;

[0050] Eye tracking module: Use an eye tracker to capture children's eye movement data to form an eye movement dataset, and construct an eye movement three-dimensional arc coordinate system. Calibrate and capture the children's gaze point coordinates and gaze duration in the visual inspection cabin, check whether there are errors and inconsistent information in the eye movement dataset, and use minimum-maximum normalization to reduce the eye movement dataset to a specified range and map it to the same scale;

[0051] Eye movement analysis module: maps the fixation point coordinates in the eye movement data set to the eye movement three-dimensional arc coordinate system, and uses text descriptions and colors to mark the fixation duration and number of fixations of the fixation point coordinates, connects the fixation points in the eye movement three-dimensional arc coordinate system with arcs, and converts them into the movement trajectory of the fixation points, and obtains the fixation point features according to the eye movement data set;

[0052] Attention analysis module: calls the eye movement data set in the eye tracking module to reflect the children's attention level and preference for visual stimuli, the children's attention maintenance ability, and the children's attention allocation to different stimuli, and builds a convolutional neural network model to identify children's attention characteristics;

[0053] Holographic linkage module: Call the convolutional neural network model of the attention analysis module to identify the attention characteristics of children for data analysis, which is used to identify the distribution and duration of children's attention to different stimuli, holographically project the eye movement three-dimensional arc coordinate system through dynamic visualization, and overlap and map the thermal holographic image and the physiological index holographic image;

[0054] Evaluation and feedback module: Use the database to record the actual feedback received by children, obtain direct feedback from children's guardians through feedback forms, and obtain historical cases related to children's attention through the hospital system, which are used to adaptively adjust the parameters of the eye movement three-dimensional arc coordinate system and the content and form of feedback, and provide visual inspectors with interactive functions through dynamic holographic projection of the eye movement three-dimensional arc coordinate system;

[0055] S101: Use an eye tracker to capture the eye movement data of children to form an eye movement dataset, and construct a three-dimensional arc coordinate system for eye movement. Use minimum-maximum normalization to reduce the eye movement dataset to a specified range and map it to the same scale.

[0056] Furthermore, an eye tracker is used to capture the eye movement data of the child to form an eye movement data set, and an eye movement three-dimensional arc coordinate system is constructed, eye movement calibration points are set and different light environments of different gaze points are adaptively adjusted, and when the eye tracker detects that the child's gaze point coincides with the calibration point, it automatically jumps to the next calibration point to calibrate and capture the child's gaze point coordinates and gaze duration in the visual inspection cabin, and the eye tracker is used to play an attention assessment animation to obtain the number of gaze points and scanning speed. The obtained child's gaze point coordinates, gaze duration, number of gaze points and scanning speed are stored in the eye movement data set via wireless Bluetooth and transmitted to the eye movement three-dimensional arc coordinate system.

[0057] Furthermore, the eye movement dataset is checked for errors and inconsistent information, and preliminary data items are prepared. The preliminary data items are used to fill missing values, and the eye movement dataset is reduced to a specified range using minimum-maximum normalization and mapped to the same scale. The minimum-maximum normalization formula is:

[0058]

[0059] Where X represents the data of the normalized eye movement dataset, y represents the data of the eye movement dataset, and y min Represents the minimum value of the data in the original eye movement dataset, y max Represents the maximum value of the data in the original eye tracking dataset

[0060] S102: Mapping the fixation point coordinates in the eye movement data set to the eye movement three-dimensional arc surface coordinate system, obtaining fixation point related features, converting the fixation point into a motion trajectory of the fixation point through an arc connecting the fixation points in the eye movement three-dimensional arc surface coordinate system, and obtaining the fixation point acceleration and dwell time according to the motion trajectory;

[0061] Furthermore, the fixation point coordinates in the eye movement data set are mapped to the eye movement three-dimensional arc surface coordinate system, and the fixation duration and the number of fixations of the fixation point coordinates are marked with text descriptions and colors. The fixation points in the eye movement three-dimensional arc surface coordinate system are connected by arcs, and the scanning speed from the starting fixation point to the next fixation point and the scanning direction of the fixation point are marked above the arcs. The eye movement three-dimensional arc surface coordinate system is used to convert the arcs connecting the fixation points in the eye movement three-dimensional arc surface coordinate system into the motion trajectory of the fixation point, and the motion trajectory of the fixation point is displayed in the form of animation. The acceleration and dwell time of the fixation point are obtained according to the motion trajectory, wherein the specific calculation formula of the acceleration is:

[0062]

[0063] Where a represents the acceleration of the fixation point, V1 represents the scanning speed of the starting fixation point, V2 represents the scanning speed of the next fixation point, t1 represents the dwell time of the starting fixation point, and t2 represents the dwell time of the next fixation point.

[0064] Furthermore, the line of sight jump feature is obtained according to the scanning speed and scanning direction of the gaze point in the eye movement dataset, the gaze point continuous movement feature is obtained according to the movement trajectory and scanning speed of the gaze point in the eye movement dataset, and the gaze point stay feature is obtained according to the gaze duration and the coordinate position of the gaze point.

[0065] S103: Correlate the eye movement dataset with the children’s attention data, and build a convolutional neural network model, using the convolutional layer, pooling layer, and fully connected layer in the model to identify the children’s attention features;

[0066] Furthermore, the eye movement data set in the eye tracking module is called, wherein the distribution position of the fixation point coordinates in the eye movement three-dimensional arc coordinate system and the fixation duration reflect the child's attention degree and preference for visual stimulation, and judge that the child's attention to visual stimulation is relatively short-lived, reflecting the child's attention maintenance ability, and the number of fixation points reflects the child's attention distribution to different stimuli, wherein frequent fixation point switching indicates that the child's attention is distracted between multiple stimuli, and less fixation point switching indicates that the child's continuous attention to one stimulus among multiple stimuli, and the scanning time reflects the speed of the child's observation and processing of stimulus information, wherein a faster scanning speed indicates that the child's attention is more distracted, and a slower scanning speed indicates that the child is thinking deeply about the stimulus and focusing on one stimulus among multiple stimuli. The eye movement data set and the child's attention data are associated through the above.

[0067] Furthermore, a convolutional neural network model is constructed, including a convolutional layer, a pooling layer, and a fully connected layer, for identifying children's attention features. The specific steps are: inputting the eye movement data set and the associated children's attention data through the convolutional layer, and performing a convolution operation on the input eye movement data set and the associated children's attention data by applying a convolution kernel to extract different data features and capture the local features and spatial structure of the eye movement three-dimensional arc coordinate system. The specific formula of the convolutional layer is:

[0068]

[0069] Where Q represents the output of extracting different data features, i and j represent the coordinate positions of the output data in the eye movement three-dimensional arc coordinate system, k represents the number of channels for outputting different data features, M and N represent the size of the convolution kernel, L represents the number of channels of the input eye movement data set and the associated children's attention data, w represents the convolution kernel, P represents the input eye movement data set and the associated children's attention data, and the pooling layer is used to reduce the dimension of the data set features mapped to the eye movement three-dimensional arc coordinate system and the feature map of its spatial structure, and the maximum pooling operation is used to select the largest eigenvalue in each pooling window as the output, which is used to retain the most significant features of the feature map and reduce the dimension of the feature map, and the average pooling operation is used to retain the feature information, which is used to reduce the number of model parameters, and the fully connected layer is used to integrate the features extracted and output by the convolution layer and the pooling layer, and output the final classification result. The specific formula of the fully connected layer is:

[0070]

[0071] Where R i Represents the output of the final classification result, S i,j represents the weight matrix, T represents the dimension of the features extracted and output by the input convolution layer and the pooling layer, and D j represents the features extracted and output by the input convolution layer and pooling layer, b i Represents the bias vector, f() represents the activation function. In the fully connected layer, each neuron is connected to the neurons in the convolutional layer and the pooling layer, and the combination and classification of features are realized by learning weights.

[0072] S104: calling the convolutional neural network model of the attention analysis module to perform data analysis on the attention features of the child identified, and overlappingly mapping the heat holographic image and the physiological index holographic image on the holographic projection of the dynamic visualization eye movement three-dimensional arc coordinate system;

[0073] Furthermore, the convolutional neural network model of the attention analysis module is called to perform data analysis on the attention features of the children identified to identify the distribution and duration of children's attention to different stimuli. A dynamic gaze heat map is used to display the distribution of children's gaze points in the holographic image to construct a heat holographic image. An eye tracker is used to obtain changes in children's pupil size and a visual detection cabin is used to obtain children's electroencephalograms to construct a physiological indicator holographic image that reflects children's physiological responses to different stimuli combined with the distribution and duration of their attention. The eye movement three-dimensional arc coordinate system is holographically projected through dynamic visualization, and the heat holographic image and the physiological indicator holographic image are overlapped and mapped. The eye movement data set and its associated children's attention data are annotated through text descriptions.

[0074] S105: using a database to record the actual reception of feedback by the child, obtaining direct feedback from the child's guardian through a feedback form, and providing a visual inspector interaction function through a dynamic holographic projection of an eye-movement three-dimensional arc coordinate system;

[0075] Furthermore, the database is used to record the children's actual reception of feedback, including changes in reactions, behaviors, and EEG performance. Direct feedback from the children's guardians is obtained through feedback forms, including their feelings and opinions on the content and form of feedback provided by the system. Holographic projection is used to compare changes in eye movement data sets in different time periods and their associated children's attention data, and to analyze whether the children's performance improves after receiving feedback. Historical case records related to children's attention are obtained through the hospital system for adaptive adjustment of the parameters of the eye movement three-dimensional arc coordinate system and the content and form of feedback.

[0076] Furthermore, the dynamic holographic projection of the eye-movement three-dimensional arc coordinate system provides interactive functions for visual inspectors, including controlling the holographic projection through gestures, touch and language commands, using gestures to zoom in and out the image to view children's attention-related data, and using language commands to select different results, so that doctors and operators can more intuitively understand children's attention and visual ability by observing the holographic projection.

[0077] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0078] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0080] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0081] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0082] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0083] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A visual detection system based on children's attention, characterized in that: The specific steps include: S101: Use an eye tracker to capture the eye movement data of children to form an eye movement dataset, and construct a three-dimensional arc coordinate system for eye movement. Use minimum-maximum normalization to reduce the eye movement dataset to a specified range and map it to the same scale. In the S101, an eye tracker is used to capture the eye movement data of the child to form an eye movement data set, and an eye movement three-dimensional arc surface coordinate system is constructed, eye movement calibration points are set and different light environments of different gaze points are adaptively adjusted, and when the eye tracker detects that the child's gaze point coincides with the calibration point, it automatically jumps to the next calibration point to calibrate and capture the child's gaze point coordinates and gaze duration in the visual inspection cabin, and the eye tracker is used to play an attention assessment animation to obtain the number of gaze points and the scanning speed, and the obtained child's gaze point coordinates, gaze duration, number of gaze points, and scanning speed are stored in the eye movement data set through wireless Bluetooth and transmitted to the eye movement three-dimensional arc surface coordinate system, and the eye movement data set is reduced to a specified range by using minimum-maximum normalization, and mapped to the same scale; S102: Mapping the fixation point coordinates in the eye movement data set to the eye movement three-dimensional arc surface coordinate system, obtaining fixation point related features, converting the fixation point into a motion trajectory of the fixation point through an arc connecting the fixation points in the eye movement three-dimensional arc surface coordinate system, and obtaining the fixation point acceleration and dwell time according to the motion trajectory; In S102, the fixation point coordinates in the eye movement data are mapped to the eye movement three-dimensional arc surface coordinate system, and the fixation duration and the number of fixation points of the fixation point coordinates are marked with text descriptions and colors, the fixation points in the eye movement three-dimensional arc surface coordinate system are connected by arcs, and the scanning speed from the starting fixation point to the next fixation point and the scanning direction of the fixation point are marked above the arcs. The eye movement three-dimensional arc surface coordinate system is used to convert the arcs connecting the fixation points in the eye movement three-dimensional arc surface coordinate system into the motion trajectory of the fixation point, and the motion trajectory of the fixation point is displayed in the form of animation, and the acceleration and dwell time of the fixation point are obtained according to the motion trajectory, wherein the specific calculation formula of the acceleration is: in represents the gaze point acceleration, represents the scanning speed of the starting fixation point, represents the scanning speed of the next fixation point, represents the dwell time of the starting fixation point, Indicates the next fixation point’s dwell time point, obtains sight jump features, viewpoint continuous movement features, and fixation point dwell features; S103: Correlate the eye movement dataset with the children’s attention data, and build a convolutional neural network model, using the convolutional layer, pooling layer, and fully connected layer in the model to identify the children’s attention features; In the S103, the eye movement data set in the eye tracking module is called, and the distribution position of the fixation point coordinates in the eye movement three-dimensional arc coordinate system and the fixation duration are used to reflect the child's attention degree and preference for visual stimulation, and it is judged that the child's attention to visual stimulation is relatively short-term, reflecting the child's attention maintenance ability, and the number of fixation points is used to reflect the child's attention distribution to different stimuli, wherein frequent fixation point switching indicates that the child's attention is scattered among multiple stimuli, and less fixation point switching indicates that the child continues to pay attention to one stimulus among multiple stimuli, and the scanning time is used to reflect the speed of the child in observing and processing stimulus information, wherein a faster scanning speed indicates that the child's attention is more scattered, and a slower scanning speed indicates that the child is thinking deeply about the stimulus and focusing on one stimulus among multiple stimuli, and the eye movement data set and the child's attention data are associated in the above way to construct a convolutional neural network model; The convolutional neural network model includes a convolution layer, a pooling layer and a fully connected layer, which is used to identify the attention characteristics of children. The specific steps are: inputting the eye movement data set and the associated child attention data through the convolution layer, and performing a convolution operation on the input eye movement data set and the associated child attention data by applying a convolution kernel to extract different data features and capture the local features and spatial structure of the eye movement three-dimensional arc coordinate system. The specific formula of the convolution layer is: in Represents the output of extracting different data features, They respectively represent the coordinate positions of the output data in the eye movement three-dimensional arc coordinate system, Indicates the number of channels for outputting different data features. represents the size of the convolution kernel, represents the number of channels of the input eye tracking dataset and the associated child attention data, represents the convolution kernel, Represents the input eye movement dataset and the associated children's attention data. The pooling layer is used to reduce the dimension of the dataset features and the feature map of its spatial structure mapped to the eye movement three-dimensional arc coordinate system. The maximum pooling operation is used to select the largest eigenvalue as the output in each pooling window to retain the most significant features of the feature map and reduce the dimension of the feature map. The average pooling operation is used to retain the feature information to reduce the number of model parameters. The fully connected layer is used to integrate the features extracted and output by the convolution layer and the pooling layer, and output the final classification result. The specific formula of the fully connected layer is: in Represents the output of the final classification result, represents the weight matrix, Represents the dimension of the features extracted and output by the input convolutional layer and pooling layer, Represents the features extracted and output by the input convolutional layer and pooling layer, represents the bias vector, Represents the activation function. In the fully connected layer, each neuron is connected to the neurons in the convolutional layer and the pooling layer, and the combination and classification of features are achieved by learning weights; S104: calling the convolutional neural network model of the attention analysis module to perform data analysis on the attention features of the child identified, and overlappingly mapping the heat holographic image and the physiological index holographic image on the holographic projection of the dynamic visualization eye movement three-dimensional arc coordinate system; In the S104, the attention features of the child identified by the convolutional neural network model of the attention analysis module are called to perform data analysis to identify the distribution and duration of the child's attention to different stimuli, and a dynamic gaze heat map is used to display the distribution of the child's gaze points in the holographic image to construct a heat holographic image, and an eye tracker is used to obtain the change in the child's pupil size and a visual detection cabin is used to obtain the child's electroencephalogram to construct a physiological index holographic image to reflect the child's physiological response to different stimuli combined with the distribution and duration of attention, and the eye movement three-dimensional arc coordinate system is holographically projected through dynamic visualization, and the heat holographic image and the physiological index holographic image are overlapped and mapped, and the eye movement data set and its associated child attention data are annotated through text descriptions; S105: Using a database to record the actual feedback received by the child, obtaining direct feedback from the child's guardian through a feedback form, and providing visual inspector interaction functions through dynamic holographic projection of an eye-movement three-dimensional arc coordinate system.

2. A visual detection system based on children's attention according to claim 1, characterized in that: The minimum-maximum normalization checks whether there are errors and inconsistent information in the eye movement dataset, prepares preliminary data items, fills missing values ​​with the preliminary data items, reduces the eye movement dataset to a specified range using the minimum-maximum normalization, and maps it to the same scale. The minimum-maximum normalization formula is: in Represents the data of the normalized eye movement dataset, represents the data of the eye-tracking dataset, represents the minimum value of the data in the original eye movement dataset, Represents the maximum value of the data in the original eye tracking dataset.

3. A visual detection system based on children's attention according to claim 1, characterized in that: The gaze jump feature is obtained through the scanning speed and scanning direction of the gaze point in the eye movement data set, the gaze point continuous movement feature is obtained according to the movement trajectory of the gaze point in the eye movement data set and the scanning speed of the gaze point, and the gaze point stay feature is obtained according to the gaze duration and the gaze point coordinate position.

4. A visual detection system based on children's attention according to claim 1, characterized in that: In S105, a database is used to record the actual feedback received by the child, including changes in reaction, behavior and EEG performance. Direct feedback from the child's guardian is obtained through a feedback form, including their feelings and opinions on the content and form of feedback provided by the system. The changes in eye movement data sets in different time periods and their associated child attention data are compared through holographic projection to analyze whether the child's performance improves after receiving feedback. Historical cases related to child attention are obtained through the hospital system for adaptive adjustment of eye movement three-dimensional arc coordinate system parameters and feedback content and form.

5. A visual detection system based on children's attention according to claim 4, characterized in that: The eye-movement three-dimensional arc coordinate system adopts dynamic holographic projection, and provides visual inspectors with interactive functions through dynamic holographic projection, including controlling the holographic projection through gestures, touch and language commands, using gestures to zoom in and out images to view children's attention-related data, and using language commands to select different results, so that doctors and operators can more intuitively understand children's attention and visual ability by observing holographic projection.

Images