Intelligent identification method, system, device and storage medium for developmental dyslexia

Abstract

The present invention discloses a method, system, device and storage medium for intelligent identification of developmental dyslexia. The method for intelligent identification of developmental dyslexia includes: obtaining eye movement signals of a subject; inputting the eye movement signals into a preset first recognition model for processing to obtain a first recognition probability that the subject suffers from developmental dyslexia; extracting specified feature data from the eye movement signals, inputting the feature data into a preset second recognition model for processing to obtain a second recognition probability corresponding to the subject suffering from developmental dyslexia; fusing the first recognition probability and the second recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia. By utilizing the modeling results of the shallow learning model and the deep learning model to have strong complementarity, the two models are integrated using model fusion technology to ensure the accuracy of identifying the subject's developmental dyslexia.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a method, system, device and storage medium for intelligently identifying developmental dyslexia. Background Art

[0002] Developmental dyslexia (DD) refers to an individual who has no significant differences from other individuals in terms of general intelligence, motivation, living environment and educational conditions, and has no obvious visual, hearing or nervous system damage, but whose reading scores are significantly lower than the average level of their peers. Developmental dyslexia is a common disorder among children and adolescents, with an incidence rate of 5%-12%, of which more than 70% of children and adolescents with disabilities have difficulty learning in school.

[0003] In the diagnosis of developmental dyslexia, because it is not considered a clinical disease and the pathogenic mechanism is still unclear, the hospital does not have special diagnosis and treatment services. The diagnosis of this disorder generally requires patients to be tested through a test scale and collect test data, and then evaluated by professionals who have received rigorous professional training based on the test data and diagnostic experience. There are many difficulties in the current diagnosis process of developmental dyslexia: the technical cost of implementation is high, the accuracy of the diagnostic results depends on the professional level of professionals, the accuracy is not high, and there is a lack of unified standards. Obviously, this diagnostic method cannot be used for screening of a large number of children. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of the prior art that developmental dyslexia needs to be manually diagnosed, the technical cost of implementation is high, the accuracy of the diagnostic results depends on the professional level of professionals, the accuracy is not high, and there is a lack of unified standards. A method, system, device and storage medium for intelligent identification of developmental dyslexia are provided.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] The present invention provides a method for intelligently identifying developmental dyslexia, the method comprising:

[0007] Obtaining the eye movement signals of the subject;

[0008] Inputting the eye movement signal into a preset first recognition model for processing to obtain a first recognition probability that the subject suffers from developmental dyslexia;

[0009] Extracting specified feature data from the eye movement signal, inputting the feature data into a preset second recognition model for processing, and obtaining a second recognition probability corresponding to the subject suffering from developmental dyslexia;

[0010] The first recognition probability and the second recognition probability are combined to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia.

[0011] Preferably, before fusing the first recognition probability and the second recognition probability, the method for intelligently identifying developmental dyslexia further comprises:

[0012] Acquiring multimodal data for characterizing the reading ability of the subject;

[0013] Inputting the multimodal data into a third recognition model for processing to obtain a third recognition probability that the subject suffers from developmental dyslexia;

[0014] The step of fusing the first recognition probability and the second recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia includes:

[0015] The first recognition probability, the second recognition probability and the third recognition probability are integrated to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia.

[0016] Preferably, the first recognition model is a bidirectional long short-term memory neural network;

[0017] Before inputting the eye movement signal into the first recognition model for processing, the method further includes:

[0018] The eye movement signal is processed uniformly in segments.

[0019] Preferably, the second recognition model is an integrated decision tree model.

[0020] Preferably, before inputting the multimodal data into the third recognition model for processing, the method further comprises:

[0021] The multimodal data is preprocessed, wherein the preprocessing includes at least one of quantization processing, missing value filling processing, and normalization processing.

[0022] Preferably, extracting specified feature data from the eye movement signal includes:

[0023] At least one of the following feature data is extracted from the eye movement signal: coordinate position information of the eye gazing at the screen, gaze duration and / or pupil size information of the eye.

[0024] Preferably, after obtaining the comprehensive recognition probability corresponding to the eye movement signal, the method further includes:

[0025] It is determined whether the comprehensive recognition probability reaches a preset value. If so, it is determined that the subject suffers from developmental dyslexia. Otherwise, it is determined that the subject does not suffer from developmental dyslexia.

[0026] The present invention also provides a developmental dyslexia intelligent recognition system, the developmental dyslexia intelligent recognition system comprising:

[0027] An eye movement signal acquisition module, used to acquire the eye movement signal of the subject;

[0028] A first recognition module, used for inputting the eye movement signal into a preset first recognition model for processing, and obtaining a first recognition probability that the subject suffers from developmental dyslexia;

[0029] A second recognition module is used to extract specified feature data from the eye movement signal, input the feature data into a preset second recognition model for processing, and obtain a second recognition probability corresponding to the subject suffering from developmental dyslexia;

[0030] The recognition probability fusion module is used to fuse the first recognition probability and the second recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia.

[0031] Preferably, the system further comprises:

[0032] A multimodal data acquisition module, used to acquire multimodal data for characterizing the reading ability of the subject;

[0033] A third recognition module, used for inputting the multimodal data into a third recognition model for processing to obtain a third recognition probability that the subject suffers from developmental dyslexia;

[0034] The recognition probability fusion module is specifically used to fuse the first recognition probability, the second recognition probability and the third recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia.

[0035] Preferably, the first recognition model is a bidirectional long short-term memory neural network;

[0036] The system further comprises an eye movement signal preprocessing module, which is used for uniformly segmenting the eye movement signal before the first recognition module performs corresponding operations.

[0037] Preferably, the second recognition model is an integrated decision tree model.

[0038] Preferably, the system further comprises:

[0039] The multimodal data preprocessing module is used to preprocess the multimodal data, wherein the preprocessing includes at least one of quantization processing, missing value filling processing and normalization processing.

[0040] Preferably, the feature data extracted from the eye movement signal by the second recognition module includes at least one of the following feature data: coordinate position information of the eye's gaze on the screen, gaze duration and / or pupil size information of the eye.

[0041] Preferably, the system further comprises a dyslexia determination module for determining whether the comprehensive recognition probability reaches a preset value; if so, it is determined that the subject suffers from developmental dyslexia; otherwise, it is determined that the subject does not suffer from developmental dyslexia.

[0042] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned intelligent identification method for developmental dyslexia when executing the computer program.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for intelligently identifying developmental dyslexia as described above is implemented.

[0044] The positive and progressive effects of the present invention are:

[0045] After acquiring the eye movement signal of the subject, the present invention directly inputs the eye movement signal into the first model for developmental dyslexia recognition, and uses the second recognition model to recognize developmental dyslexia according to the specified feature data of the eye movement signal, and then fuses the two recognition results, which reduces the deviation problem caused by only using the specified feature data for developmental dyslexia recognition to a certain extent, and can obtain a more accurate recognition result. Therefore, the present invention can automatically and accurately recognize whether the subject suffers from developmental dyslexia, avoiding the corresponding defects caused by manual diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of the method for intelligently identifying developmental dyslexia according to Embodiment 1 of the present invention.

[0047] Figure 2 This is a flow chart of the method for intelligently identifying developmental dyslexia according to Embodiment 2 of the present invention.

[0048] Figure 3 This is a schematic diagram of the intelligent identification method for developmental dyslexia according to Embodiment 2 of the present invention.

[0049] Figure 4This is a structural block diagram of the intelligent recognition system for developmental dyslexia according to Embodiment 3 of the present invention.

[0050] Figure 5 This is a structural block diagram of the intelligent recognition system for developmental dyslexia according to embodiment 4 of the present invention.

[0051] Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0052] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0053] Example 1

[0054] like Figure 1 As shown, this embodiment provides a method for intelligently identifying developmental dyslexia. The method for intelligently identifying developmental dyslexia includes:

[0055] S101, obtaining eye movement signals of the subject.

[0056] S102: Input the eye movement signal into a preset first recognition model for processing to obtain a first recognition probability that the subject suffers from developmental dyslexia.

[0057] The first recognition model may be a recognition model using deep learning technology, which may directly extract features from eye movement signals and predict whether the subject has developmental dyslexia.

[0058] S103, extracting designated feature data from the eye movement signal, inputting the feature data into a preset second recognition model for processing, and obtaining a second recognition probability corresponding to the subject suffering from developmental dyslexia.

[0059] The second recognition model may be a shallow learning model. The feature data of pre-defined eye movement indicators are input into the shallow learning model by means of feature engineering to predict whether the subject has developmental dyslexia.

[0060] It can be seen that the feature data input into the second recognition model are manual features, that is, eye movement behavior indicators with strong prior information, and these eye movement indicators obtained by using manual features may have lost strong and effective information for the task, that is, the manually extracted features may not be the optimal solution for the task to a large extent; in contrast, the input of the first recognition model is the most original eye movement coordinate timing signal, and deep learning technology is used to automatically extract task-specific effective features, which largely avoids the problem of manual features carrying too strong prior information. The solution space for feature extraction is larger, and it is easier to find a better solution.

[0061] S104, fusing the first recognition probability and the second recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia.

[0062] In summary, after acquiring the eye movement signal of the subject, the present invention directly inputs the eye movement signal into the first model for developmental dyslexia recognition, and uses the second recognition model to recognize developmental dyslexia according to the specified feature data of the eye movement signal, and then fuses the two recognition results, which reduces the deviation problem caused by only using the specified feature data for developmental dyslexia recognition to a certain extent, and can obtain a more accurate recognition result. Therefore, the present invention can automatically and accurately recognize whether the subject suffers from developmental dyslexia, avoiding the corresponding defects caused by manual diagnosis.

[0063] In addition, the intelligent identification method for developmental dyslexia in this embodiment uses a shallow learning model and a deep learning model to jointly build a model. The modeling results of the shallow learning model and the deep learning model are highly complementary. The deep learning model can alleviate the feature deviation problem caused by the preset eye movement indicators. The integration of the two models using model fusion technology can further improve the performance of the model, thereby ensuring the accuracy of identifying the developmental dyslexia of the subject.

[0064] Example 2

[0065] like Figure 2 and 3 As shown, the intelligent identification method for developmental dyslexia in this embodiment is a further improvement on Embodiment 1, specifically:

[0066] Step S101 can obtain the eye movement signal of the subject by the following method:

[0067] In this embodiment, the user's eye movement signals can be collected during the short reading ability test of the subject using an eye tracker. Usually, the correct answer rate of patients with developmental dyslexia is low; and during the reading process, the patient's eye movement signals are different from those of normal people. Therefore, by recording the eye movement signals during the reading process, the reading ability and disability level of the subject can be comprehensively and objectively determined.

[0068] Considering that the reading ability of the subjects is in dynamic development, the reading texts used in the eye movement test are selected according to age, for example, from 7 to 16 years old, each age is benchmarked with 3 basic level articles. In order to obtain the reading efficiency of the subjects, each article is followed by multiple-choice questions based on the content of the article. The subjects select through the response box buttons, and the eye tracker system can record the reaction time and accuracy.

[0069] In order to fully evaluate the actual reading development level of the test subjects, the test subjects aged n are tested using nine articles, namely n-1, n, and n+1. For special test subjects, the test age standard can be adjusted according to specific circumstances (e.g., the test age standard can be lowered according to the actual intelligence level of children with intellectual retardation).

[0070] In this embodiment, the first recognition model used in step S102 is preferably a bidirectional long short-term memory neural network (Bi-LSTM). Of course, any other suitable deep learning model can also be used for implementation, and this embodiment does not impose any specific restrictions on this. When the first recognition model is Bi-LSTM, before step S102, the intelligent recognition method for developmental dyslexia also includes:

[0071] S201, performing uniform segmentation processing on the eye movement signal.

[0072] Among them, the preset duration d of each eye movement signal is preferably 3s (seconds), and of course the duration can be adjusted as needed. This embodiment does not limit the specific duration of each eye movement signal. Since the amount of original data of the eye movement signal is large, it is reflected in the high sampling frequency and long sampling duration of the instrument. In order to ensure feasibility, the signal needs to be segmented (such as uniformly sampling a 30s signal into 10 segments of 3s signal) before the signal can be input into the commonly used deep neural network for modeling time series signals: Bi-LSTM. By uniformly segmenting the eye movement signal, the timing of each signal can be reflected. Preferably, before the eye movement signal is uniformly segmented, this embodiment can also pre-normalize it.

[0073] In this embodiment, step S103 specifically includes:

[0074] S202. Extract at least one of the following feature data from the eye movement signal: coordinate position information of the eye gazing at the screen, gaze duration, and / or pupil size information of the eye.

[0075] S203, inputting the characteristic data into a preset second recognition model for processing to obtain a second recognition probability corresponding to the subject suffering from developmental dyslexia.

[0076] Specifically, the second recognition model is an integrated decision tree model (Gradient Boosting Decision Tree, GBDT); of course, any other suitable shallow learning model can also be used for implementation, and this embodiment does not impose any specific restrictions on this.

[0077] In this example, the bidirectional long short-term memory network directly extracts features from eye movement signals, which to some extent alleviates the feature deviation problem caused by the preset eye movement indicators. The integrated decision tree model has strong nonlinear fitting ability and can better adapt to the complexity of developmental dyslexia identification.

[0078] Among them, the formula of Bi-LSTM network is:

[0079]

[0080]

[0081]

[0082] in, represents the forward\reverse input of the eye movement signal in time series, W and b are the parameters of the fully connected layer of the model, σ(·) is the Sigmoid function, Identifying probabilities for developmental dyslexia corresponding to eye movement signals.

[0083] In this embodiment, before step S104, the method for intelligently identifying developmental dyslexia further includes:

[0084] S204: Obtain multimodal data for characterizing the reading ability of the subject.

[0085] Specifically, the multimodal data used to characterize the reading ability of the testee may include behavioral indicators of eye movement tests (i.e., the testee's reaction time and accuracy) and the testee's vision (such as vision in the past six months). If the testee is a student, the multimodal data may also include the testee's academic performance, such as collecting the testee's Chinese and math scores within the most recent predetermined period (six months), and using the average score as the school score. If there is only one test in the past six months, the single score is used.

[0086] S206: Input the multimodal data into a third recognition model for processing to obtain a third recognition probability that the subject suffers from developmental dyslexia.

[0087] The third recognition model can select a shallow learning model, such as an integrated decision tree model.

[0088] In addition, before step S206, the method for intelligently identifying developmental dyslexia further includes:

[0089] S205: Preprocess the multimodal data. For example, the preprocessing includes at least one of quantization processing, missing value filling processing, and normalization processing.

[0090] Among them, quantification processing refers to converting non-numerical data into digital data, such as converting the grade "A" into 90 points; missing value filling processing refers to filling missing data according to a predetermined strategy; normalization processing refers to mapping data to the range of 0 to 1.

[0091] In this embodiment, step S104 specifically includes:

[0092] S207: Fusing the first recognition probability, the second recognition probability and the third recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia. In this embodiment, for example, the recognition probabilities may be fused in a weighted average manner.

[0093] Given that the test results of the literacy assessment and the reading and writing test are very closely related, it is also possible to consider combining the attributes of the two assessment results so that the machine learning modeling process can use the relationship information between the two to jointly predict the degree of dyslexia of the subject. Therefore, the multimodal data in this embodiment can also include the literacy assessment results of the subject.

[0094] In another example, after step S207, the method for intelligently identifying developmental dyslexia further includes:

[0095] S208, determining whether the comprehensive recognition probability reaches a preset value, if so, determining that the subject suffers from developmental dyslexia, otherwise, determining that the subject does not suffer from developmental dyslexia.

[0096] Specifically, in step S208, the preset value is 0.5, that is, when the diagnosis probability of developmental dyslexia of the subject reaches 0.5, it is diagnosed that the subject suffers from developmental dyslexia, and when it is less than 0.5, it is diagnosed that the subject does not suffer from developmental dyslexia.

[0097] The intelligent identification method for developmental dyslexia in this embodiment processes eye movement signals in segments and inputs them into a bidirectional long-short-term memory neural network, extracts specified features from eye movement signals and inputs them into an integrated decision tree model, and preprocesses multimodal data before inputting them into an integrated decision tree model. The modeling results of the integrated decision tree model and the bidirectional long-short-term memory neural network are highly complementary. The deep learning model directly processes eye movement signals to alleviate the feature deviation problem caused by preset eye movement indicators. The integration of deep and shallow models using model fusion technology can further improve the performance of the model, thereby ensuring the accuracy of identifying the developmental dyslexia of the subject.

[0098] Example 3

[0099] like Figure 4As shown, this embodiment provides a developmental dyslexia intelligent identification system. The developmental dyslexia intelligent identification system includes:

[0100] The eye movement signal acquisition module 1 is used to acquire the eye movement signal of the subject.

[0101] The first recognition module 2 is used to input the eye movement signal into a preset first recognition model for processing, and obtain a first recognition probability that the subject suffers from developmental dyslexia.

[0102] The first recognition model may be a recognition model using deep learning technology, which may directly extract features from eye movement signals and predict whether the subject has developmental dyslexia.

[0103] The second recognition module 3 is used to extract specified feature data from the eye movement signal, input the feature data into a preset second recognition model for processing, and obtain a second recognition probability corresponding to the subject suffering from developmental dyslexia.

[0104] The second recognition model may be a shallow learning model. The feature data of the preset eye movement index is input into the shallow learning model by means of feature engineering to predict whether the subject has developmental dyslexia.

[0105] It can be seen that the feature data input into the second recognition model are manual features, that is, eye movement behavior indicators with strong prior information, and these eye movement indicators obtained by using manual features may have lost strong and effective information for the task, that is, the manually extracted features may not be the optimal solution for the task to a large extent; in contrast, the input of the first recognition model is the most original eye movement coordinate timing signal, and deep learning technology is used to automatically extract task-specific effective features, which largely avoids the problem of manual features carrying too strong prior information. The solution space for feature extraction is larger, and it is easier to find a better solution.

[0106] The recognition probability fusion module 4 is used to fuse the first recognition probability and the second recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia.

[0107] In summary, after acquiring the eye movement signal of the subject, the present invention directly inputs the eye movement signal into the first model for developmental dyslexia recognition, and uses the second recognition model to recognize developmental dyslexia according to the specified feature data of the eye movement signal, and then fuses the two recognition results, which reduces the deviation problem caused by only using the specified feature data for developmental dyslexia recognition to a certain extent, and can obtain a more accurate recognition result. Therefore, the present invention can automatically and accurately recognize whether the subject suffers from developmental dyslexia, avoiding the corresponding defects caused by manual diagnosis.

[0108] In addition, the developmental dyslexia intelligent identification system in this embodiment is jointly modeled by using a shallow learning model and a deep learning model. The modeling results of the shallow learning model and the deep learning model are highly complementary. The deep learning model can alleviate the feature deviation problem caused by the preset eye movement indicators. The integration of the two models using model fusion technology can further improve the performance of the model, thereby ensuring the accuracy of identifying the developmental dyslexia of the subject.

[0109] Example 4

[0110] like Figure 5 As shown, the intelligent recognition system for developmental dyslexia in this embodiment is a further improvement of the embodiment 3, specifically:

[0111] The eye movement signal acquisition module 1 can acquire the eye movement signal of the subject by the following method:

[0112] In this embodiment, the user's eye movement signals can be collected during the short reading ability test of the subject using an eye tracker. During the test of the subject, the eye tracker will record the coordinates of the subject's eyes looking at the screen. Usually, patients with developmental dyslexia have a low accuracy rate in answering questions; and during the reading process, the patient's various reading gaze characteristics (such as the number of glances back, gaze time and other indicators) are different from those of normal people. Therefore, by recording the eye movement signals during the reading process, the reading ability and disability level of the subject can be comprehensively and objectively determined.

[0113] Considering that the reading ability of the subjects is in dynamic development, the reading texts used in the eye movement test are selected according to age, for example, from 7 to 16 years old, each age is benchmarked with 3 basic level articles. In order to obtain the reading efficiency of the subjects, each article is followed by multiple-choice questions based on the content of the article. The subjects select through the response box buttons, and the eye tracker system can record the reaction time and accuracy.

[0114] In order to fully evaluate the actual reading development level of the test subjects, the test subjects aged n are tested using nine articles, namely n-1, n, and n+1. For special test subjects, the test age standard can be adjusted according to specific circumstances (e.g., the test age standard can be lowered according to the actual intelligence level of children with intellectual retardation).

[0115] In this embodiment, the first recognition model used in the first recognition module 2 is preferably a bidirectional long short-term memory neural network (Bidirectional long short-term memory network, Bi-LSTM), and of course, any other suitable deep learning model can be used for implementation, and this embodiment does not impose any specific restrictions on this. When the first recognition model is Bi-LSTM, the developmental dyslexia intelligent recognition system also includes an eye movement signal preprocessing module 5, which is used to uniformly segment the eye movement signal before the first recognition module performs the corresponding operation.

[0116] Among them, the preset duration d of each eye movement signal is preferably 3s (seconds), and of course the duration can be adjusted as needed. This embodiment does not limit the specific duration of each eye movement signal. Since the amount of original data of the eye movement signal is large, it is reflected in the high sampling frequency and long sampling duration of the instrument. In order to ensure feasibility, the signal needs to be segmented (such as uniformly sampling a 30s signal into 10 segments of 3s signal) before the signal can be input into the commonly used deep neural network for modeling time series signals: Bi-LSTM. By uniformly segmenting the eye movement signal, the timing of each signal can be reflected. Preferably, before the eye movement signal is uniformly segmented, this embodiment can also pre-normalize it.

[0117] In this embodiment, the feature data extracted from the eye movement signal by the second recognition module 3 includes at least one of the following feature data: coordinate position information of the eye gazing at the screen, gaze duration and / or pupil size information of the eye.

[0118] Specifically, the second recognition model is an integrated decision tree model; of course, any other suitable shallow learning model can also be used to implement it, and this embodiment does not impose any specific restrictions on this.

[0119] In this example, the bidirectional long short-term memory network directly extracts features from eye movement signals, which to some extent alleviates the feature deviation problem caused by the preset eye movement indicators. The integrated decision tree model has strong nonlinear fitting ability and can better adapt to the complexity of developmental dyslexia identification.

[0120] Among them, the formula of Bi-LSTM network is:

[0121]

[0122]

[0123]

[0124] in, represents the forward\reverse input of the eye movement signal in time series, W and b are the parameters of the fully connected layer of the model, σ(·) is the Sigmoid function, Identifying probabilities for developmental dyslexia corresponding to eye movement signals.

[0125] In this embodiment, the developmental dyslexia intelligent identification system further includes:

[0126] The multimodal data acquisition module 6 is used to acquire multimodal data for characterizing the reading ability of the subject.

[0127] Specifically, the multimodal data used to characterize the reading ability of the testee may include behavioral indicators of eye movement tests (i.e., the testee's reaction time and accuracy) and the testee's vision (such as vision in the past six months). If the testee is a student, the multimodal data may also include the testee's academic performance, such as collecting the testee's Chinese and math scores within the most recent predetermined period (six months), and using the average score as the school score. If there is only one test in the past six months, the single score is used.

[0128] The third recognition module 7 is used to input the multimodal data into a third recognition model for processing to obtain a third recognition probability that the subject suffers from developmental dyslexia.

[0129] The third recognition model can select a shallow learning model, such as an integrated decision tree model.

[0130] In addition, the developmental dyslexia intelligent identification system also includes:

[0131] The multimodal data preprocessing module 8 is used to preprocess the multimodal data. For example, the preprocessing includes at least one of quantization processing, missing value filling processing and normalization processing.

[0132] Among them, quantification processing refers to converting non-numerical data into digital data, such as converting the grade "A" into 90 points; missing value filling processing refers to filling missing data according to a predetermined strategy; normalization processing refers to mapping data to the range of 0 to 1.

[0133] In this embodiment, the recognition probability fusion module 4 is specifically used to fuse the first recognition probability, the second recognition probability and the third recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia. In this embodiment, for example, the recognition probabilities can be fused in a weighted average manner.

[0134] Given that the test results of the literacy assessment and the reading and writing test are very closely related, it is also possible to consider combining the attributes of the two assessment results so that the machine learning modeling process can use the relationship information between the two to jointly predict the degree of dyslexia of the subject. Therefore, the multimodal data in this embodiment can also include the literacy assessment results of the subject.

[0135] In another example, the developmental dyslexia intelligent identification system also includes a dyslexia determination module 9, which is used to determine whether the comprehensive identification probability reaches a preset value. If so, it is determined that the subject suffers from developmental dyslexia, otherwise, it is determined that the subject does not suffer from developmental dyslexia.

[0136] Specifically, in step S208, the preset value is 0.5, that is, when the diagnosis probability of developmental dyslexia of the subject reaches 0.5, it is diagnosed that the subject suffers from developmental dyslexia, and when it is less than 0.5, it is diagnosed that the subject does not suffer from developmental dyslexia.

[0137] The intelligent recognition system for developmental dyslexia in this embodiment processes eye movement signals in segments and inputs them into a bidirectional long-short-term memory neural network, extracts specified features from eye movement signals and inputs them into an integrated decision tree model, and preprocesses multimodal data before inputting them into an integrated decision tree model. The modeling results of the integrated decision tree model and the bidirectional long-short-term memory neural network are highly complementary. The deep learning model directly processes eye movement signals to alleviate the feature deviation problem caused by preset eye movement indicators. The integration of deep and shallow models using model fusion technology can further improve the performance of the model, thereby ensuring the accuracy of identifying the developmental dyslexia of the subject.

[0138] Example 5

[0139] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Example 5 of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the intelligent identification method for developmental dyslexia of Example 1 or Example 2 when executing the program. Figure 6 The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0140] like Figure 6 As shown, the electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0141] The bus 33 includes a data bus, an address bus, and a control bus.

[0142] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0143] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0144] The processor 31 executes various functional applications and data processing by running the computer programs stored in the memory 32, such as the intelligent identification method for developmental dyslexia of embodiment 1 or embodiment 2 of the present invention.

[0145] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model-generated device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via a bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0146] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0147] Example 6

[0148] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method for intelligently identifying developmental dyslexia in embodiment 1 or embodiment 2 are implemented.

[0149] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0150] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes a program code. When the program product runs on a terminal device, the program code is used to enable the terminal device to execute the steps of the intelligent identification method for developmental dyslexia of Example 1 or Example 2.

[0151] The program code for executing the present invention may be written in any combination of one or more programming languages, and may be executed entirely on a user device, partially on a user device, as an independent software package, partially on a user device and partially on a remote device, or entirely on a remote device.

[0152] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A method for intelligently identifying developmental dyslexia, characterized in that: The developmental dyslexia intelligent identification method comprises: Acquire the eye movement signal of the subject; the eye movement signal is the original eye movement coordinate timing signal; Inputting the eye movement signal into a preset first recognition model for processing to obtain a first recognition probability that the subject suffers from developmental dyslexia; Extracting specified feature data from the eye movement signal, inputting the feature data into a preset second recognition model for processing, and obtaining a second recognition probability corresponding to the subject suffering from developmental dyslexia; The first recognition probability and the second recognition probability are combined to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia; Before fusing the first recognition probability and the second recognition probability, the method for intelligently identifying developmental dyslexia further includes: Acquiring multimodal data for characterizing the reading ability of the subject; Inputting the multimodal data into a third recognition model for processing to obtain a third recognition probability that the subject suffers from developmental dyslexia; The step of fusing the first recognition probability and the second recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia includes: The first recognition probability, the second recognition probability and the third recognition probability are combined to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia; Among them, the first recognition model is a bidirectional long short-term memory neural network, and the second recognition model is an integrated decision tree model.

2. The intelligent identification method for developmental dyslexia according to claim 1, characterized in that: Before inputting the eye movement signal into the first recognition model for processing, the method further includes: The eye movement signal is processed uniformly in segments.

3. The intelligent identification method for developmental dyslexia according to claim 1, characterized in that: Before inputting the multimodal data into the third recognition model for processing, the method further includes: The multimodal data is preprocessed, wherein the preprocessing includes at least one of quantization processing, missing value filling processing, and normalization processing.

4. The intelligent identification method for developmental dyslexia according to claim 1, characterized in that: The step of extracting specified feature data from the eye movement signal comprises: At least one of the following feature data is extracted from the eye movement signal: coordinate position information of the eye gazing at the screen, gaze duration and / or pupil size information of the eye.

5. The intelligent identification method for developmental dyslexia according to claim 1, characterized in that: After obtaining the comprehensive recognition probability corresponding to the eye movement signal, the method further includes: It is determined whether the comprehensive recognition probability reaches a preset value. If so, it is determined that the subject suffers from developmental dyslexia. Otherwise, it is determined that the subject does not suffer from developmental dyslexia.

6. An intelligent recognition system for developmental dyslexia, characterized in that: The developmental dyslexia intelligent identification system comprises: An eye movement signal acquisition module is used to acquire the eye movement signal of the subject; the eye movement signal is the original eye movement coordinate timing signal; A first recognition module, used for inputting the eye movement signal into a preset first recognition model for processing, and obtaining a first recognition probability that the subject suffers from developmental dyslexia; A second recognition module is used to extract specified feature data from the eye movement signal, input the feature data into a preset second recognition model for processing, and obtain a second recognition probability corresponding to the subject suffering from developmental dyslexia; A recognition probability fusion module, used for fusing the first recognition probability and the second recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia; The system further comprises: A multimodal data acquisition module, used to acquire multimodal data for characterizing the reading ability of the subject; A third recognition module, used for inputting the multimodal data into a third recognition model for processing to obtain a third recognition probability that the subject suffers from developmental dyslexia; The recognition probability fusion module is specifically used to fuse the first recognition probability, the second recognition probability and the third recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia; Among them, the first recognition model is a bidirectional long short-term memory neural network, and the second recognition model is an integrated decision tree model.

7. The intelligent recognition system for developmental dyslexia according to claim 6, characterized in that: The system further comprises an eye movement signal preprocessing module, which is used for uniformly segmenting the eye movement signal before the first recognition module performs corresponding operations.

8. The intelligent identification system for developmental dyslexia according to claim 6, characterized in that: The system further comprises: The multimodal data preprocessing module is used to preprocess the multimodal data, wherein the preprocessing includes at least one of quantization processing, missing value filling processing and normalization processing.

9. The intelligent identification system for developmental dyslexia according to claim 6, characterized in that: The feature data extracted by the second recognition module from the eye movement signal includes at least one of the following feature data: coordinate position information of the eye's gaze on the screen, gaze duration and / or pupil size information of the eye.

10. The intelligent identification system for developmental dyslexia according to claim 6, characterized in that: The system also includes a dyslexia determination module, which is used to determine whether the comprehensive recognition probability reaches a preset value. If so, it is determined that the subject suffers from developmental dyslexia, otherwise, it is determined that the subject does not suffer from developmental dyslexia.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for intelligently identifying developmental dyslexia as claimed in any one of claims 1 to 5 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for intelligently identifying developmental dyslexia as claimed in any one of claims 1 to 5 is implemented.

Images