Man-machine interaction system and method for cross-channel time processing of developmental reading disorder

Abstract

The invention discloses a man-machine interaction system and method for cross-channel time processing of developmental reading disorder. The man-machine interaction system comprises an evaluation unit which is used for collecting personal information of a user and evaluation results of audio-visual simultaneous judgment; the parameter configuration unit is connected with the evaluation unit, is provided with a preset element library and a preset rule library, and is used for configuring a practice interaction task and a formal interaction task with preset difficulty for the user based on the evaluation record; the exercise task unit and the formal task unit are both connected with the parameter configuration unit so as to perform man-machine interaction with the user based on the exercise interaction task and the formal interaction task according to the sequence of first connection and then formal; and the feedback unit is connected with the formal task unit so as to obtain man-machine interaction data of finishing the formal interaction task by the user and feed back an interaction result to the user. The method aims at the underlying logic of the time processing defect, stimulates the visual sense and the auditory sense at the same time, and promotes the improvement of the audio-visual integration ability of children.

Description

Technical Field

[0001] The present invention relates to a human-computer interaction system for cross-channel temporal processing of developmental dyslexia, and also relates to a corresponding human-computer interaction method, belonging to the technical field of human-computer interaction. Background Art

[0002] Temporal processing is a cognitive neural mechanism that involves the encoding, decoding, and evaluation of temporal structures, including duration, sequence, speed, or regularity of events. This process is essential for low-level sensory processing, such as perceiving the duration, rapid changes, and sequence of stimuli. Researchers have suggested that speech skill deficits in developmental dyslexia may be related to basic auditory temporal processing deficits. Rise time, as a key event in speech signals, helps segment sound signals and provides cues for speech perception. Brain neural networks use these cues to synchronize and calibrate neural activity to synchronize with speech signals. Therefore, abnormal synchronous processing of rise time by the nervous system may be the neural basis of speech deficits in developmental dyslexia.

[0003] Further research has shown that dyslexics lag behind normal people in processing rapidly presented stimuli, both in the auditory and visual channels. This may indicate that dyslexics have general time control problems. Studies have also found that individuals with poor auditory rapid processing ability also have poor visual rapid processing ability, a finding that supports the view that dyslexics have general time control problems.

[0004] The Chinese patent application with publication number CN115547151A discloses an audio-visual integrated reading training system and method thereof, which aims to improve dyslexia through cross-channel training methods. The system combines an audio reading device and Chinese text reading materials to train users to listen to voice information while reading, and match the heard pronunciation with the seen text one by one. This training method simulates the learning process of natural reading, with the aim of improving the cross-channel integration ability of users with dyslexia. However, although the system takes into account the defects of dyslexics in visual and auditory processing, the intervention materials it selects are relatively traditional, mainly simulating a real reading learning environment, training at the behavioral level, and not using interactive task-based training materials to improve users' cognitive abilities from the bottom up. This means that although the system helps to improve cross-channel integration capabilities, it may have limitations in improving cognitive abilities, and further exploration of more effective training methods is needed. Summary of the invention

[0005] The primary technical problem to be solved by the present invention is to provide a human-computer interaction system for cross-channel temporal processing of developmental dyslexia.

[0006] Another technical problem to be solved by the present invention is to provide a human-computer interaction method for cross-channel temporal processing of developmental dyslexia.

[0007] In order to achieve the above technical objectives, the present invention adopts the following technical solutions:

[0008] According to a first aspect of an embodiment of the present invention, a human-computer interaction system for cross-channel temporal processing of developmental dyslexia is provided, comprising:

[0009] The evaluation unit includes an information input module and a cognitive evaluation module; wherein the information input module is used to collect the user's personal information, and the cognitive evaluation module is used to evaluate the user's audio-visual simultaneity judgment and obtain the evaluation result;

[0010] A parameter configuration unit, connected to the evaluation unit and having a preset element library and a preset rule library, to configure a practice interactive task and a formal interactive task of preset difficulty for the user based on the evaluation result; wherein the preset element library at least includes starlight, stars, speakers, a plurality of continuously presented interactive buttons, and a star mark containing a lighting progress; the preset rule library at least includes difficulty level, visual stimulation, sound stimulation, audiovisual stimulation time interval combination, starlight quantity requirement for collecting stars, and correct and incorrect feedback;

[0011] A practice task unit is connected to the parameter configuration unit to perform human-computer interaction with the user based on the practice interaction task; wherein a task trial of the practice interaction task is configured to determine whether the sound and the starlight appear at the same time according to the played sound and the presented starlight; and the difficulty level of each task trial of the practice interaction task is equal, and correctness feedback is immediately given after the user answers;

[0012] A formal task unit is connected to the practice task unit and the parameter configuration unit to perform human-computer interaction with the user based on the formal interaction task; wherein a task trial of the formal interaction task is configured to determine whether the sound and the starlight appear at the same time according to the played sound and the presented starlight; and the formal task unit also includes a difficulty adjustment module for obtaining behavioral response data of the user performing human-computer interaction and dynamically adjusting the task difficulty based on the behavioral response data;

[0013] A feedback unit is connected to the formal task unit to obtain the human-computer interaction data of the user completing the formal interaction task and to feed back the interaction result to the user.

[0014] Preferably, a task trial of the practice task unit is configured in the following manner:

[0015] Retrieving a pronunciation speaker, starlight, and a plurality of continuously presented interactive buttons from the preset element library;

[0016] Retrieving difficulty levels, visual stimulation, sound stimulation, audio-visual stimulation time interval combinations, and correct and incorrect feedback from the preset rule library;

[0017] Determine the presentation order of sound playback and starlight twinkling based on the audio-visual stimulation time interval combination; wherein the presentation order includes sound stimulation first and then visual stimulation, visual stimulation first and then sound stimulation, or audio-visual stimulation at the same time;

[0018] According to the presentation order, sound is played through the speaker based on the sound stimulation, and visual presentation is performed through twinkling stars based on the visual stimulation, so as to require the user to judge whether the sound and the starlight appear at the same time, and correctness feedback is given immediately after the user answers.

[0019] Preferably, a task trial of the formal task unit is configured in the following manner:

[0020] Retrieving a sound speaker, starlight, a star logo with a lighting progress, and a plurality of continuously presented interactive buttons from the preset element library;

[0021] Retrieving the initial difficulty level, sound stimulation, visual stimulation, audio-visual stimulation time interval combination, and the number of starlights required to collect stars from the preset rule library;

[0022] Determine the presentation order of sound playback and starlight twinkling based on the audio-visual stimulation time interval combination; wherein the presentation order includes sound stimulation first and then visual stimulation, visual stimulation first and then sound stimulation, or audio-visual stimulation at the same time;

[0023] According to the presentation order, based on the sound stimulus, a sound is played through the speaker, and based on the visual stimulus, a visual presentation is performed through twinkling starlight, so as to require the user to judge whether the sound and the starlight appear at the same time;

[0024] Each time the user answers correctly, he or she obtains starlight once and collects the starlight in the star sign containing the lighting progress, and the star sign lights up after collecting a preset number of starlights.

[0025] Preferably, the formal task unit includes a plurality of task trials, and every N task trials form a task group, and the correct rate of the task group is greater than or equal to 75% indicating that the task group is answered correctly, and the correct rate of the task group is less than 75% indicating that the task group is answered incorrectly; wherein N is a positive integer, and the difficulty level of the N task trials in the same task group is the same;

[0026] If the user answers two task groups correctly in succession, the difficulty level of the next task group increases by one level; if the user answers one task group incorrectly, the difficulty level of the next task group decreases by one level.

[0027] Preferably, the interaction result includes at least accuracy, interaction star rating, interaction score and maximum difficulty level; wherein the accuracy rate and the interaction star rating jointly determine the interaction stage corresponding to the user, and each interaction stage has a corresponding purpose explanation and meaning explanation.

[0028] Preferably, if the user is a new user, the parameter configuration unit calculates the initial difficulty level of the formal interactive task according to the evaluation result of the user based on a preset algorithm;

[0029] If the user is a historical user, the parameter configuration unit reduces the final difficulty level of the user's last interaction by two levels and uses it as the initial difficulty level of the formal interaction task.

[0030] Preferably, the human-computer interaction system is an APP installed on an electronic device or an accessible website set on a browser.

[0031] According to a second aspect of an embodiment of the present invention, there is provided a human-computer interaction method using the above human-computer interaction system, comprising the following steps:

[0032] Obtaining the user's personal information and audio-visual simultaneity judgment results;

[0033] Based on the evaluation result of the user, configuring the practice interaction task and the formal interaction task of preset difficulty for the user based on the parameter configuration unit;

[0034] Performing human-computer interaction with the user based on the practice interaction task through a practice task unit until the user completes the practice interaction task;

[0035] After the user completes the practice interaction task, a formal task unit performs human-computer interaction with the user based on the formal interaction task, and dynamically adjusts the task difficulty through a difficulty adjustment module until the human-computer interaction ends;

[0036] The interaction result of this human-computer interaction is fed back to the user through the feedback unit and stored in the user database; wherein the interaction result at least includes: accuracy, interaction star rating, interaction score and maximum difficulty level.

[0037] Preferably, the dynamically adjusting the task difficulty through the difficulty adjustment module specifically includes:

[0038] In the process of the user completing the formal interactive task, the user's behavioral response data for human-computer interaction is obtained through the difficulty adjustment module, and the behavioral response data includes the answer result and reaction time of each task trial;

[0039] According to the behavioral response data, if the user answers two task groups correctly in succession, the difficulty level of the next task group increases by one level; if the user answers one task group incorrectly, the difficulty level of the next task group decreases by one level;

[0040] Wherein, the formal task unit includes multiple task trials, and every N task trials form a task group. If the accuracy rate of the task group is greater than or equal to 75%, it means that the task group has a correct answer, and if the accuracy rate of the task group is less than 75%, it means that the task group has an incorrect answer; N is a positive integer, and the difficulty level of the N task trials in the same task group is the same.

[0041] Preferably, every week or month that the user interacts, a weekly or monthly interaction report is presented to the user based on the result of each interaction of the user, so as to display the user's cumulative interaction time, accuracy, average interaction star rating and current level grade.

[0042] Compared with the prior art, the present invention has the following technical effects:

[0043] (1) Aiming at the underlying logic of the influence of temporal processing defects, unlike traditional single sensory training, the embodiments of the present invention adopt an interactive method of visual and auditory simultaneity judgment, which promotes the improvement of children's visual and auditory integration ability by stimulating the visual and auditory senses at the same time, thereby enhancing language and reading skills.

[0044] (2) Each interactive task is generated in advance using computer software. Based on the correctness of the user input options, the interactive parameters are adjusted according to the parameter setting rules, and the corresponding interactive conditions are changed to provide users with interactive parameters that are closest to their zone of proximal development and exercises that are most suitable and timely meet user needs. This setting is targeted and adaptive, and users can also practice repeatedly according to their own situation, making full use of the principle of neuroplasticity, improving auditory temporal processing defects, and promoting reading-related cognitive development.

[0045] (3) In order to increase children’s participation and the fun of interaction, gamification design is used to transform boring interactive tasks into attractive gaming experiences and stimulate children’s intrinsic motivation.

[0046] (4) Non-verbal interactive tasks designed based on user characteristics are suitable for children of all ages, literacy levels, and reading levels. They avoid the impact of dyslexic children’s textual deficiencies on the interactive effects. They can be applied to children with developmental dyslexia from different language backgrounds, so that the interactive effects can be directly compared across language systems, which helps to determine the causes of dyslexia. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic structural diagram of a human-computer interaction system for cross-channel temporal processing of developmental dyslexia provided by the first embodiment of the present invention;

[0048] Figure 2 An overall flow chart of a human-computer interaction method for cross-channel temporal processing of developmental dyslexia provided by the second embodiment of the present invention;

[0049] Figure 3 A detailed flow chart of a human-computer interaction method for cross-channel temporal processing of developmental dyslexia provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0050] The technical content of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] The technical concept of the embodiment of the present invention is: combining the relationship between auditory processing ability, rapid time processing ability and reading ability, based on the hypothesis of audio-visual integration ability defect and time control problem, interactive exercises are conducted on the audio-visual simultaneity judgment ability of children with developmental dyslexia, aiming to improve the general ability of children with dyslexia from the basic cognitive ability related to phonetic awareness, and then improve their reading level. The embodiment of the present invention constructs a human-computer interaction system for cross-channel time processing of developmental dyslexia, which is used to solve the impact of time processing defects on children with developmental dyslexia and improve their audio-visual integration ability, aiming to enhance the basic physiological perception ability of children with developmental dyslexia, rather than just improving their processing skills for specific texts.

[0052] It is understandable that the human-computer interaction system provided by the embodiment of the present invention broadens the population that can benefit, and does not need to rely on the children's existing reading and literacy skills, thereby effectively reducing the impact of individual literacy differences on the training effect, and is applicable to all children with developmental dyslexia. Compared with conventional intervention methods using text materials, the interactive program provided by the present invention is more concise and intuitive, easy to operate and understand, especially the game-based task design, which allows children to receive intervention while playing, which helps to improve children's compliance and interest in interaction.

[0053] In addition, the human-computer interaction system provided by the embodiment of the present invention can effectively eliminate the interference that different language environments may have on children's training results, allowing us to compare the interaction results of children with dyslexia in the Chinese background with the results in other language systems, thereby helping to more accurately understand the root causes of dyslexia.

[0054] First embodiment

[0055] like Figure 1 As shown, a human-computer interaction system for cross-channel temporal processing of developmental dyslexia provided by the first embodiment of the present invention includes an evaluation unit 1, a parameter configuration unit 2, a practice task unit 3, a formal task unit 4, and a feedback unit 5. Among them, the evaluation unit 1 is used to collect the user's personal information and the evaluation results of audio-visual simultaneity judgment. The parameter configuration unit 2 is connected to the evaluation unit 1 and has a preset element library and a preset rule library to configure practice interaction tasks and formal interaction tasks of preset difficulty for the user based on the evaluation demerits. The practice task unit 3 and the formal task unit 4 are both connected to the parameter configuration unit 2 to perform human-computer interaction with the user based on the practice interaction tasks and the formal interaction tasks in the order of first connection and then formal. The feedback unit 5 is connected to the formal task unit 4 to obtain the human-computer interaction data of the user completing the formal interaction task, and feedback the interaction results to the user.

[0056] Below, each functional unit of the human-computer interaction system is described in detail:

[0057] 1. Evaluation Unit

[0058] In this embodiment, the evaluation unit 1 includes an information entry module 11 and a cognitive evaluation module 12. Specifically, the information entry module 11 is used to collect the user's personal information, such as age, gender, education level, etc., and store it in the user database. The cognitive evaluation module 12 is used to evaluate the user's audio-visual simultaneity judgment (for example, through a built-in scale or a built-in evaluation task) and obtain the evaluation results.

[0059] 2. Parameter Configuration Unit

[0060] In this embodiment, the parameter configuration unit 2 has a preset element library 21 and a preset rule library 22. The preset element library 21 at least includes starlight, stars, speakers, multiple continuously presented interactive buttons, and star signs with lighting progress. The preset rule library at least includes difficulty level, visual stimulation, sound stimulation, audiovisual stimulation time interval combination, starlight quantity requirements for collecting stars, and correct and incorrect feedback.

[0061] The following describes in detail the configuration process of the practice interaction task and the formal interaction task:

[0062] (1) Practice interactive task configuration

[0063] ①Call out the speaker, starlight, and multiple continuously displayed interactive buttons from the preset element library.

[0064] ② Retrieve the difficulty level, visual stimulation, sound stimulation, audio-visual stimulation time interval combination and correct or incorrect feedback from the preset rule library.

[0065] ③ The order of sound playback and starlight flashing is determined based on the combination of audio-visual stimulation time intervals. The order of presentation includes sound stimulation first and then visual stimulation, visual stimulation first and then sound stimulation, or audio-visual stimulation at the same time.

[0066] ④ According to the presentation order, sound is played through the speaker based on the sound stimulation, and starlight is visually presented based on the visual stimulation, so as to require the user to judge whether the sound and starlight appear at the same time.

[0067] In this embodiment, a practice interaction task includes multiple task trials (3 in this embodiment, and can also be set to 4, 5 or more), and each task trial is configured through the above steps ①-④ to be configured into the final practice interaction task.

[0068] It is understandable that the purpose of the practice interactive task is to help users understand the rules. Therefore, the difficulty level of each task trial is level 1, and the user is given immediate correct or incorrect feedback after each answer so that the user can quickly understand the task requirements.

[0069] (2) Configuration of formal interaction tasks

[0070] ① Select the speaker, starlight, star icon with lighting progress and multiple continuously presented interactive buttons from the preset element library.

[0071] ② Retrieve the initial difficulty level, sound stimulation, visual stimulation, audio-visual stimulation time interval combination, and the number of starlights required to collect stars from the preset rule library.

[0072] ③ The order of sound playback and starlight flashing is determined based on the combination of audio-visual stimulation time intervals. The order of presentation includes sound stimulation first and then visual stimulation, visual stimulation first and then sound stimulation, or audio-visual stimulation at the same time.

[0073] ④ According to the presentation order, the sound is played through the speaker based on the sound stimulation, and the visual presentation is performed through the flashing of stars based on the visual stimulation, so as to require the user to judge whether the sound and the starlight appear at the same time. Among them, the user obtains a starlight every time he answers correctly and collects it in the star mark containing the lighting progress, and the star mark lights up when the preset number of starlights are collected.

[0074] In this embodiment, a formal interaction task includes multiple task trials and a preset interaction duration. Within the preset interaction duration, the user can continuously complete different task trials (the specific number varies according to the user's ability). Each task trial is configured through the above steps ①-④ to form the final formal interaction task.

[0075] It is understandable that the purpose of this formal interactive task is to improve the user's ability to judge the audio-visual simultaneity. Therefore, the process of immediate right and wrong feedback is removed in the formal interactive task, and only the user's behavioral response data is recorded to avoid affecting the user's current answer. In addition, the recorded behavioral response data is used to provide data support for subsequent task push and difficulty adjustment.

[0076] In addition, in this embodiment, if the user is a new user, the parameter configuration unit 2 calculates the initial difficulty level of the formal interaction task based on the user's evaluation results based on a preset algorithm. If the user is a historical user, the parameter configuration unit 2 obtains the final difficulty level of the user's last interaction based on the user database, and reduces the final difficulty level by two levels as the initial difficulty level of the formal interaction task.

[0077] 3. Practice Task Unit

[0078] The practice task unit 3 is connected to the parameter configuration unit 2. When the parameter configuration unit 2 completes the configuration of the practice interaction task and the formal interaction task, the user first enters the practice task unit 3 to perform human-computer interaction based on the practice interaction task. Among them, a task trial of the practice interaction task is configured to determine whether the sound and the starlight appear at the same time based on the played sound and the presented starlight; and the difficulty level of each task trial of the practice interaction task is equal.

[0079] In the rule description stage, text and pictures are included to explain the training process and rules in detail, making it easier for users to remember the rules. Finally, a start button appears just below the rule description box in the middle of the screen, which is required for users to click to confirm and start the interaction.

[0080] After completing the configuration of the practice interaction task and the formal interaction task, the user clicks OK to start the interaction process.

[0081] The following are the specific steps for practicing interactive tasks:

[0082] In the first step, the user clicks OK to start the exercise.

[0083] Select a background image from the background gallery.

[0084] Retrieve from the preset rule library: (1) Difficulty level (2) Starlight (visual stimulation) (3) Sound (auditory stimulation) (4) Combination of time intervals between sound and starlight (5) Required number of stars to be collected (6) True or false feedback.

[0085] Select from the preset element library: (1) Starlight (2) Stars (3) Simultaneous button (4) Non-simultaneous button (5) Pause button (6) Star icon with lighting progress (7) Activity button.

[0086] The audio-visual combination that needs to be judged is presented according to the difficulty level 1. In addition, some interactive buttons are presented in a long-term continuous mode, including: (1) Activity button, which is presented at the top of the screen. The button is a specific activity icon extracted from the element library, including: difficulty level, remaining time, score (2) Pause button, which is presented at the upper left corner of the screen. The button has the word "Help" written on it.

[0087] For example: the user clicks OK to start the interaction. After the practice interaction task begins, the instructions are displayed first: Please judge whether the sound and the starlight appear at the same time? The audio-visual stimulation time interval combination is called up, the sound is presented to the user through the speaker, and the visual stimulation is a flashing star pattern. The rules require the user to judge whether the sound and the starlight appear at the same time, and press the corresponding button to make a judgment.

[0088] In the second step, the user makes a judgment and clicks the corresponding button to complete the judgment. After the user makes a choice, the following feedback will be given according to whether the user's choice is correct or not:

[0089] If the user fails to make a selection within the time limit, the screen will prompt that the time has expired and the user will proceed to the next question. If the user makes a correct selection, the screen will prompt that the answer is correct and the starlight will be collected into the star with a lighting progress in the upper right corner. The lighting progress will be accelerated and a certain number of starlights can be collected to light up the star and the user will proceed to the next question. If the user makes an incorrect selection, the screen will prompt that the answer is incorrect and the lighting progress will remain unchanged and the user will proceed to the next question.

[0090] The third step is that the practice duration is t. The user can end the practice early or complete the entire practice.

[0091] When the user completes the practice interaction task, the system will guide the user to enter the formal interaction unit 4 to perform human-computer interaction based on the formal interaction task.

[0092] 4. Formal Task Unit

[0093] In this embodiment, the formal task unit 4 is connected to the practice task unit 3 and the parameter configuration unit 2, so as to perform human-computer interaction with the user based on the formal interaction task after the user completes the practice interaction task. Among them, a task trial of the formal interaction task is configured to determine whether the sound and the starlight appear at the same time based on the sound played and the starlight presented. In addition, the formal task unit also includes a difficulty adjustment module 41, which is used to obtain the behavioral response data of the user performing human-computer interaction, and dynamically adjust the task difficulty based on the behavioral response data.

[0094] In the rule description stage, text and pictures are included to explain the interaction process and rules in detail, making it easier for users to remember the rules. Finally, a start interaction button appears just below the rule description box in the middle of the screen, which users need to click to confirm and start the interaction.

[0095] The following are the specific steps of the formal interaction task:

[0096] In the first step, the user clicks OK to start the formal interactive task.

[0097] Select a background image from the background gallery.

[0098] Retrieve from the preset rule library: (1) Difficulty level (2) Starlight (visual stimulation) (3) Sound (auditory stimulation) (4) Combination of time intervals between sound and starlight (5) Required number of stars to be collected (6) True or false feedback.

[0099] Select from the preset element library: (1) Starlight (2) Stars (3) Simultaneous button (4) Non-simultaneous button (5) Pause button (6) Star icon with lighting progress (7) Activity button.

[0100] The audio-visual stimulation time interval combination that needs to be judged is presented according to the difficulty level question library, and four questions are presented for each difficulty level. In addition, some interactive design buttons are presented in a long-term continuous mode, including: (1) Activity button, which is presented at the top of the screen. The button is a specific activity icon extracted from the element library, including: remaining time, score (2) Pause button, which is presented at the bottom left of the screen, and the word "Help" is written on the button.

[0101] For example: the user clicks OK to start the interaction. After the interactive task starts, the instructions are displayed first: Please judge whether the sound and the starlight appear at the same time? The audio-visual stimulation time interval combination is called up, the sound is presented to the user through the speaker, and the visual stimulation is a flashing star pattern. The rules require the user to judge whether the sound and the starlight appear at the same time, and press the corresponding button to make a judgment.

[0102] In the second step, the user performs the judgment task and clicks the corresponding button to complete the judgment. After the user makes a choice, the following feedback is given according to whether the user's choice is correct or not:

[0103] If the user fails to make a selection within the time limit, the screen will prompt that the time has expired and the user will proceed to the next question. If the user makes a correct selection, the screen will prompt that the answer is correct, and the starlight will be collected into the stars with a lighting progress in the upper right corner. The lighting progress will be accelerated and the user will proceed to the next question. If the user makes an incorrect selection, the screen will prompt that the answer is incorrect, and the lighting progress will remain unchanged, and the user will proceed to the next question.

[0104] The third step is to record the correctness of the user's choice in the user database, and adjust the difficulty level of the next round according to the user's accuracy rate. Each group of difficulty has 4 questions. If the accuracy rate is greater than or equal to 75%, it is considered a correct answer; if it is less than 75%, it is considered a wrong answer. If two groups of questions are answered correctly in succession, the difficulty level will increase by one level; if one group of questions is answered incorrectly, the difficulty level will decrease by one level.

[0105] In addition, each time the user answers correctly, he or she can collect a point of starlight. At different levels of difficulty, a certain amount of starlight can be collected to light up a star. A maximum of a stars can be lit up per training session. After the training time countdown ends, this training session ends.

[0106] The fourth step is to adjust and present a new round of formal interaction tasks based on the user's accuracy rate. In addition, the user's final task score is used as the evaluation result to update the task difficulty of the user's next interaction accordingly.

[0107] 5. Feedback Unit

[0108] The feedback unit 5 is connected to the formal task unit 4 to obtain the human-computer interaction data of the user completing the formal interaction task and to feed back the interaction result to the user.

[0109] Specifically, in this embodiment, the feedback unit 5 obtains the human-computer interaction result of this time according to the human-computer interaction data of the user, and stores it in the user database. The human-computer interaction result includes at least: accuracy, interaction star rating, interaction score and maximum difficulty level; the accuracy rate and interaction star rating jointly determine the user's corresponding interaction stage, and each interaction stage has a corresponding purpose explanation and meaning explanation. In this way, not only can the user be presented with the interaction situation of this time, but also data support can be provided for the next human-computer interaction based on the human-computer interaction data of this time.

[0110] In the above embodiment, preferably, the human-computer interaction system is an APP installed on an electronic device or an accessible website set on a browser. Therefore, it can be adapted to conventional electronic devices on the market, and the availability of interaction is achieved, which not only facilitates parents and doctors to conduct remote real-time monitoring, but also allows children to conduct interactive exercises anytime and anywhere without being restricted by time and place.

[0111] Second embodiment

[0112] like Figure 2 and Figure 3 As shown, based on the above first embodiment, the second embodiment of the present invention further provides a human-computer interaction method for cross-channel temporal processing of developmental dyslexia, which specifically includes steps S1 to S5:

[0113] S1: Obtain the user's personal information and the evaluation results of audio-visual simultaneity judgment.

[0114] Specifically, when a user logs into the human-computer interaction system, he or she needs to first enter personal information, and then conduct an audio-visual simultaneity judgment assessment to form a user profile for the user.

[0115] S2: Based on the evaluation result of the user, the parameter configuration unit 2 configures the practice interaction task and the formal interaction task of preset difficulty for the user.

[0116] Specifically, in this step, the parameter configuration unit 2 will retrieve the data in the user database, and then according to the data in the user database, the system will calculate and allocate and generate an audio-visual simultaneity judgment task suitable for the user level. The specific task configuration process refers to the description of the parameter configuration unit 2 in the first embodiment, which will not be repeated here.

[0117] Among them, during the calculation process, the system will determine whether the user is an initial user. If it is an initial user, the user's age, education level, current audiovisual simultaneity judgment assessment score and task-based assessment score are retrieved to calculate the audiovisual simultaneity judgment ability, and then the initial task level is determined based on the user's audiovisual simultaneity judgment ability. If it is a historical user, the above steps are omitted, and the task difficulty level (the final difficulty level is reduced by two levels) and interaction time corresponding to the user's audiovisual simultaneity judgment ability obtained at the end of the last interaction are inherited as the initial task difficulty level and interaction time of this user's interaction.

[0118] S3: Performing human-computer interaction with the user based on the practice interaction task through the practice task unit until the user completes the practice interaction task.

[0119] After completing the configuration of the practice interaction task and the formal interaction task based on step S2, the user will enter the practice task unit 3 to perform human-computer interaction based on the practice interaction task. The specific human-computer interaction steps refer to the description of the practice task unit 3 in the first embodiment above, which will not be repeated here.

[0120] S4: Conduct human-computer interaction with users based on formal interaction tasks through formal task units.

[0121] In this embodiment, after the user completes the practice interaction task in step S3, the user will enter the formal task unit 4 to perform human-computer interaction based on the formal interaction task until the preset interaction time is reached. The specific human-computer interaction steps refer to the description of the formal task unit 4 in the first embodiment above, which will not be repeated here.

[0122] It should be noted that during the human-computer interaction process, the difficulty adjustment module 41 will dynamically adjust the task difficulty based on preset rules (ie, two increases and one decrease).

[0123] S5: Feedback the interaction result of this human-computer interaction to the user through the feedback unit and store it in the user database.

[0124] Specifically, when the user completes the formal interaction task in the formal task unit 4, the feedback unit 6 will feedback the interaction result of this human-computer interaction to the user, and the interaction result at least includes: accuracy, interaction star rating, interaction score and maximum difficulty level.

[0125] In addition, the interaction result will be stored in the user database for subsequent data retrieval. When the user starts the next human-computer interaction, the final difficulty of the previous human-computer interaction is directly retrieved from the user database and reduced by two levels as the initial difficulty of the next human-computer interaction.

[0126] In summary, the human-computer interaction system and method for cross-channel temporal processing of developmental dyslexia provided by the embodiments of the present invention have the following beneficial effects:

[0127] (1) Aiming at the underlying logic of the influence of temporal processing defects, unlike traditional single sensory training, the embodiments of the present invention adopt an interactive method of visual and auditory simultaneity judgment, which promotes the improvement of children's visual and auditory integration ability by stimulating the visual and auditory senses at the same time, thereby enhancing language and reading skills.

[0128] (2) Each interactive task is generated in advance using computer software. Based on the correctness of the user input options, the interactive parameters are adjusted according to the parameter setting rules, and the corresponding interactive conditions are changed to provide users with interactive parameters that are closest to their zone of proximal development and exercises that are most suitable and timely meet user needs. This setting is targeted and adaptive, and users can also practice repeatedly according to their own situation, making full use of the principle of neuroplasticity, improving auditory temporal processing defects, and promoting reading-related cognitive development.

[0129] (3) In order to increase children’s participation and the fun of interaction, gamification design is used to transform boring interactive tasks into attractive gaming experiences and stimulate children’s intrinsic motivation.

[0130] (4) Non-verbal interactive tasks designed based on user characteristics are suitable for children of all ages, literacy levels, and reading levels. They avoid the impact of dyslexic children’s textual deficiencies on the interactive effects. They can be applied to children with developmental dyslexia from different language backgrounds, so that the interactive effects can be directly compared across language systems, which helps to determine the causes of dyslexia.

[0131] It should be noted that the above embodiments are only examples, and the technical solutions of the various embodiments can be combined, all within the protection scope of the present invention.

[0132] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0133] The above is a detailed description of the human-computer interaction system and method for cross-channel temporal processing of developmental dyslexia provided by the present invention. For those skilled in the art, any obvious changes made to it without departing from the essence of the present invention will constitute an infringement of the patent right of the present invention and will bear corresponding legal responsibilities.

Claims

1. A human-computer interaction system for cross-modal temporal processing of developmental dyslexia, characterized by include: The evaluation unit includes an information input module and a cognitive evaluation module; wherein the information input module is used to collect the user's personal information, and the cognitive evaluation module is used to evaluate the user's audio-visual simultaneity judgment and obtain the evaluation result; A parameter configuration unit, connected to the evaluation unit and having a preset element library and a preset rule library, to configure a practice interactive task and a formal interactive task of preset difficulty for the user based on the evaluation result; wherein the preset element library at least includes starlight, stars, speakers, a plurality of continuously presented interactive buttons, and a star mark containing a lighting progress; the preset rule library at least includes difficulty level, visual stimulation, sound stimulation, audiovisual stimulation time interval combination, starlight quantity requirement for collecting stars, and correct and incorrect feedback; A practice task unit is connected to the parameter configuration unit to perform human-computer interaction with the user based on the practice interaction task; wherein a task trial of the practice interaction task is configured to determine whether the sound and the starlight appear at the same time according to the played sound and the presented starlight; and the difficulty level of each task trial of the practice interaction task is equal, and correctness feedback is immediately given after the user answers; A formal task unit is connected to the practice task unit and the parameter configuration unit to perform human-computer interaction with the user based on the formal interaction task; wherein a task trial of the formal interaction task is configured to determine whether the sound and the starlight appear at the same time according to the played sound and the presented starlight; and the formal task unit also includes a difficulty adjustment module for obtaining behavioral response data of the user performing human-computer interaction and dynamically adjusting the task difficulty based on the behavioral response data; A feedback unit is connected to the formal task unit to obtain the human-computer interaction data of the user completing the formal interaction task and to feed back the interaction result to the user.

2. The human-computer interaction system according to claim 1, characterized in that A task trial of the practice task unit is configured in the following way: Retrieving a pronunciation speaker, starlight, and a plurality of continuously presented interactive buttons from the preset element library; Retrieving difficulty levels, visual stimulation, sound stimulation, audio-visual stimulation time interval combinations, and correct and incorrect feedback from the preset rule library; Determine the presentation order of sound playback and starlight twinkling based on the audio-visual stimulation time interval combination; wherein the presentation order includes sound stimulation first and then visual stimulation, visual stimulation first and then sound stimulation, or audio-visual stimulation at the same time; According to the presentation order, sound is played through the speaker based on the sound stimulation, and visual presentation is performed through twinkling stars based on the visual stimulation, so as to require the user to judge whether the sound and the starlight appear at the same time, and correctness feedback is given immediately after the user answers.

3. The human-computer interaction system according to claim 1, characterized in that A task trial of the formal task unit is configured in the following way: Retrieving a pronunciation speaker, starlight, a star logo with a lighting progress, and a plurality of continuously presented interactive buttons from the preset element library; Retrieving the initial difficulty level, sound stimulation, visual stimulation, audio-visual stimulation time interval combination, and the number of starlights required to collect stars from the preset rule library; Determine the presentation order of sound playback and starlight twinkling based on the audio-visual stimulation time interval combination; wherein the presentation order includes sound stimulation first and then visual stimulation, visual stimulation first and then sound stimulation, or audio-visual stimulation at the same time; According to the presentation order, based on the sound stimulus, a sound is played through the speaker, and based on the visual stimulus, a visual presentation is performed through twinkling starlight, so as to require the user to judge whether the sound and the starlight appear at the same time; Each time the user answers correctly, he or she obtains starlight once and collects the starlight in the star sign containing the lighting progress, and the star sign lights up after collecting a preset number of starlights.

4. The human-computer interaction system according to claim 3, characterized in that: The formal task unit includes a plurality of task trials, and every N task trials form a task group, and a correct rate of the task group greater than or equal to 75% indicates that the task group is answered correctly, and a correct rate of the task group less than 75% indicates that the task group is answered incorrectly; wherein N is a positive integer, and the difficulty level of the N task trials in the same task group is the same; If the user answers two task groups correctly in succession, the difficulty level of the next task group increases by one level; if the user answers one task group incorrectly, the difficulty level of the next task group decreases by one level.

5. The human-computer interaction system according to claim 3, characterized in that: The interaction result includes at least the accuracy rate, interaction star rating, interaction score and the highest difficulty level; wherein the accuracy rate and the interaction star rating jointly determine the interaction stage corresponding to the user, and each interaction stage has a corresponding purpose explanation and meaning explanation.

6. The human-computer interaction system according to claim 1, characterized in that: If the user is a new user, the parameter configuration unit calculates the initial difficulty level of the formal interactive task according to the evaluation result of the user based on a preset algorithm; If the user is a historical user, the parameter configuration unit reduces the final difficulty level of the user's last interaction by two levels and uses it as the initial difficulty level of the formal interaction task.

7. The human-computer interaction system according to claim 1, characterized in that: The human-computer interaction system is an APP installed on an electronic device or an accessible website set on a browser.

8. A human-computer interaction method for cross-channel temporal processing of developmental dyslexia, implemented using the human-computer interaction system described in any one of claims 1 to 7, characterized in that The steps include: Obtaining the user's personal information and audio-visual simultaneity judgment results; Based on the evaluation result of the user, configuring the practice interaction task and the formal interaction task of preset difficulty for the user based on the parameter configuration unit; Performing human-computer interaction with the user based on the practice interaction task through a practice task unit until the user completes the practice interaction task; After the user completes the practice interaction task, a formal task unit performs human-computer interaction with the user based on the formal interaction task, and dynamically adjusts the task difficulty through a difficulty adjustment module until the human-computer interaction ends; The interaction result of this human-computer interaction is fed back to the user through the feedback unit and stored in the user database; wherein the interaction result at least includes: accuracy, interaction star rating, interaction score and maximum difficulty level.

9. The human-computer interaction method according to claim 8, characterized in that The dynamically adjusting the task difficulty through the difficulty adjustment module specifically includes: In the process of the user completing the formal interactive task, the user's behavioral response data for human-computer interaction is obtained through the difficulty adjustment module, and the behavioral response data includes the answer result and reaction time of each task trial; According to the behavioral response data, if the user answers two task groups correctly in succession, the difficulty level of the next task group increases by one level; if the user answers one task group incorrectly, the difficulty level of the next task group decreases by one level; Wherein, the formal task unit includes multiple task trials, and every N task trials form a task group. If the accuracy rate of the task group is greater than or equal to 75%, it means that the task group has a correct answer, and if the accuracy rate of the task group is less than 75%, it means that the task group has an incorrect answer; N is a positive integer, and the difficulty level of the N task trials in the same task group is the same.

10. The human-computer interaction method according to claim 8, characterized in that: Every week or month that the user interacts, a weekly or monthly interaction report is presented to the user based on the results of each interaction of the user, so as to display the user's cumulative interaction time, accuracy, average interaction star rating and current level grade.

Images