Fusion special education-oriented artificial intelligence interaction system and device

Abstract

The invention discloses an artificial intelligence interaction system and device for fusion special education, and belongs to the technical field of fusion education. In the artificial intelligence interaction system, a learning task module obtains learning tasks through a cloud server, and the learning tasks comprise an audio-visual task, a cognitive task, a read-write task and an action training task; after the learning task module analyzes the learning task, the voice interaction module and the data acquisition module are called to execute the learning task, an obtained learning result is transmitted to the learning evaluation module, and the learning evaluation module evaluates the learning result based on a personalized federated learning algorithm to generate a personalized learning condition model of the user; the cloud server is used for adjusting the learning task content in combination with the personalized data and generating the next learning task of the user. According to the invention, personalized education services can be provided for special education objects.

Description

Technical Field

[0001] The present invention belongs to the field of integrated special education, and specifically relates to an artificial intelligence interaction system and device for integrated special education. Background Art

[0002] Inclusion is a special education theory that aims to enable children with different characteristics to learn in a regular school environment through specially designed educational environments and teaching methods. Inclusion emphasizes equality, respect, harmony and progress, and enables special children to better integrate into the mainstream of education, physical environment and social life through group cooperative learning and peer learning.

[0003] The goal of inclusive education is to enable children with special needs to receive education in ordinary schools together with children with normal development, reduce isolation and exclusion, and provide a supportive teaching environment. This helps children with special needs to receive normalized education, promote their physical and mental development, and enhance their social adaptability.

[0004] In the field of special education, in addition to visually impaired children (blind children), hearing-impaired children (deaf-mute children) and children with physical disabilities, there is also a large group of children with cognitive dysfunction. The causes of their illness mainly include traumatic brain injury; brain nerve dysfunction such as cerebral palsy, mental retardation, mental retardation, aphasia and movement disorders; and unfavorable environmental education factors, which bring learning skill development disorders to susceptible children.

[0005] Learning activities require the participation of multiple cognitive processes (such as perception, memory comprehension, generalization, comparison, reasoning, etc.). As long as children have problems with any of these cognitive abilities, their learning ability will be affected. According to conservative estimates, there are at least 10 million children with dyslexia in my country. Children with mild cognitive impairment, or autism, speech and language disorders, ADHD and other diseases may still study in ordinary schools, but face various difficulties or have special needs.

[0006] For the above-mentioned children with special needs in general schools, the comprehensive education and training methods should be individualized, targeted, and planned. Start with the easy and then move on to the difficult, and combine a variety of reward methods to strengthen the positive reinforcement effect. Parents and educators should have great patience and flexibility with children with special needs. For children who also suffer from emotional disorders, they need to be corrected by psychological counselors first.

[0007] In the field of audio-visual education, some auxiliary teaching systems and a large number of electronic resources have been developed and applied. They can include simple text, symbols, pictures, gestures, as well as more complex voice, video and electronic equipment. However, the existing auxiliary teaching systems and electronic resources are usually oriented to normal students and general education scenarios, and even the content is usually prefabricated. They often do not have flexible adaptability, and cannot meet the needs of multi-subject personalized teaching of special needs children in middle grades, especially in high grades, as well as the needs of moral education and connotation-based cultivation.

[0008] The large-scale collection of learning data of children with special needs is an indispensable foundation for the use of big data and artificial intelligence technologies to empower inclusive education. However, children with special needs who attend general schools are often more sensitive in character and psychology than normal students, and may even suffer from emotional disorders. They and their families are not willing to share their learning data easily. Therefore, in the entire process of collecting and analyzing their learning data, more attention should be paid to data security and privacy protection. Federated learning, with its ability to "keep the data in place but the model moves", the ability to conduct comprehensive analysis of the data streams of each edge and terminal node on the central server without the need for the original data to leave the domain and be invisible to the outside world, has undoubtedly become the most suitable artificial intelligence algorithm framework for empowering inclusive education.

[0009] However, there are large differences in ability and quality between different types of children with special needs, and even between each child with special needs. To this end, we should pursue "one policy for each student, teaching in accordance with their aptitude"; artificial intelligence technology should be able to accurately evaluate the effectiveness of education and teaching based on input data streams with different characteristics, and provide personalized guidance and iterative optimization for further training. This requires the design and application of personalized federated learning (PFL) algorithms to provide each child with special needs with a personalized optimal sub-model, and its actual effect will be far better than a single "full network system sum optimal" model.

[0010] In the existing related research work in the world, the most common type of PFL algorithm is that each node first collaboratively trains a global optimal model, and then tunes its own sub-model locally; but if the data streams of each node are very different (in sharp contrast to the common simplified assumption of "independent and identically distributed"), the training accuracy of this type of algorithm will be affected and vulnerable to attack. Therefore, this type of algorithm obviously cannot meet the actual needs of empowering inclusive education. In another major type of PFL algorithm, each node can perform personalized information aggregation locally; this type of algorithm calculates faster and more accurately, but existing algorithms usually require that all nodes participating in the networked system can be divided into a small number of groups according to attributes or similarities of data streams. When the number of children with special needs who provide learning data is large, and the sources of variables such as region, age, causes and degree of learning difficulties are scattered, existing algorithms cannot meet the actual needs of empowering inclusive education.

[0011] In summary, it is necessary to design and apply a new personalized federated learning algorithm, which should be loaded on the artificial intelligence interactive device and system described in this application to effectively enable integrated education. Summary of the invention

[0012] Purpose of the invention: The present invention aims to provide an artificial intelligence interactive system and device for inclusive special education. The system is loaded with a new personalized federated learning algorithm to achieve personalized and flexible learning for special education subjects and promote the technical development and application of inclusive education.

[0013] Technical solution: An artificial intelligence interactive system for integrated special education, including:

[0014] A learning task module, which communicates with a cloud server to obtain teaching content and tasks for cognitive function training, including updating the learning content of the module;

[0015] Voice interaction module, including voice packages corresponding to commonly used Chinese and English vocabulary and corpus, used to understand the user's voice input and play voice output on demand;

[0016] The data collection module is used to collect real-time data throughout the teaching interaction process, including task completion, answer accuracy, error rate, reaction time, usage time, and usage preferences;

[0017] The learning evaluation module executes the personalized federated learning algorithm and evaluates the learner's task completion through the data provided by the data collection module, including processing local data and feeding back the processing results to the cloud server;

[0018] The information management module is used for user information management, device management, resource download management, and includes historical training record viewing and additional task setting. The additional task setting is to manually intervene in the setting of learning tasks based on the content of teaching interaction.

[0019] Furthermore, in the system, the learning task module obtains learning tasks through the cloud server, and the learning tasks include audio-visual tasks, cognitive tasks, reading and writing tasks, and motor training tasks; after the learning task module analyzes the learning tasks, it mobilizes the voice interaction module and the data acquisition module to execute the learning tasks, and transmits the obtained learning results to the learning evaluation module. The learning evaluation module evaluates the learning results based on the personalized federated learning algorithm to generate a personalized learning situation model for the user. The cloud server adjusts the learning task content in combination with personalized data to generate the user's next learning task.

[0020] Furthermore, the personalized federated learning algorithm described in the present invention decomposes the local sub-model of each federated participant according to the classes in its data set, and then for each class, superimposes the components of each local sub-model according to certain weights to obtain a series of global classification models, and then for each federated participant, superimposes each global classification model according to preset weights to finally obtain a personalized sub-model.

[0021] Furthermore, the mathematical expression of the personalized federated learning algorithm is as follows:

[0022] Let the number of federation participants be N, and the raw data they hold be divided into M categories;

[0023] The initialization process includes:

[0024] For any class Randomly initialize the global classification model on the cloud server

[0025] For any federation participant Initialize the personalized sub-model randomly locally

[0026] right Will The value is initialized to In the formula Yes i,j , both of which represent the statistical distribution of the original data held by federation participant i; and is the proportion of the original data held by federation participant i that belongs to category j;

[0027] The cycle process includes:

[0028] For rounds r = 1, 2, ..., R, R is the autonomously determined upper limit;

[0029] The superimposed global classification model is:

[0030] Download local sub-model: Each federation participant downloads from the cloud server

[0031] Optimize the local sub-model: Each federation participant optimizes the original loss function On the basis of Then with the new loss function As the goal, the parameter family of the local sub-model is optimized;

[0032] Upload local sub-model: Each federation participant uploads to the cloud server

[0033] Superimpose local sub-model: The cloud server update is expressed as:

[0034]

[0035] in The connection weight vector between the second-last layer and the output layer of the neural network representing federation participant i; and superimposed in

[0036] Output: and

[0037] Furthermore, the personalized federated learning algorithm includes providing a personalized and iteratively optimized education and teaching model for local users, while being able to desensitize the original training data and securely transmit effective information to enhance privacy protection.

[0038] Furthermore, the data acquisition module includes encapsulating the collected data into a standard data interface Interface, and providing it to the visualization display unit and training result evaluation.

[0039] The learning evaluation module includes preprocessing the received data, extracting the feature data in the learning results, and then training the feature data based on the personalized federated learning algorithm to obtain the evaluation results;

[0040] The module will also upload the desensitized local model to the cloud server regularly according to the preset schedule, and form a personalized intervention training plan after model optimization, providing targeted and guiding improvement suggestions for further education and training.

[0041] The system of the present invention is also provided with a power supply module and a visual display unit. The power supply module is used to supply power to the system, and the visual display unit includes a touch display screen and an indicator light.

[0042] The present invention also provides an artificial intelligence interaction device for integrated special education, which implements the hardware layout of the above-mentioned artificial intelligence interaction system through a shell, including circuit connections between various modules of the hardware.

[0043] Beneficial effects: The present invention is based on the improved personalized federated learning algorithm, intelligent data collection and analysis, and can achieve accurate evaluation of educational and teaching effects. It optimizes the local model through the personalized federated learning algorithm, provides personalized guidance and iterative optimization for further training, and has high flexibility, interest and scientificity to use, which helps quality education and connotation-based development, and is flexible and applicable to special education subjects. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1It is a framework diagram of the personalized federated learning algorithm described in the present invention. DETAILED DESCRIPTION

[0045] To illustrate the technical solution provided by the present invention in detail, further elaboration is given below in conjunction with the accompanying drawings.

[0046] The present invention is an artificial intelligence interactive device and system for special education. During the reading, cognition and learning process, young children cooperate with the software system for interactive training. The system is based on intelligent data collection and analysis to achieve intelligent evaluation of educational and teaching effects, and quantitatively connects national and local curriculum standards to generate personalized training plans for students and push further educational and teaching tasks, so as to greatly improve the fun and effectiveness of teaching activities. In this process, the intelligent teaching management of inclusive education schools is also realized, thereby constructing an overall support service plan for inclusive education. The technical solution of the present invention will be introduced in detail below:

[0047] The artificial intelligence interactive system for special education described in the present invention includes a learning task module, a voice interaction module, a data acquisition module, a learning evaluation module and an information management module.

[0048] Learning task module: The task module contains various teaching contents and tasks for cognitive function training, and can download updated content and tasks from the cloud server.

[0049] Voice interaction module: mainly includes voice packages corresponding to commonly used Chinese and English vocabulary libraries and corpora, which are used to understand the user's voice input and play voice output as needed.

[0050] Data collection module: The data module is used for real-time data collection during the entire use process, including task completion status, answer accuracy, error rate, reaction time, usage time, usage preferences, etc. All the above data are encapsulated into a standard data interface and provided for visualization, training result evaluation, and intelligent algorithm judgment.

[0051] Learning Assessment Module: The assessment module can assess children’s task completion based on the data provided by the data module. Specifically, the module needs to pre-process the data, complete feature extraction through a preset method, and then input the data into the predictive assessment model to obtain the assessment results, and then perform corresponding processing based on the assessment results. In addition, the assessment module will also upload the local model (parameters) after data desensitization to the cloud server regularly according to the preset, and form a personalized intervention training plan after model optimization, so as to provide targeted and guiding improvement suggestions for further education and training.

[0052] Information management module: The management module is used for user information management, equipment management, resource download management, etc., and has the functions of viewing historical training records and additional settings. Administrators (professional medical staff, instructors, parents, etc.) can view the completed training tasks and related data in the historical training record interface, including task completion status, answer accuracy, error rate, reaction time, usage time, usage preferences, etc. In addition, administrators can also preset some additional conditions that are invisible to children when executing the task module through the background, so as to achieve the "silent" education and training effect.

[0053] For personalized federated learning algorithms, when the "federated participants" are children with special needs, the terminal node is our product "Rong AI" family cube, and the central server is the "Rong AI" smart workstation set up in the school (computing power may also be provided by the cloud, but each local device has the ability to store local data, cache and transmit necessary information); when the "federated participants" are schools, the terminal node is the "Rong AI" smart workstation, and the central server is preferably set up in the regional education authority.

[0054] The core idea of ​​the algorithm design is: decompose the local sub-models of each federation participant according to the classes in its data set, and then for each class, superimpose the components of each local sub-model according to a certain weight to obtain a series of global classification models. Then, for each federation participant, superimpose each global classification model according to a certain weight to finally obtain a personalized sub-model.

[0055] Combination Figure 1 The left and right halves of the algorithm framework shown respectively show the upload and download processes of the personalized federated learning algorithm, and the orange background box represents the cloud server.

[0056] The formal expression of the algorithm is as follows:

[0057] Let the number of federation participants be N, and the raw data they hold be divided into M categories;

[0058] The initialization process includes:

[0059] For any class Randomly initialize the global classification model on the cloud server

[0060] For any federation participant Initialize the personalized sub-model randomly locally

[0061] right Will The value is initialized to In the formula Yesi,j , both of which represent the statistical distribution of the original data held by federation participant i; and is the proportion of the original data held by federation participant i that belongs to category j;

[0062] The cycle process includes:

[0063] For rounds r = 1, 2, ..., R, R is the autonomously determined upper limit;

[0064] The superimposed global classification model is:

[0065] Download local sub-model: Each federation participant downloads from the cloud server

[0066] Optimize the local sub-model: Each federation participant optimizes the original loss function On the basis of Then with the new loss function As the goal, the parameter family of the local sub-model is optimized;

[0067] Upload local sub-model: Each federation participant uploads to the cloud server

[0068] Superimpose local sub-model: The cloud server update is expressed as:

[0069]

[0070] in The connection weight vector between the second-last layer and the output layer of the neural network representing federation participant i; and superimposed in

[0071] Output: and

[0072] This algorithm can provide a personalized and iteratively optimized education and teaching model for each local user of the present invention. Different from the personalized federated learning algorithms published in relevant literature, this algorithm has the following characteristics:

[0073] 1) Instead of training a single global model and then performing personalized processing for each federation participant, the system adjusts the superposition of each local sub-model during the federation learning process based on the information of each federation participant, and then trains a personalized sub-model for each federation participant. This can reduce disturbances caused by potential attacks.

[0074] 2) When each local sub-model is superimposed, its weight value does not need to be learned, which can greatly reduce computing power consumption.

[0075] 3) It is not required that all federated participants must be clustered into a few groups. This is particularly important for inclusive education application scenarios.

[0076] Furthermore, based on the personalized federated learning algorithm, the present invention can also achieve desensitization of original training data and secure transmission of effective information with strong privacy protection; the present invention also provides effective support for the moral education function of integrating special education by designing a background management module, which can bring more comprehensive help to children's subject education, social interaction, mental health, emotional value attitudes, and even the ability to participate in social production division after graduation from school.

[0077] According to research, as of now, the personalized federated learning (PFL) algorithm in this application is the first PFL algorithm that has been put into practical application (not limited to specific application fields and scenarios), and does not require all system nodes to be clustered into a few groups, nor does it require training to know the superposition weights of each sub-model; and the results of testing this algorithm on several examples and data sets are higher in accuracy or only have a small (about 1%) difference compared with the best of the existing PFL algorithms.

Claims

1. An artificial intelligence interactive system for integrated special education, characterized in that: include: A learning task module, which communicates with a cloud server to obtain teaching content and tasks for cognitive function training, including updating the learning content of the module; Voice interaction module, including voice packages corresponding to commonly used Chinese and English vocabulary and corpus, used to understand the user's voice input and play voice output on demand; The data collection module is used to collect real-time data throughout the teaching interaction process, including task completion, answer accuracy, error rate, reaction time, usage time, and usage preferences; The learning evaluation module executes the personalized federated learning algorithm and evaluates the learner's task completion through the data provided by the data collection module, including processing local data and feeding back the processing results to the cloud server; The information management module is used for user information management, device management, resource download management, and includes historical training record viewing and additional task setting. The additional task setting refers to the manual intervention in setting learning tasks based on the content of teaching interaction.

2. The artificial intelligence interactive system according to claim 1, characterized in that: In this system, the learning task module obtains learning tasks through the cloud server, and the learning tasks include audio-visual tasks, cognitive tasks, reading and writing tasks, and motor training tasks; after the learning task module analyzes the learning tasks, it mobilizes the voice interaction module and the data acquisition module to execute the learning tasks, and transmits the obtained learning results to the learning evaluation module. The learning evaluation module evaluates the learning results based on the personalized federated learning algorithm to generate a personalized learning situation model for the user. The cloud server adjusts the learning task content in combination with personalized data to generate the user's next learning task.

3. The artificial intelligence interactive system according to claim 1, characterized in that: The personalized federated learning algorithm decomposes the local sub-model of each federated participant according to the classes in its data set, and then for each class, superimposes the components of each local sub-model according to a certain weight to obtain a series of global classification models. Then, for each federated participant, superimposes each global classification model according to a preset weight to finally obtain a personalized sub-model.

4. The artificial intelligence interactive system according to claim 1, characterized in that: The mathematical expression of the personalized federated learning algorithm is as follows: Let the number of federation participants be N, and the raw data they hold be divided into M categories; The initialization process includes: For any class Randomly initialize the global classification model on the cloud server For any federation participant Initialize the personalized sub-model randomly locally right Will The value is initialized to In the formula Yes i,j , both of which represent the statistical distribution of the original data held by federation participant i; and is the proportion of the original data held by federation participant i that belongs to category j; The cycle process includes: For rounds r = 1, 2, ..., R, R is the autonomously determined upper limit; The superimposed global classification model is: Download local sub-model: Each federation participant downloads from the cloud server Optimize the local sub-model: Each federation participant optimizes the original loss function On the basis of Then with the new loss function As the goal, the parameter family of the local sub-model is optimized; Upload local sub-model: Each federation participant uploads to the cloud server Superimpose local sub-model: The cloud server update is expressed as: in The connection weight vector between the second-last layer and the output layer of the neural network representing federation participant i; and superimposed in Output: and 5. The artificial intelligence interactive system according to claim 1, characterized in that: The personalized federated learning algorithm includes providing a personalized and iteratively optimized education and teaching model for local users, while being able to desensitize original training data and securely transmit effective information to enhance privacy protection.

6. The artificial intelligence interactive system according to claim 1, characterized in that: The data acquisition module includes encapsulating the collected data into a standard data interface and providing it to the visualization display unit and training result evaluation.

7. The artificial intelligence interactive system according to claim 1, characterized in that: The learning evaluation module includes preprocessing the received data, extracting the feature data in the learning results, and then training the feature data based on the personalized federated learning algorithm to obtain the evaluation results; The module will also upload the desensitized local model to the cloud server regularly according to the preset schedule, and form a personalized intervention training plan after model optimization, providing targeted and guiding improvement suggestions for further education and training.

8. The artificial intelligence interactive system according to claim 1, characterized in that: The system is also provided with a power module and a visual display unit. The power module is used to supply power to the system. The visual display unit includes a touch display screen and an indicator light.

9. An artificial intelligence interactive device for integrated special education, characterized in that: The device implements the hardware arrangement of the artificial intelligence interaction system according to any one of claims 1 to 8 through a shell, including circuit connections between the hardware modules.

Images