A special education auxiliary method, device, electronic equipment, storage medium and product

Abstract

The application provides a special education auxiliary method, device, electronic equipment, storage medium and product, and relates to the technical field of auxiliary education, and comprises the following steps: determining a plurality of candidate modal combination according to the interactive feedback signal of a special student; determining the modal bandwidth and resource consumption of the plurality of candidate modal combination, and determining the student cognitive adaptation degree according to the real-time student portrait of the special student; determining the combination score of each candidate modal combination based on the modal bandwidth, the resource consumption and the student cognitive adaptation degree; selecting the best modal combination from the plurality of candidate modal combination based on the combination score of each candidate modal combination, and presenting adaptive education content to the special student based on the best modal combination. The application can provide more accurate adaptive interactive support for special students.

Description

Technical Field

[0001] This invention relates to the field of assistive education technology, and in particular to a special education assistive method, device, electronic device, storage medium and product. Background Technology

[0002] For AI (Artificial Intelligence) assistive technologies in the field of special education, existing solutions mainly fall into the following categories: 1. Dedicated systems for specific disabilities, such as assisting visually impaired students through voice interaction, relying on speech recognition and synthesis to compensate for visual defects; 2. General-purpose multimodal dialogue systems, integrating multimodal inputs such as voice, images, and gestures, but their multimodal processing is mainly used to achieve natural interaction and basic tasks, and functions such as facial recognition are often focused on security and have not been optimized for cognitive compensation in special education; 3. Teacher-side AI monitoring systems, which analyze students' attention through visual algorithms and issue warnings to teachers, are teaching management tools rather than direct student-facing interactive assistance.

[0003] However, these existing solutions cannot achieve real-time personal profiling of special needs students or dynamic analysis and intelligent scheduling of resource bandwidth for different response methods. Therefore, they are unable to provide special needs students with accurate, smooth and resource-efficient adaptive interaction support in complex and ever-changing real-world scenarios. Summary of the Invention

[0004] This invention provides a special education assistance method, device, electronic device, storage medium, and product to solve the technical problem that traditional solutions in the prior art are unable to provide accurate, smooth, and resource-efficient adaptive interactive support for special needs students in complex and ever-changing real-world scenarios.

[0005] This invention provides a special education assistance method, comprising the following steps: Multiple candidate modality combinations were determined based on the interactive feedback signals of special students; Determine the modal bandwidth and resource consumption of the multiple candidate modal combinations, and determine the student cognitive fit based on the real-time student profile of the special student; Based on the modal bandwidth, resource consumption, and student cognitive fit, a combination score is determined for each candidate modal combination. Based on the combination score of each candidate modality combination, the best modality combination is selected from the multiple candidate modality combinations, and adaptive educational content is presented to the special student based on the best modality combination.

[0006] According to a special education assistance method provided by the present invention, the step of determining multiple candidate modality combinations based on the interactive feedback signals of special students includes: After receiving the interaction request from the special student, a question is sent to the special student, and the interaction feedback signal from the special student in response to the question is received. The interactive feedback signal is processed to obtain the intent recognition result, and multiple candidate modality combinations are determined based on the intent recognition result.

[0007] According to a special educational assistance method provided by the present invention, the step of processing the interactive feedback signal to obtain an intent recognition result, and determining multiple candidate modality combinations based on the intent recognition result, includes: The interactive feedback signal is processed by a preset processing model to obtain the confidence score of the candidate intent, and the model weight value is determined according to the real-time student profile of the special student. Based on the confidence score and the model weight value, the intent score of the candidate intent is determined, the intent recognition result is determined according to the intent score, and multiple candidate modality combinations are determined according to the intent recognition result.

[0008] According to a special educational assistance method provided by the present invention, the candidate intent includes a definite intent and an indefinite intent; wherein, determining a combination of multiple candidate modalities based on the intent recognition result includes: Determine whether the intent score of the determined intent is greater than or equal to the intent score of the uncertain intent, and determine whether the intent score of the determined intent is greater than or equal to an intent score threshold; When the intent score of the determined intent is greater than or equal to the intent score of the uncertain intent and the intent score of the determined intent is greater than or equal to the intent score threshold, the intent recognition result is determined as a determined intent, and multiple candidate modal combinations of the interaction feedback signal are determined.

[0009] According to a special educational assistance method provided by the present invention, the candidate intentions include uncertain intentions; wherein, after determining whether the intention score of the certain intention is greater than or equal to the intention score of the uncertain intention, the method further includes: When the intent score of the determined intent is less than the intent score of the uncertain intent or the intent score of the determined intent is less than the intent score threshold, the intent recognition result is determined to be an uncertain intent. Based on the clarification strategy and the special student's historical interaction information, clarification question information is generated, and the special student is guided to provide feedback again based on the clarification question information.

[0010] According to a special education assistance method provided by the present invention, after the step of presenting adaptive educational content to the special student based on the optimal modality combination, the method further includes: Record multidimensional feedback data during this interaction process, including task completion status, student participation, student real-time cognitive load, strategy fit, and modality optimization. The multidimensional feedback data is processed by a preset interaction performance evaluation model to obtain the total interaction performance score. Based on the total interaction performance score, the real-time student profile and the preset interaction performance evaluation model are dynamically optimized. Based on the dynamically optimized real-time student profile and the preset interaction performance evaluation model, a new modal combination is generated for the real-time state of the special student.

[0011] According to a special educational assistance method provided by the present invention, the step of dynamically optimizing the real-time student profile and the preset interaction effectiveness evaluation model based on the total interaction effectiveness score includes: When the total interaction performance score is less than the preset performance threshold, a negative adjustment suggestion is generated for the special student. When the total interaction performance score is greater than or equal to the preset performance threshold, a positive adjustment suggestion is generated for the special student. Based on the total interaction performance score, the personalized weights of the preset interaction performance evaluation model are optimized.

[0012] The present invention also provides a special education assistive device, comprising the following modules: The determination module is used to determine multiple candidate modality combinations based on the interactive feedback signals of special students. The determining module is also used to determine the modal bandwidth and resource consumption of the multiple candidate modal combinations, and to determine the student cognitive fit based on the real-time student profile of the special student. The determining module is further configured to determine the combination score of each candidate modality combination based on the modal bandwidth, the resource consumption, and the student cognitive fit. The presentation module is used to select the best modality combination from the multiple candidate modality combinations based on the combination score of each candidate modality combination, and to present adaptive educational content to the special student based on the best modality combination.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the special education assistance method as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the special education assistance method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the special education assistance method as described above.

[0016] This invention provides a special education assistance method, device, electronic device, storage medium, and product. The method involves: determining multiple candidate modal combinations based on interactive feedback signals from special needs students; determining the modal bandwidth and resource consumption of the multiple candidate modal combinations; determining the student's cognitive fit based on the student's real-time student profile; determining a combination score for each candidate modal combination based on the modal bandwidth, resource consumption, and student cognitive fit; selecting the optimal modal combination from the multiple candidate modal combinations based on the combination score of each candidate modal combination; and presenting appropriate educational content to the special needs student based on the optimal modal combination. This invention addresses the technical challenge of providing precise, smooth, and resource-efficient adaptive interactive support for students with special needs in complex and ever-changing real-world scenarios using traditional solutions. Compared to existing technologies, this invention determines the cognitive fit of students with special needs by constructing real-time updated student profiles. It then determines the modal combination that best matches the user's real-time state and task requirements by quantitatively evaluating modal bandwidth, resource consumption, and student cognitive fit. Finally, it generates and presents adapted content based on this optimal modal combination. This enables precise, smooth, natural, resource-efficient, and continuously evolving adaptive interactive support for students with special needs in real and dynamic teaching scenarios, significantly improving the usability, inclusivity, and effectiveness of human-computer interaction in special education settings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts of the special education assistance method provided by the present invention.

[0019] Figure 2 This is the second flowchart of the special education assistance method provided by the present invention.

[0020] Figure 3 This is a schematic diagram of the structure of the special education auxiliary device provided by the present invention.

[0021] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0023] The following is combined Figure 1 and Figure 2 The present invention describes a special education assistance method applicable to any special education assistance. The subject executing the method can be an electronic device or a special education assistance device installed in the electronic device. The special education assistance device can be implemented by software, hardware, or a combination of both.

[0024] Figure 1 This is one of the flowcharts of the special education assistance method provided by the present invention, such as... Figure 1 As shown, the method includes the following: Step 101: Determine multiple candidate modality combinations based on the interactive feedback signals of special students.

[0025] It should be noted that interactive feedback signals refer to the raw information generated by special needs students through multiple channels such as voice, gestures, and touch during their interaction with electronic devices (such as electronic student ID cards), which can reflect their intentions, cognitive state, and interaction preferences; candidate modality combinations refer to multiple candidate modality combinations generated from the available output modalities based on real-time analysis of interactive feedback signals, such as Scheme A (plain text), Scheme B (simplified text + color animation), Scheme C (simplified text + black and white line drawing), and Scheme D (Braille + voice), etc.

[0026] Step 102: Determine the modal bandwidth and resource consumption of the multiple candidate modal combinations, and determine the student cognitive fit based on the real-time student profile of the special student.

[0027] It should be noted that modal bandwidth refers to the theoretical efficiency of a specific modal combination in transmitting a specific type of information; resource consumption includes the computational and energy costs required to realize a specific modal combination (i.e., resource consumption cost) and its dynamic sensitivity to external conditions (such as device power consumption) (i.e., resource sensitivity). Resource sensitivity refers to a dynamic weight that reflects the current sensitivity of the device to resource consumption, and resource consumption cost refers to the computational and energy costs required to present a specific modal combination; student cognitive fit is jointly determined by content complexity, student cognitive capacity, and individual preference weight. Content complexity refers to the complexity assessment of the core information content to be output, student cognitive capacity refers to the information processing ability of the student in the current state extracted from the real-time student profile, and individual preference weight can be extracted from the real-time student profile to reflect the tendency of special students to use specific modal combinations.

[0028] Step 103: Based on the modal bandwidth, resource consumption, and student cognitive fit, determine the combination score for each candidate modal combination.

[0029] In the specific implementation, the formula for calculating the combination score of each candidate mode combination is as follows: In the formula, M score B represents the combination score of candidate mode combinations; mod Indicates modal bandwidth; C c Indicates content complexity; w pref S represents the individual preference weights; cap R represents the student's cognitive capacity. cost γ represents the cost of resource consumption; γres represents resource sensitivity.

[0030] In the actual implementation, suppose we are explaining the "water cycle" to a special student who is "visually inclined and has cognitive impairment". The parameters in the student profile in real time are: individual preference weight (i.e., visual preference) = 0.8, student cognitive capacity = 0.5, and content complexity = 0.7. However, at this time, the terminal device's battery is below 20%, so the resource sensitivity is increased from the usual 0.1 to 0.8.

[0031] In the specific implementation, the calculation process of the combined score is as follows:

[0032] Understandably, given limited resources, the company intelligently abandoned the most efficient but resource-intensive option B and chose option A, which ultimately scored the highest, demonstrating its ability to make trade-offs under realistic constraints.

[0033] Step 104: Based on the combination score of each candidate modality combination, select the best modality combination from the multiple candidate modality combinations, and present adaptive educational content to the special student based on the best modality combination.

[0034] It should be noted that the candidate modality combination with the highest combined score among all candidate modality combinations is usually determined as the optimal modality combination. To further optimize the decision, a secondary verification can be performed by combining a scoring threshold mechanism and student feedback history to ensure that the selected combination is not only theoretically optimal but also has good acceptability and stability in historical interactions. Finally, the optimal modality combination is used to generate the final optimized response package. The optimized response package contains specific content and detailed presentation parameters (such as font size, image style, and speech rate) to present appropriate educational content to students with special needs based on the specific content and presentation parameters.

[0035] Understandably, once the optimal modal combination is determined, a series of precise presentation instructions can be generated based on the modal composition of this optimal combination. For example, when the combination includes "auditory + tactile," the semantic structure and logic of the original response information can be analyzed, and multimodal generation can be driven based on the analysis results. Specifically, dynamic prosodic control based on the content structure of the original response information can be implemented for speech output: if the original response information contains step-by-step instructions, a clear pause and emphasis are inserted at the step transitions; if the original response information contains key concepts, the speech rate is automatically slowed down and the pitch is raised to create auditory emphasis. At the same time, structured vibration codes synchronized with the speech rhythm are generated for tactile feedback, such as outputting a short confirmation vibration when each step is completed, or generating a continuous differentiated vibration when emphasizing key concepts, thereby achieving complementary reinforcement and cognitive synchronization of information between the auditory and tactile channels. For another example, when the combination includes "visual + auditory," visual element analysis and auditory script generation can be performed on the original response information. Specifically, dynamic visual scheduling based on information structure is implemented for visual output: if the original response information contains process descriptions, it is broken down into coherent illustrated steps, and the sequence of steps is clarified through highlighted arrows or progressive animations; if it contains abstract concepts, concrete diagrams or metaphorical animations are automatically generated, and the visual salience of core elements is enhanced through color contrast and motion trajectories. Simultaneously, layered explanations are generated for auditory output that are strictly synchronized with the visual evolution: whenever the visual interface enters a new step or presents a new element, the corresponding explanation is provided in real time, and by adjusting the speech rate and inserting appropriate pauses, students can naturally switch their attention between the visual and auditory channels. This achieves a deep integration of visual guidance and auditory explanation at the temporal, rhythmic, and semantic levels, forming a multi-channel collaborative experience that supports understanding and memory.

[0036] This invention determines multiple candidate modality combinations based on interactive feedback signals from special needs students; determines the modal bandwidth and resource consumption of the multiple candidate modality combinations, and determines the student's cognitive fit based on the real-time student profile of the special needs student; determines the combination score of each candidate modality combination based on the modal bandwidth, the resource consumption, and the student's cognitive fit; selects the optimal modality combination from the multiple candidate modality combinations based on the combination score of each candidate modality combination, and presents appropriate educational content to the special needs student based on the optimal modality combination. This invention addresses the technical challenge of providing precise, smooth, and resource-efficient adaptive interactive support for students with special needs in complex and ever-changing real-world scenarios using traditional solutions. Compared to existing technologies, this invention determines the cognitive fit of students with special needs by constructing real-time updated student profiles. It then determines the modal combination that best matches the user's real-time state and task requirements by quantitatively evaluating modal bandwidth, resource consumption, and student cognitive fit. Finally, it generates and presents adapted content based on this optimal modal combination. This enables precise, smooth, natural, resource-efficient, and continuously evolving adaptive interactive support for students with special needs in real and dynamic teaching scenarios, significantly improving the usability, inclusivity, and effectiveness of human-computer interaction in special education settings.

[0037] Based on any of the above embodiments, step 101 includes: After receiving the interaction request from the special student, a question is sent to the special student, and the interaction feedback signal from the special student in response to the question is received. The interactive feedback signal is processed to obtain the intent recognition result, and multiple candidate modality combinations are determined based on the intent recognition result.

[0038] In practical implementation, the interactive feedback signals can be processed using a preset processing model to obtain intent recognition results, and multiple candidate modality combinations can be determined based on these results. Specifically, the preset processing model typically integrates multiple different modality signal processing models, enabling simultaneous analysis and feature extraction of parallel, heterogeneous interactive feedback signals (such as the spatial trajectory of gestures, semantic and emotional features of speech, and the force and position sequence of touch). Then, a fusion decision algorithm is used to infer the student's current intent. Based on the student's historical interaction records, cognitive ability assessment, and current learning stage, questions with clear intent guidance can be dynamically generated, aiming to guide students to make identifiable and analyzable feedback behaviors, thereby reducing the ambiguity of intent inference.

[0039] In its implementation, upon receiving an interaction request from a special student, the system first refers to the student's historical behavior information to initiate personalized questions. Subsequently, it simultaneously receives and processes interactive feedback signals that the student may send in parallel through gestures, voice, touch, and other means. Through a preset processing model, these interactive feedback signals are analyzed to identify the student's true intention. Finally, based on the intention recognition results, one or more suitable combinations of interaction methods (i.e., candidate modal combinations) can be dynamically selected to achieve more accurate, efficient, and humanized interaction support.

[0040] The special education assistance method provided in this invention involves, after receiving an interaction request from a special student, initiating a question to the student and receiving the student's interactive feedback signal in response to the question; processing the interactive feedback signal to obtain an intent recognition result, and determining multiple candidate modality combinations based on the intent recognition result. This method can transform ambiguous student interaction signals into analyzable intent data, overcoming the problem of misjudgment of intent caused by differences in students' expressive abilities in traditional methods; and dynamically generating multiple suitable candidate modality combinations based on objective intent recognition results, providing accurate decision-making basis for subsequent personalized resource scheduling and content presentation, thereby significantly improving the accuracy of human-computer interaction and the pertinence of teaching intervention, effectively supporting the personalized learning process of special students.

[0041] Based on any of the above embodiments, in this method, processing the interactive feedback signal to obtain the intent recognition result includes: The interactive feedback signal is processed by a preset processing model to obtain the confidence score of the candidate intent, and the model weight value is determined based on the real-time student profile of the special student.

[0042] Based on the confidence score and the model weight value, the intent score of the candidate intent is determined, the intent recognition result is determined according to the intent score, and multiple candidate modality combinations are determined according to the intent recognition result.

[0043] It should be noted that the preset processing model includes at least one of the auditory processing model, visual processing model, and tactile processing model. Instead of simply selecting one preset processing model for processing these interactive feedback signals, multiple preset processing models are used to process the interactive feedback signals in parallel. For example, each specialized processing model (auditory, visual, tactile) provides a confidence score for each candidate intent. Then, the model weight values ​​of different preset processing models are extracted from the student profile (for example, if the student relies more on the visual processing model, the model weight value of the visual processing model will be higher). Next, based on all the model weight values ​​and confidence scores, an intent score is calculated for each candidate intent. Finally, the candidate intent with the highest intent score is determined as the final intent recognition result.

[0044] In practical implementation, a special student who primarily uses sign language (visual channel) but sometimes expresses "agreement" using specific vocalizations (vocal channel) can be identified as primarily relying on visual and auditory processing models for intent recognition. Specifically, the model weights of different processing models can be determined from the student's real-time profile, i.e., visual processing model (omega_visual) = 0.7, and auditory processing model (omega_audio) = 0.3.

[0045] Interaction: When our electronic device (such as an AI-powered electronic student ID) asks a special needs student, "Do you need me to explain the 'water cycle' to you?" (This question involves two candidate intentions: "confirmation" and "uncertainty"), the student nods slightly and makes a muffled sound.

[0046] Parallel processing and output of the processing model: Visual processing model: Recognizes the "nodding" gesture and outputs confidence scores for different candidate intentions: {"Candidate Intention": "Confirmation", "Confidence Score": 0.8} and {"Intention": "Uncertain", "Confidence": 0}.

[0047] Auditory processing model: When an ambiguous sound is detected, based on the student's personal vocalization database, two possible candidate intentions are output: {"Candidate Intention": "Confirmation", "Confidence Score": 0.5} and {"Candidate Intention": "Uncertainty", "Confidence Score": 0.4}.

[0048] In practice, the formula for calculating the intent score is as follows: In the formula, I fused_score Represents a candidate intent H k Intent score; n represents the number of preset processing models for parallel processing (e.g., auditory, visual, tactile); C i (Hk ) indicates that the i-th preset processing model is for the candidate intent H k The output confidence score (between 0 and 1) indicates whether the preset processing model does not support or recognize the candidate intent H. k If so, then this term is 0; w i This represents the model weight value of the specific student extracted from the student profile for the i-th preset processing model.

[0049] In the specific implementation, the intent score is calculated as follows:

[0050] The special education assistance method provided in this invention processes the interactive feedback signal through a preset processing model to obtain a confidence score for the candidate intent, and determines the model weight value based on the real-time student profile of the special student. Based on the confidence score and the model weight value, the intent score of the candidate intent is determined, and the intent recognition result is determined based on the intent score. This technical solution can adaptively adjust the contribution ratio of different modal processing models in the final decision based on each student's sensory ability advantages, cognitive habits, and real-time state (such as attention distribution and emotional fluctuations), thereby significantly improving the accuracy and robustness of understanding individual differences in expression patterns. For example, for students with strong visual perception, the visual processing model can be given a higher weight; in noisy environments, the weight value of the auditory model may be reduced. This personalized weight allocation strategy effectively solves the problem of misjudgment of intent caused by differences in students' sensory and expressive abilities in special education scenarios by traditional multimodal fusion methods, enabling the system to capture students' true interactive intent more accurately and stably, laying a reliable foundation for providing accurate and adaptive learning support in the future.

[0051] Based on any of the above embodiments, in this method, the candidate intent includes a determined intent and an uncertain intent, and the step of determining a combination of multiple candidate modalities based on the intent recognition result includes: When the intent score of the determined intent is greater than or equal to the intent score of the uncertain intent and the intent score of the determined intent is greater than or equal to the intent score threshold, the intent recognition result is determined as a determined intent, and multiple candidate modal combinations of the interaction feedback signal are determined. When the intent score of the determined intent is less than the intent score of the uncertain intent or the intent score of the determined intent is less than the intent score threshold, the intent recognition result is determined to be an uncertain intent. Based on the clarification strategy and the special student's historical interaction information, clarification question information is generated, and the special student is guided to provide feedback again based on the clarification question information.

[0052] It should be noted that after obtaining the intent scores of each candidate intent (such as "confirmed" and "uncertain"), a comparison needs to be made between the two key opposing categories of "definite intent" and "uncertain intent": if the intent score of "definite intent" is not lower than the score of "uncertain intent" and not lower than the intent score threshold set by the system, then the student's intent is determined to be clear, and the system can directly determine multiple candidate modality combinations accordingly.

[0053] The special educational assistance method provided in this invention determines the intent recognition result as a definite intent when the intent score of the definite intent is greater than or equal to the intent score of the uncertain intent and the intent score of the definite intent is greater than or equal to an intent score threshold, and then determines multiple candidate modal combinations of the interaction feedback signal. When the intent score of the definite intent simultaneously satisfies "not lower than the uncertain intent score" and "reaches or exceeds a preset intent score threshold", the intent is determined to be clear, and multiple suitable candidate modal combinations are determined accordingly. This effectively avoids making erroneous decisions under low confidence and ensures that subsequent interaction content is highly matched with the student's actual needs.

[0054] Based on any of the above embodiments, after determining whether the intent score of the determined intent is greater than or equal to the intent score of the uncertain intent, the method further includes: When the intent score of the determined intent is less than the intent score of the uncertain intent or the intent score of the determined intent is less than the intent score threshold, the intent recognition result is determined to be an uncertain intent. Based on the clarification strategy and the special student's historical interaction information, clarification question information is generated, and the special student is guided to provide feedback again based on the clarification question information.

[0055] It should be noted that if the intention score of "definite intention" is lower than that of "uncertain intention" or if it does not reach the minimum confidence threshold (i.e., the intention score of "definite intention" is less than the intention score threshold), then the current understanding confidence is deemed insufficient, and the intention state is "uncertain." In this case, the system will not force modality selection but will trigger an intelligent clarification process. This process combines general clarification strategies (such as providing options and narrowing the question scope) with the student's historical interaction information (such as past successful clarification cases and preferred question modalities) to generate and output appropriate clarification questions, guiding the student to provide supplementary or corrective feedback. For example, when clarifying for a student with cognitive impairment, a clarification strategy can be invoked to generate highly simplified binary speech options (such as "Is this it? Yes / No"). Simultaneously, the clarification process will be recorded, and data such as the number of clarifications will be amplified using a power function and incorporated into the cognitive load assessment, ultimately forming a closed loop for updating the student profile and optimizing subsequent clarification strategies.

[0056] The special education assistance method provided in this invention automatically determines an uncertain intent when the intent score does not simultaneously meet the above conditions. It then triggers an intelligent clarification process based on clarification strategies and students' historical interaction information. This process generates guiding clarification questions to proactively guide students to provide further feedback, thus constructing a multi-round, progressive interactive loop even when the initial intent is unclear. This significantly enhances the system's tolerance and analytical capabilities for ambiguous expressions and unconventional feedback, preventing interaction interruptions. Furthermore, it naturally integrates teaching support into the clarification process through personalized guidance strategies, helping students gradually clarify their intent.

[0057] Figure 2 This is the second flowchart of the special education assistance method provided by the present invention, as shown below. Figure 2 As shown, step 104 is followed by steps 105 to 108: Step 105: Record the multidimensional feedback data during this interaction process, wherein the multidimensional feedback data includes task completion status, student participation, student real-time cognitive load, strategy fit, and modality optimization.

[0058] It should be noted that: Task completion is used to measure the degree of success of students in completing the task; student participation is used to assess students' initiative and emotional investment in the interaction; strategy fit is used to assess whether the system's chosen coping strategies are accurate and effective when encountering interaction obstacles; modal optimization is used to assess whether the system's final output response and its multimodal combination perfectly adapt to students' perceptual preferences; real-time cognitive load of students is used to quantify the mental effort students put into the interaction, with particular attention to the number of clarifications; personalized weights are used to reflect the importance of each dimension in assessing the current interaction effectiveness of a specific student, thereby achieving individualized customization of assessment standards; personalized weights can be dynamically optimized based on the total interaction effectiveness score obtained from the previous round of interaction, enabling the system's ability to identify and emphasize key dimensions affecting the student's effectiveness to continuously evolve as the interaction progresses.

[0059] Step 106: Process the multidimensional feedback data using a preset interaction performance evaluation model to obtain the total interaction performance score.

[0060] In summary, the specific formula for calculating the total interaction performance score using the preset interaction performance evaluation model is as follows: In the formula, E proc_score P represents the total score for interaction performance. task Indicates task completion status; E level Indicates student participation; S fit Indicates the degree of strategy fit; M opt Indicates modal optimization degree; Cload This represents the student's real-time cognitive load; w represents the individualized weight (w P w E w S w M w C ).

[0061] In the specific implementation, the calculation process for the total interaction performance score is as follows:

[0062] Step 107: Based on the total interaction performance score, dynamically optimize the real-time student profile and the preset interaction performance evaluation model.

[0063] It should be noted that a dual-loop learning mechanism can be used to dynamically optimize real-time student profiles and pre-defined interaction performance evaluation models. Specifically, the dual-loop learning mechanism includes an individual learning mechanism and a group learning mechanism. The individual learning mechanism mainly focuses on updating real-time student profiles, such as feeding analysis results back to the student profile database to continuously refine the understanding of the individual (e.g., updating the profile preference if it is found that the student responds best to a certain visual cue). The group learning mechanism mainly focuses on the pre-defined interaction performance model: machine learning techniques (such as reinforcement learning) can be applied to extract "universal patterns" from a large amount of interaction data.

[0064] Step 108: Based on the dynamically optimized real-time student profile and the preset interaction performance evaluation model, generate a new modal combination in the real-time state of the special student.

[0065] It can be explained that the real-time student profile (representing the student's latest status) after the previous round of interaction verification and optimization, and the preset interaction effectiveness evaluation model (representing the system's evaluation criteria) together serve as the input basis for the new round of modality matching. This generates a new optimal modality combination, which is the intelligent output of the system after personalized learning in the previous interaction. It can dynamically fit the changing curve of students' abilities and needs, thereby driving the auxiliary intervention to evolve towards a more precise and effective direction in continuous iteration.

[0066] This invention records multidimensional feedback data during the interaction process, including task completion status, student participation, real-time cognitive load, strategy fit, and modality optimization. The multidimensional feedback data is processed using a preset interaction effectiveness evaluation model to obtain a total interaction effectiveness score. Based on this total score, the real-time student profile and the preset interaction effectiveness evaluation model are dynamically optimized. Based on the dynamically optimized real-time student profile and the preset interaction effectiveness evaluation model, a new modality combination for the specific student's real-time state is generated. By recording multidimensional feedback data during the interaction process (including task completion status, student participation, cognitive load, strategy fit, and modality optimization), and comprehensively quantifying it using a preset interaction effectiveness evaluation model, a total interaction effectiveness score is generated. Based on this score, the real-time student profile and the evaluation model themselves are simultaneously optimized, thereby driving the dynamic generation of the optimal modality combination in subsequent interactions. This closed-loop mechanism achieves a leap from single-interaction adaptation to continuous collaborative evolution: based on objective performance feedback, student profiles can be continuously refined and assessment models can be continuously adapted, ultimately providing highly personalized and constantly optimized interactive support for each special student in long-term teaching, significantly improving learning outcomes and experience.

[0067] Based on any of the above embodiments, step 107 includes: When the total interaction performance score is less than the preset performance threshold, a negative adjustment suggestion is generated for the special student. When the total interaction performance score is greater than or equal to the preset performance threshold, a positive adjustment suggestion is generated for the special student. Based on the total interaction performance score, the personalized weights of the preset interaction performance evaluation model are optimized.

[0068] It's important to note that the core function of the negative adjustment suggestion lies in dynamically calibrating the system's internal "real-time student profile." Specifically, when the total effectiveness score of a teaching interaction falls below a preset threshold, the interaction is deemed unsatisfactory, and the system automatically analyzes the evaluation results to pinpoint specific weak interaction dimensions (such as communication initiative or task persistence). These weak dimensions are typically associated with insufficient effectiveness in a student's sensory processing channel (such as auditory, visual, or tactile). The key operation of the negative adjustment suggestion is then executed: the model weight value corresponding to the "preset processing model" (such as the auditory processing model or visual processing model) in the real-time student profile that has the highest correlation with this ineffective interaction is lowered.

[0069] It's important to note that the positive adjustment suggestion function aims to dynamically strengthen and consolidate the "real-time student profile" within the system. Specifically, when the total effectiveness score of a teaching interaction reaches or exceeds a preset threshold, it indicates that the current interaction strategy is effective. The system will automatically analyze and evaluate the results, identifying the key strengths that facilitated the efficient interaction (such as high communication initiative or strong task persistence). These strengths are typically closely related to the effectiveness of a student's sensory processing channel (such as visual, auditory, or tactile). The system then executes the key operation of the positive adjustment suggestion: increasing or maintaining the model weight value corresponding to the "preset processing model" (such as the visual processing model or auditory processing model) in the real-time student profile that has the highest correlation with this efficient interaction.

[0070] It should be noted that the individual learning mechanism primarily targets real-time student profiles. It correlates the total effectiveness score with the detailed values ​​of each feedback dimension. If the total score or the score of a specific dimension is unsatisfactory, the model weights of the preset processing models corresponding to the associated abilities or preferences in the profile are adjusted accordingly; conversely, they are enhanced. This allows the profile to more accurately reflect the dynamic changes in the student's state and real needs. Specifically, a preset effectiveness threshold can be used as a decision node. When the total interaction effectiveness score is below this threshold, it indicates that the current strategy or modality adaptation has not met expectations. Negative adjustment suggestions can be generated for this specific student, aiming to reduce the model weights of the preset processing models corresponding to the ability or preference items in the real-time student profile related to this inefficient interaction, thereby reducing the reliance on such strategies in future interactions. Conversely, when the total score is not lower than the threshold, positive adjustment suggestions are generated to maintain or increase the corresponding weights in the real-time student profile that contributed to this successful interaction, strengthening the effective strategy. Simultaneously, regarding the pre-defined interaction effectiveness evaluation model itself, the system utilizes accumulated interaction data and effectiveness results to optimize and update its internal evaluation parameters, particularly the personalized weights used to integrate multi-dimensional feedback data, through algorithms (such as machine learning gradient descent). This improves the accuracy of the model's prediction and evaluation of the student's individual feedback patterns. This allows the pre-defined interaction effectiveness evaluation model to more accurately quantify the student's unique feedback patterns, enabling the system's evaluation criteria and decision-making logic to continuously evolve alongside the interaction process, achieving synchronized and refined adaptation to the student's developmental status. Through this dual dynamic optimization, the system becomes more "familiar" with the student after each interaction cycle, and its evaluation criteria and decision-making basis evolve accordingly, ultimately achieving continuous improvement in the accuracy of teaching strategies and modal delivery through iteration.

[0071] This invention provides a mechanism that generates negative adjustment suggestions for special students when the total interaction effectiveness score is less than a preset effectiveness threshold, and positive adjustment suggestions when the total interaction effectiveness score is greater than or equal to the preset effectiveness threshold. Based on the total interaction effectiveness score, the personalized weights of the preset interaction effectiveness assessment model are optimized. By introducing a two-way adjustment mechanism based on effectiveness thresholds and a model parameter optimization process, an adaptive and refined evolution of the special education support system is achieved. This not only provides sensitive feedback on the student's real-time ability profile based on the total interaction effectiveness score—when effectiveness is below standard, negative adjustment suggestions promptly weaken the unsatisfactory model weight values ​​to prevent the recurrence of ineffective interaction patterns; when effectiveness is at or above standard, positive adjustment suggestions consolidate or strengthen the current effective strategies, thus ensuring that the student profile always maintains a high-fidelity representation of the individual's true cognitive state and preferences—but also continuously optimizes the personalized weights of the preset interaction effectiveness assessment model itself, enabling the assessment criteria to be dynamically calibrated along with the teaching process and student growth, continuously improving its predictive accuracy and personalized adaptability. Overall, this mechanism transforms the results of a single interaction into a driving force for the long-term evolution of the system. While ensuring the stability of teaching, it promotes the continuous iteration of auxiliary strategies towards higher efficiency and better meeting the real-time needs of students, ultimately achieving an auxiliary effect that becomes smarter and more personalized with use.

[0072] The special education assistive device provided by the present invention is described below. The special education assistive device described below can be referred to in correspondence with the special education assistive method described above.

[0073] like Figure 3 As shown, the special education assistive device includes: Module 10 is used to determine multiple candidate modality combinations based on the interactive feedback signals of special students; The determining module 10 is also used to determine the modal bandwidth and resource consumption of the multiple candidate modal combinations, and to determine the student cognitive fit based on the real-time student profile of the special student. The determining module 10 is further configured to determine the combination score of each candidate modality combination based on the modal bandwidth, the resource consumption, and the student cognitive fit. The presentation module 20 is used to select the best modality combination from the multiple candidate modality combinations based on the combination score of each candidate modality combination, and to present adaptive educational content to the special student based on the best modality combination.

[0074] Optionally, the determining module 10 is further configured to: After receiving the interaction request from the special student, a question is sent to the special student, and the interaction feedback signal from the special student in response to the question is received. The interactive feedback signal is processed to obtain the intent recognition result, and multiple candidate modality combinations are determined based on the intent recognition result.

[0075] Optionally, the determining module 10 is further configured to: The interactive feedback signal is processed by a preset processing model to obtain the confidence score of the candidate intent, and the model weight value is determined according to the real-time student profile of the special student. Based on the confidence score and model weight value of the preset processing model, the intent score of the candidate intent is determined, and the intent recognition result is determined according to the intent score.

[0076] Optionally, the determining module 10 is further configured to: Determine whether the intent score of the determined intent is greater than or equal to the intent score of the uncertain intent, and determine whether the intent score of the determined intent is greater than or equal to an intent score threshold; When the intent score of the determined intent is greater than or equal to the intent score of the uncertain intent and the intent score of the determined intent is greater than or equal to the intent score threshold, the intent recognition result is determined as a determined intent, and multiple candidate modal combinations of the interaction feedback signal are determined. Optionally, the determining module 10 is further configured to: When the intent score of the determined intent is less than the intent score of the uncertain intent or the intent score of the determined intent is less than the intent score threshold, the intent recognition result is determined to be an uncertain intent. Based on the clarification strategy and the special student's historical interaction information, clarification question information is generated, and the special student is guided to provide feedback again based on the clarification question information.

[0077] Optionally, the presentation module 20 is further configured to: Record multidimensional feedback data during this interaction process, including task completion status, student participation, student real-time cognitive load, strategy fit, and modality optimization. The multidimensional feedback data is processed by a preset interaction performance evaluation model to obtain the total interaction performance score. Based on the total interaction performance score, the real-time student profile and the preset interaction performance evaluation model are dynamically optimized. Based on the dynamically optimized real-time student profile and the preset interaction performance evaluation model, a new modal combination in the real-time state of the special student is generated.

[0078] Optionally, the presentation module 20 is further configured to: When the total interaction performance score is less than the preset performance threshold, a negative adjustment suggestion is generated for the special student. When the total interaction performance score is greater than or equal to the preset performance threshold, a positive adjustment suggestion is generated for the special student. Based on the total interaction performance score, the personalized weights of the preset interaction performance evaluation model are optimized.

[0079] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a special education assistance method. This method includes: determining multiple candidate modality combinations based on the interactive feedback signals of the special needs student; determining the modality bandwidth and resource consumption of the multiple candidate modality combinations, and determining the student's cognitive fit based on the student's real-time student profile; determining a combination score for each candidate modality combination based on the modality bandwidth, the resource consumption, and the student's cognitive fit; selecting the optimal modality combination from the multiple candidate modality combinations based on the combination score of each candidate modality combination, and presenting appropriate educational content to the special needs student based on the optimal modality combination.

[0080] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the special education assistance method provided by the above methods. The method includes: determining multiple candidate modality combinations based on the interactive feedback signals of special students; determining the modality bandwidth and resource consumption of the multiple candidate modality combinations, and determining the student's cognitive fit based on the real-time student profile of the special student; determining the combination score of each candidate modality combination based on the modality bandwidth, the resource consumption, and the student's cognitive fit; selecting the best modality combination from the multiple candidate modality combinations based on the combination score of each candidate modality combination, and presenting appropriate educational content to the special student based on the best modality combination.

[0082] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processing model, is implemented to perform the special education assistance methods provided by the above methods. The method includes: determining multiple candidate modality combinations based on interactive feedback signals from a special needs student; determining the modal bandwidth and resource consumption of the multiple candidate modality combinations, and determining the student's cognitive fit based on the student's real-time student profile; determining a combination score for each candidate modality combination based on the modal bandwidth, the resource consumption, and the student's cognitive fit; selecting the optimal modality combination from the multiple candidate modality combinations based on the combination score of each candidate modality combination, and presenting appropriate educational content to the special needs student based on the optimal modality combination.

[0083] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0084] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A special education support method, characterized in that, include: Multiple candidate modality combinations were determined based on the interactive feedback signals of special students; Determine the modal bandwidth and resource consumption of the multiple candidate modal combinations, and determine the student cognitive fit based on the real-time student profile of the special student; Based on the modal bandwidth, resource consumption, and student cognitive fit, a combination score is determined for each candidate modal combination. Based on the combined score of each candidate modality combination, the best modality combination is selected from the plurality of candidate modality combinations, and adaptive educational content is presented to the special student based on the best modality combination.

2. The special education support method according to claim 1, characterized in that, The step of determining multiple candidate modality combinations based on the interactive feedback signals of special students includes: After receiving the interaction request from the special student, a question is sent to the special student, and the interaction feedback signal from the special student in response to the question is received. The interactive feedback signal is processed to obtain the intent recognition result, and multiple candidate modality combinations are determined based on the intent recognition result.

3. The special education support method according to claim 2, characterized in that, The process of processing the interactive feedback signal to obtain an intent recognition result, and determining multiple candidate modality combinations based on the intent recognition result, includes: The interactive feedback signal is processed by a preset processing model to obtain the confidence score of the candidate intent, and the model weight value is determined according to the real-time student profile of the special student. Based on the confidence score and the model weight value, the intent score of the candidate intent is determined, the intent recognition result is determined according to the intent score, and multiple candidate modality combinations are determined according to the intent recognition result.

4. The special education support method according to claim 3, characterized in that, The candidate intents include definite intents and indefinite intents; wherein, determining multiple candidate modality combinations based on the intent recognition result includes: Determine whether the intent score of the determined intent is greater than or equal to the intent score of the uncertain intent, and determine whether the intent score of the determined intent is greater than or equal to an intent score threshold; When the intent score of the determined intent is greater than or equal to the intent score of the uncertain intent and the intent score of the determined intent is greater than or equal to the intent score threshold, the intent recognition result is determined as a determined intent, and multiple candidate modal combinations of the interaction feedback signal are determined.

5. The special education support method according to claim 4, characterized in that, After determining whether the intent score of the determined intent is greater than or equal to the intent score of the uncertain intent, the method further includes: When the intent score of the determined intent is less than the intent score of the uncertain intent or the intent score of the determined intent is less than the intent score threshold, the intent recognition result is determined to be an uncertain intent. Based on the clarification strategy and the special student's historical interaction information, clarification question information is generated, and the special student is guided to provide feedback again based on the clarification question information.

6. The special education support method according to claim 1, characterized in that, Following the step of presenting adaptive educational content to the special student based on the optimal modality combination, the method further includes: Record multidimensional feedback data during this interaction process, including task completion status, student participation, student real-time cognitive load, strategy fit, and modality optimization. The multidimensional feedback data is processed by a preset interaction performance evaluation model to obtain the total interaction performance score. Based on the total interaction performance score, the real-time student profile and the preset interaction performance evaluation model are dynamically optimized. Based on the dynamically optimized real-time student profile and the preset interaction performance evaluation model, a new modal combination is generated for the real-time state of the special student.

7. The special education support method according to claim 6, characterized in that, The step of dynamically optimizing the real-time student profile and the preset interaction performance evaluation model based on the total interaction performance score includes: When the total interaction performance score is less than the preset performance threshold, a negative adjustment suggestion is generated for the special student. When the total interaction performance score is greater than or equal to the preset performance threshold, a positive adjustment suggestion is generated for the special student. Based on the total interaction performance score, the personalized weights of the preset interaction performance evaluation model are optimized.

8. A special education assistive device, characterized in that, include: The determination module is used to determine multiple candidate modality combinations based on the interactive feedback signals of special students. The determining module is also used to determine the modal bandwidth and resource consumption of the multiple candidate modal combinations, and to determine the student cognitive fit based on the real-time student profile of the special student. The determining module is further configured to determine the combination score of each candidate modality combination based on the modal bandwidth, the resource consumption, and the student cognitive fit. The presentation module is used to select the best modality combination from the multiple candidate modality combinations based on the combination score of each candidate modality combination, and to present adaptive educational content to the special student based on the best modality combination.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the special education assistance method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the special education assistance method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the special education assistance method as described in any one of claims 1 to 7.

Images