Classroom accidental condition analysis and coping system and method based on large model

Abstract

The invention discloses a classroom accidental condition analysis and response system based on a large model, and the system comprises a multi-modal data collection module which employs multiple devices to collect classroom data, mark timestamps, automatically adjust parameters, and carry out the caching and encryption transmission; the data preprocessing and feature engineering module is used for preprocessing data, including abnormal value correction, noise reduction, normalization, filling, desensitization, feature extraction and dimension reduction; the classroom accidental condition recognition module is used for constructing and training a large model, improving the accuracy and outputting a condition category and confidence by using multiple technologies; the influence evaluation and strategy generation module evaluates influences, matches or generates strategies according to results, and ensures feasibility; and the execution feedback and system optimization module is used for pushing a strategy and collecting a feedback optimization system. Through cooperation of multiple modules, accurate condition identification, scientific influence evaluation and effective strategy generation are realized, the classroom coping ability is improved, the teaching process is optimized, the teaching efficiency and quality are improved, and intelligent development of education is promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of teaching systems, and in particular to a system and method for analyzing and coping with incidental classroom situations based on a large model. Background Art

[0002] In today's education field, classroom teaching is a key link in knowledge transfer and student growth. However, the classroom environment is complex and changeable, full of various uncertainties, and incidental situations occur frequently. These incidental situations cover multiple aspects such as student behavior, teacher teaching process, and teaching equipment operation. For example, students may be distracted or lose control of their emotions due to personal emotions, external interference, etc., which affects their own learning effects and may disrupt classroom order; teachers may encounter sudden failures of teaching equipment during teaching, such as damage to multimedia projectors and failure of sound systems, resulting in interruptions in teaching, affecting teaching progress and students' learning experience; there may also be sudden order problems in the classroom, such as disputes and conflicts between students.

[0003] The traditional way of dealing with incidental situations in the classroom mainly relies on the teacher's personal experience and immediate judgment. Teachers need to balance teaching and classroom management during the teaching process. When faced with incidental situations, they may not be able to deal with them in a timely and accurate manner due to limited energy. Moreover, different teachers have different experiences and handling methods, and lack unified and scientific standards, making it difficult to ensure the stability and effectiveness of the response effect. At the same time, with the advancement of educational informatization, although teaching equipment is becoming increasingly intelligent, in terms of dealing with incidental situations, there is a lack of collaborative work and intelligent decision-making between devices, and an effective overall solution cannot be formed.

[0004] In recent years, artificial intelligence technology has developed rapidly, and big model technology has achieved remarkable results in the fields of natural language processing and image recognition. Applying big model technology to the analysis and response of incidental situations in the classroom is expected to break through the limitations of traditional methods. By integrating multi-source data and using the powerful learning and analysis capabilities of big models, it is possible to automatically identify incidental situations, accurately assess their impact, and intelligently generate response strategies, thereby improving the timeliness, scientificity, and effectiveness of responses, improving the quality of classroom teaching and management efficiency, and promoting the development of education and teaching in the direction of intelligence and precision. Summary of the invention

[0005] The present invention proposes a large-model-based classroom incidental situation analysis and response system and method to solve the problems mentioned in the above-mentioned prior art.

[0006] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solution: a classroom incidental situation analysis and response system based on a large model, comprising:

[0007] The multimodal data acquisition module is used to collect classroom-related data through high-definition cameras, high-sensitivity microphone arrays, environmental sensors and data interfaces connected to teaching equipment arranged in different locations in the classroom. The collected data are marked with precise timestamps and packaged and transmitted at preset time intervals, and have real-time data caching function;

[0008] The data preprocessing and feature engineering module performs comprehensive preprocessing on the collected multimodal data; uses rule-based and statistical analysis methods to identify and correct outliers in the data, and determines whether the current temperature data is abnormal by comparing the temperature range in historical data; uses signal processing algorithms to reduce the noise of audio data, and uses image enhancement technology to improve the quality of image data; uses normalization algorithms to map sensor data of different ranges to a unified interval; fills in missing data based on the time correlation of the data and the distribution of surrounding data; desensitizes and encrypts the data based on data privacy protection regulations and encryption standards; uses deep learning feature extraction algorithms to automatically mine key features related to sporadic classroom conditions from processed data, and fuses and reduces the dimensionality of these features;

[0009] The classroom incidental situation recognition module builds a large model based on a deep learning architecture. It uses a large-scale labeled classroom scene data set to pre-train the model and learn common classroom behavior patterns and situation characteristics. During the pre-training process, a distributed computing framework is used to accelerate the training process. Then, samples of classroom incidental situations in specific scenarios are used for adjustment. The attention mechanism is introduced into the model, and the model outputs the category label and corresponding confidence score of the classroom incidental situation. For the recognition results with a confidence score lower than the set threshold, a secondary recognition or manual review process is automatically triggered.

[0010] The impact assessment and strategy generation module uses the analytic hierarchy process and fuzzy comprehensive evaluation method to quantitatively assess the impact of the situation on the current teaching effect and student learning process based on the identified incidental classroom situations. Based on the impact assessment results, it intelligently matches or dynamically generates corresponding response strategies from the predefined strategy library.

[0011] The execution feedback and system optimization module is responsible for pushing the generated response strategies to teacher terminals, smart teaching devices or student terminals in a timely and accurate manner.

[0012] Furthermore, the high-definition camera in the multimodal data acquisition module has automatic zoom and panoramic stitching functions, and automatically adjusts the shooting angle according to the classroom layout and the range of personnel activities. The microphone array uses beamforming technology to enhance the sound collection effect in specific areas. The environmental sensor has a self-calibration function to ensure data accuracy. The data interface supports multiple communication protocols and data format conversion. The acquisition frequency can be dynamically adjusted between 1 time per minute and 10 times per second. The data cache capacity is not less than 10GB and has a power-off protection function.

[0013] Furthermore, the data preprocessing and feature engineering module adopts a dynamic threshold setting method when processing outliers, and dynamically updates the anomaly judgment threshold according to the real-time distribution of the data; a machine learning algorithm is used to predict missing values ​​in the data filling process to improve the accuracy of the filled data; data desensitization uses an irreversible encryption algorithm to encrypt sensitive information, and the encrypted data cannot be restored to the original data; in the feature extraction process, a method combining principal component analysis PCA and linear discriminant analysis LDA is used to perform dimensionality reduction processing.

[0014] Furthermore, during the training of the large model in the classroom incidental situation recognition module, the stochastic gradient descent SGD algorithm combined with the adaptive learning rate strategy is used to optimize the model parameters. The initial value of the learning rate is set to 0.01, and is dynamically adjusted according to the number of training rounds and the change of the loss function. An adversarial training mechanism is introduced in the fine-tuning stage to enhance the robustness of the model to noisy data and abnormal conditions. The model evaluation indicators use accuracy, recall rate, F1 value and confusion matrix multi-dimensional indicators for comprehensive evaluation, and the validation set data is used regularly to evaluate the model performance. When the performance declines, the model retraining process is automatically triggered.

[0015] Furthermore, when the analytic hierarchy process in the impact assessment and strategy generation module determines the weights, a judgment matrix is ​​constructed by combining expert questionnaires with actual teaching data analysis, and the membership function in the fuzzy comprehensive evaluation method is customized according to historical data and expert experience; the strategies in the strategy library are classified, stored and managed according to different situation types, impact levels and applicable scenarios, and support online editing, adding and deleting operations of strategies. It also has a strategy recommendation function, which automatically recommends the optimal strategy combination according to the current classroom situation.

[0016] Furthermore, a method for analyzing and coping with incidental situations in the classroom based on a large model includes the following steps:

[0017] Data collection and integration step, obtaining classroom-related data, including student behavior data, teacher behavior data, classroom environment data, and teaching equipment operation data, through the multimodal data collection module; real-time verification and cleaning of the collected data to ensure the accuracy and completeness of the data, including data format check and data range verification; categorizing and storing the cleaned data according to time sequence and data type, establishing data indexes to facilitate subsequent retrieval and analysis, and backing up the data to a distributed storage system to prevent data loss, with a backup frequency of no less than once per hour;

[0018] Feature extraction and model adaptation steps: Use data preprocessing and feature engineering modules to preprocess and extract features from stored data, such as using wavelet transform to remove noise from audio data and using image segmentation algorithms to extract key area features from images; input the extracted features into a pre-trained large model for identifying incidental classroom situations, and standardize the features before input to make them meet the model input requirements; select appropriate model parameter configurations or fine-tune the model according to different course types and teaching scenarios to ensure the model's adaptability to current classroom conditions; use small sample incremental learning methods during fine-tuning to reduce the impact on the original model and improve the model's ability to learn new conditions;

[0019] The situation identification and judgment step analyzes the input feature data based on the big model to identify whether there are occasional situations in the classroom. If so, determine its specific type and occurrence time. In the identification process, set the confidence threshold to 0.7. For suspected situations with a confidence lower than the threshold, further analysis is performed in combination with contextual information and historical data, such as by analyzing student behavior data in the previous and subsequent periods to assist in judgment. Visually display the identification results, and prompt abnormal situations in a striking manner on the teacher's terminal and management background. The display content includes the situation type, occurrence time, and related video clips or image screenshots.

[0020] Impact assessment and strategy generation step: after identifying the occasional situation in the classroom, according to the rules and algorithms of the impact assessment and strategy generation module, comprehensively consider the current teaching progress, course objectives, individual differences of students and school resource conditions, and evaluate the impact of the situation on the teaching process and student learning; according to the impact assessment results, select or generate corresponding response strategies from the strategy library, and fully consider the feasibility, effectiveness and timeliness of the strategies when generating strategies. For example, for equipment failure conditions, give priority to strategies that can quickly restore the normal operation of the equipment; prioritize the generated strategies to ensure that important and urgent strategies are implemented first;

[0021] Strategy execution and feedback collection steps: push the response strategy to relevant personnel or equipment according to the priority of the strategy, and ensure that the information is accurate and delivered in a timely manner during the push process, such as notifying teachers through SMS, pop-up windows, and voice broadcasts; during the strategy execution process, monitor the execution status in real time, and record the start time, end time, and key information of the execution results; collect feedback from teachers, students, and other relevant personnel on the effectiveness of strategy execution. Feedback methods include evaluation on terminal devices and filling out questionnaires. The collection frequency is immediately after each strategy execution;

[0022] In the system optimization and knowledge updating steps, the strategy execution feedback information is collated and analyzed, and valuable information is extracted to optimize the classroom incidental situation identification model, impact assessment model, and strategy generation model; based on emerging classroom incidental situation cases and effective response strategies, the system's knowledge base is updated, including the case base, strategy base, and related model parameters and rules; the system's performance is evaluated and optimized regularly, the operating status of each module of the system is checked, and the system parameters are adjusted to ensure that the system is always in an efficient and stable operating state. The performance evaluation cycle is once a week.

[0023] Furthermore, in the data collection and integration steps, the installation location and parameter settings of the data collection equipment are optimized according to the classroom layout and personnel distribution, such as adjusting the shooting angle and focal length of the camera so that the image can clearly cover all areas of the classroom; in the data cleaning process, a data consistency check algorithm is used to ensure the logical consistency between data from different sources, such as checking whether the student attendance data matches the seat occupancy data.

[0024] Furthermore, in the feature extraction and model adaptation steps, a feature selection algorithm is used for high-dimensional feature data to screen out feature subsets that are more correlated with incidental classroom situations, reduce data dimensions, and improve model calculation efficiency; in the model fine-tuning process, transfer learning technology is used to migrate model parameters trained in other similar classroom scenarios to the current model, speeding up fine-tuning while avoiding overfitting.

[0025] Furthermore, in the step of situation identification and judgment, multimodal information fusion technology is used to comprehensively analyze multiple data modalities such as images, voice, and text to improve the accuracy of situation identification. When a new type of sporadic situation that is not recognized by the model occurs, the sample collection and labeling process is automatically started to collect and label relevant data to expand the training data set and update the model to adapt to the new situation.

[0026] Furthermore, in the impact assessment and strategy generation steps, a user participation mechanism is introduced to allow teachers and students to propose modification suggestions to the impact assessment results and generated strategies based on actual conditions, and the system makes adjustments based on feedback. In the strategy execution and feedback collection steps, a feedback information authenticity verification mechanism is established to ensure the reliability of feedback information by comparing feedback information from different sources and analyzing the rationality of feedback data, and to filter out false or invalid feedback.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] The present invention has significant beneficial effects in many aspects. First, in terms of situation recognition, by acquiring rich information through the multimodal data acquisition module, and using advanced deep learning algorithms and large model technology, it can accurately identify a variety of occasional classroom situations, and the recognition accuracy rate is significantly improved compared with traditional methods, reaching more than 90%, reducing misjudgments and missed judgments, and providing a reliable basis for subsequent effective response. For example, it can accurately distinguish whether students are thinking normally or distracted, and promptly discover hidden dangers of subtle equipment failures.

[0029] Secondly, the impact assessment and strategy generation module is based on big data and scientific algorithms, which can comprehensively and objectively assess the impact of incidental situations on teaching and generate targeted and feasible response strategies. This helps teachers to quickly take appropriate measures to reduce the interference of incidental situations on teaching progress and student learning, such as timely adjusting teaching methods when students are emotionally unstable, quickly switching to backup equipment or starting maintenance processes when equipment fails, to ensure the smooth progress of teaching activities and improve classroom teaching efficiency.

[0030] Furthermore, the closed-loop optimization mechanism formed by the execution feedback and system optimization module enables the system to continuously improve itself. Based on the strategy execution feedback, the model and strategy are continuously optimized to adapt to different classroom scenarios and changes in teaching styles, and maintain good performance for a long time. At the same time, the continuous updating of the knowledge base accumulates rich experience, provides decision-making support and teaching improvement reference for teachers, and improves teachers' coping ability and teaching level.

[0031] In addition, the visual display and interactive functions of the system facilitate teachers and administrators to understand the classroom situation in real time and intervene and manage in time. Overall, the present invention improves the level of intelligent classroom management, optimizes the teaching environment, promotes the improvement of education quality, and has positive significance for promoting the process of education modernization. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A schematic block diagram of a classroom incidental situation analysis and response system based on a large model proposed by the present invention;

[0033] Figure 2 This is a schematic diagram of the large-model-based classroom incidental situation analysis and response method proposed in the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise" and "counterclockwise" indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0036] In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined. In addition, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, and it can be the internal connection of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0037] Reference Figure 1-2 :A classroom incidental situation analysis and response system based on a large model, including:

[0038] The multimodal data acquisition module is used to comprehensively collect classroom-related data through high-definition cameras, high-sensitivity microphone arrays, various environmental sensors and data interfaces connected to teaching equipment arranged in different locations in the classroom. The cameras can capture students' facial expressions, body movements, teachers' teaching postures and the overall classroom activity picture. The microphone array can accurately obtain voice information of teachers and students. Environmental sensors monitor temperature, humidity, light intensity and noise level. The data interface collects teaching equipment operation status data in real time. The collected data are marked with precise timestamps and packaged and transmitted at preset time intervals. At the same time, it has real-time data caching function to prevent data transmission interruption and loss. During the data collection process, the collection parameters can be automatically adjusted according to changes in the classroom scene, such as improving the audio collection sensitivity during group discussions and enhancing the image acquisition frame rate of specific areas during the demonstration experiment stage.

[0039] The data preprocessing and feature engineering module performs comprehensive preprocessing on the collected multimodal data; uses rule-based and statistical analysis methods to identify and correct outliers in the data, for example, by comparing the temperature range in historical data to determine whether the current temperature data is abnormal; uses advanced signal processing algorithms to reduce the noise of audio data, and uses image enhancement technology to improve the quality of image data; uses normalization algorithms to map sensor data of different ranges to a unified interval; for missing data, intelligently fills in the missing data based on the time correlation of the data and the distribution of surrounding data; desensitizes and encrypts the data based on data privacy protection regulations and encryption standards to ensure data security; uses deep learning feature extraction algorithms to automatically mine key features related to incidental classroom situations from the processed data, including extracting expression and action feature vectors from image data, parsing semantic features from voice data, and refining state features from environment and equipment data, and fusing and reducing the dimensions of these features to generate a low-dimensional and representative feature set for subsequent analysis.

[0040] The classroom incidental situation recognition module builds a large model based on a deep learning architecture, such as a hybrid model combining a convolutional neural network (CNN) and a recurrent neural network (RNN); the model is pre-trained using a large-scale annotated classroom scene data set to learn common classroom behavior patterns and situation characteristics, and a distributed computing framework is used to accelerate the training process during the pre-training process; then the classroom incidental situation samples in specific scenarios are used for fine-tuning to adapt the model to situation recognition tasks under different course types and teaching styles; the attention mechanism is introduced into the model to focus on key data features and time series information to improve recognition accuracy; the model outputs the category label and corresponding confidence score of the classroom incidental situation, which can recognize a variety of common situations, such as student distraction (by analyzing eye contact, head movements and facial expression features related to learning concentration), student emotional fluctuations (judged by combining voice intonation changes and facial expression changes), classroom disorder (based on comprehensive judgment of multimodal information such as student group movement amplitude and sound noise level), and abnormal teaching equipment (through real-time monitoring and analysis of equipment operation data). For recognition results with confidence levels lower than the set threshold, secondary recognition or manual review processes are automatically triggered.

[0041] The impact assessment and strategy generation module uses the analytic hierarchy process and fuzzy comprehensive evaluation method to quantify the assessment based on the identified incidental classroom situations, combined with the classroom teaching progress, course content difficulty, student individual learning ability model and school teaching resource allocation, to analyze the degree and scope of the impact of the situation on the current teaching effect and student learning process; based on the impact assessment results, intelligently match or dynamically generate corresponding response strategies from the predefined strategy library. The strategies cover multiple aspects such as adjusting teaching methods (such as changing the rhythm of explanations and increasing interactive links), guiding student behavior (such as sending personalized reminders through smart devices and guiding students to adjust their status), and handling equipment faults (such as automatically triggering maintenance work orders and switching to backup equipment); in the process of strategy generation, the school's existing resources and actual operational feasibility are fully considered to ensure the effectiveness and enforceability of the strategy, while supporting teachers to manually adjust and optimize the strategy according to on-site conditions.

[0042] The execution feedback and system optimization module is responsible for pushing the generated response strategies to the teacher's terminal, intelligent teaching equipment or student terminal in a timely and accurate manner. The push method is adaptively selected according to the usage status and device type of the recipient, such as pushing through a combination of pop-up windows and voice prompts during the teacher's teaching process, and pushing through device vibration and screen prompts when the student is learning; real-time collection of feedback information during the strategy execution process, including execution time, execution results, and subjective evaluations of teachers and students, and use natural language processing technology to perform semantic analysis on the feedback text and extract key information; feedback data is sent back to the system to optimize the parameters and rules of the classroom incidental situation recognition model, impact assessment model, and strategy generation model, forming a data-driven closed-loop optimization mechanism, and continuously improving the system's performance and accuracy in responding to classroom incidental situations.

[0043] In the present invention, the high-definition camera in the multimodal data acquisition module has automatic zoom and panoramic stitching functions, and can automatically adjust the shooting angle according to the classroom layout and the range of personnel activities. The microphone array adopts beamforming technology to enhance the sound collection effect of specific areas. The environmental sensor has a self-calibration function to ensure data accuracy. The data interface supports multiple communication protocols and data format conversion. The acquisition frequency can be dynamically adjusted between 1 time per minute and 10 times per second. The data cache capacity is not less than 10GB and has a power-off protection function.

[0044] In the present invention, the data preprocessing and feature engineering module adopts a dynamic threshold setting method when processing outliers, and dynamically updates the outlier judgment threshold according to the real-time distribution of the data; in the data filling process, a machine learning algorithm is used to predict missing values ​​to improve the accuracy of the filled data; data desensitization adopts an irreversible encryption algorithm to encrypt sensitive information, and the encrypted data cannot be restored to the original data; in the feature extraction process, a method combining principal component analysis (PCA) and linear discriminant analysis (LDA) is used to perform dimensionality reduction processing, retaining the principal components with a feature contribution rate of more than 90%, and using a deep belief network (DBN) to further mine deep-level features.

[0045] In the present invention, during the training process of the large model in the classroom incidental situation recognition module, the stochastic gradient descent (SGD) algorithm is combined with an adaptive learning rate strategy to optimize the model parameters, the initial value of the learning rate is set to 0.01, and it is dynamically adjusted according to the number of training rounds and the change of the loss function; an adversarial training mechanism is introduced in the fine-tuning stage to enhance the robustness of the model to noisy data and abnormal conditions; the model evaluation indicators are comprehensively evaluated using accuracy, recall rate, F1 value and confusion matrix multi-dimensional indicators, and the validation set data is used regularly to evaluate the model performance. When the performance declines, the model retraining process is automatically triggered.

[0046] In the present invention, when the weights are determined by the hierarchical analysis method in the impact assessment and strategy generation module, a judgment matrix is ​​constructed by combining expert questionnaire surveys with actual teaching data analysis, and the membership function in the fuzzy comprehensive evaluation method is customized according to historical data and expert experience; the strategies in the strategy library are classified, stored and managed according to different situation types, impact levels and applicable scenarios, and support online editing, adding and deleting operations of strategies. At the same time, it has a strategy recommendation function, which automatically recommends the optimal strategy combination according to the current classroom situation.

[0047] The present invention also discloses a method for analyzing and coping with incidental situations in a classroom based on a large model, comprising the following steps:

[0048] The data collection and integration step obtains classroom-related data, including student behavior data, teacher behavior data, classroom environment data, and teaching equipment operation data, through the multimodal data collection module; performs real-time verification and cleaning of the collected data to ensure the accuracy and completeness of the data, and the verification content includes data format check and data range verification; the cleaned data is classified and stored according to time sequence and data type, and a data index is established to facilitate subsequent retrieval and analysis. At the same time, the data is backed up to a distributed storage system to prevent data loss, and the backup frequency is not less than once an hour.

[0049] In the feature extraction and model adaptation steps, data preprocessing and feature engineering modules are used to preprocess and extract features from the stored data, such as using wavelet transform to remove noise from audio data and using image segmentation algorithms to extract key area features from images. The extracted features are input into a pre-trained large model for identifying incidental classroom situations, and the features are standardized before input to meet the model input requirements. According to different course types and teaching scenarios, appropriate model parameter configurations are selected or the model is fine-tuned to ensure the model's adaptability to the current classroom situation. A small sample incremental learning method is used during the fine-tuning process to reduce the impact on the original model while improving the model's ability to learn new situations.

[0050] In the step of situation identification and judgment, the input feature data is analyzed based on the big model to identify whether there are occasional situations in the classroom. If so, its specific type and occurrence time are determined. In the identification process, the confidence threshold is set to 0.7. For suspected situations with confidence lower than the threshold, further analysis is performed in combination with contextual information and historical data, such as by analyzing student behavior data in the previous and subsequent periods to assist in judgment. The recognition results are visualized and abnormal situations are highlighted in a striking manner on the teacher's terminal and the management background. The displayed content includes the situation type, occurrence time, and related video clips or image screenshots.

[0051] Impact assessment and strategy generation step: After identifying incidental classroom situations, based on the rules and algorithms of the impact assessment and strategy generation module, comprehensively consider the current teaching progress, course objectives, individual differences of students, and school resource conditions to assess the impact of the situation on the teaching process and student learning; based on the impact assessment results, select or generate corresponding response strategies from the strategy library, and fully consider the feasibility, effectiveness, and timeliness of the strategies when generating strategies. For example, for equipment failure conditions, give priority to strategies that can quickly restore normal equipment operation; prioritize the generated strategies to ensure that important and urgent strategies are executed first.

[0052] The strategy execution and feedback collection steps are as follows: push the response strategy to relevant personnel or equipment according to the priority of the strategy. Ensure that the information is accurate and delivered in a timely manner during the push process, such as notifying teachers through text messages, pop-up windows, and voice broadcasts. During the strategy execution process, monitor the execution status in real time and record the key information of the execution start time, end time, and execution results. Collect feedback from teachers, students, and other relevant personnel on the effectiveness of strategy implementation. Feedback methods include evaluation on terminal devices and filling out questionnaires. The collection frequency is immediately after each strategy execution.

[0053] In the system optimization and knowledge updating steps, the strategy execution feedback information is collated and analyzed, and valuable information is extracted to optimize the classroom incidental situation identification model, impact assessment model, and strategy generation model; based on emerging classroom incidental situation cases and effective response strategies, the system's knowledge base is updated, including the case base, strategy base, and related model parameters and rules; the system's performance is evaluated and optimized regularly, the operating status of each module of the system is checked, and the system parameters are adjusted to ensure that the system is always in an efficient and stable operating state. The performance evaluation cycle is once a week.

[0054] In the present invention, in the data collection and integration steps, the installation position and parameter settings of the data collection equipment are optimized according to the classroom layout and personnel distribution, such as adjusting the shooting angle and focal length of the camera so that the image can clearly cover all areas of the classroom; in the data cleaning process, a data consistency check algorithm is used to ensure the logical consistency between data from different sources, such as checking whether the student attendance data matches the seat occupancy data.

[0055] In the present invention, in the feature extraction and model adaptation steps, a feature selection algorithm is used for high-dimensional feature data to screen out feature subsets with high correlation with incidental classroom situations, reduce data dimensions, and improve model calculation efficiency; in the model fine-tuning process, transfer learning technology is used to migrate model parameters trained in other similar classroom scenarios to the current model, speed up the fine-tuning speed, and avoid overfitting.

[0056] In the present invention, in the condition identification and judgment steps, multimodal information fusion technology is used to comprehensively analyze multiple data modes such as images, voice, and text to improve the accuracy of condition identification; when a new type of sporadic condition that is not recognized by the model occurs, the sample collection and labeling process is automatically started, and relevant data is collected and labeled for use in expanding the training data set, and the model is updated to adapt to the new condition.

[0057] In the present invention, in the impact assessment and strategy generation steps, a user participation mechanism is introduced to allow teachers and students to propose modification suggestions for the impact assessment results and the generated strategies according to actual conditions, and the system makes adjustments based on the feedback. In the strategy execution and feedback collection steps, a feedback information authenticity verification mechanism is established to ensure the reliability of the feedback information by comparing feedback information from different sources and analyzing the rationality of the feedback data, and to filter out false or invalid feedback.

[0058] The above are only preferred specific implementation modes of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can replace or change the technical solution and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A classroom incidental situation analysis and response system based on a large model, characterized by: include: The multimodal data acquisition module is used to collect classroom-related data through high-definition cameras, high-sensitivity microphone arrays, environmental sensors and data interfaces connected to teaching equipment arranged in different locations in the classroom. The collected data are marked with precise timestamps and packaged and transmitted at preset time intervals, and have real-time data caching function; The data preprocessing and feature engineering module performs comprehensive preprocessing on the collected multimodal data; uses rule-based and statistical analysis methods to identify and correct outliers in the data, and determines whether the current temperature data is abnormal by comparing the temperature range in the historical data; uses signal processing algorithms to reduce the noise of audio data, and uses image enhancement technology to improve the quality of image data; uses normalization algorithms to map sensor data of different ranges to a unified range; For missing data, fill it in according to the time correlation of the data and the distribution of surrounding data; desensitize and encrypt the data based on data privacy protection regulations and encryption standards; use deep learning feature extraction algorithms to automatically mine key features related to classroom incidental conditions from the processed data, and fuse and reduce the dimension of these features; The classroom incidental situation recognition module builds a large model based on a deep learning architecture. It uses a large-scale labeled classroom scene data set to pre-train the model and learn common classroom behavior patterns and situation characteristics. During the pre-training process, a distributed computing framework is used to accelerate the training process. Then, samples of classroom incidental situations in specific scenarios are used for adjustment. The attention mechanism is introduced into the model, and the model outputs the category label and corresponding confidence score of the classroom incidental situation. For the recognition results with a confidence score lower than the set threshold, a secondary recognition or manual review process is automatically triggered. The impact assessment and strategy generation module uses the analytic hierarchy process and fuzzy comprehensive evaluation method to quantitatively assess the impact of the situation on the current teaching effect and student learning process based on the identified incidental classroom situations. Based on the impact assessment results, it intelligently matches or dynamically generates corresponding response strategies from the predefined strategy library. The execution feedback and system optimization module is responsible for pushing the generated response strategies to teacher terminals, smart teaching devices or student terminals in a timely and accurate manner.

2. The classroom incidental situation analysis and response system based on a large model according to claim 1 is characterized in that: The high-definition camera in the multimodal data acquisition module has automatic zoom and panoramic stitching functions, and automatically adjusts the shooting angle according to the classroom layout and the range of personnel activities. The microphone array uses beamforming technology to enhance the sound collection effect in specific areas. The environmental sensor has a self-calibration function to ensure data accuracy. The data interface supports multiple communication protocols and data format conversion. The acquisition frequency can be dynamically adjusted between 1 time per minute and 10 times per second. The data cache capacity is not less than 10GB and has a power-off protection function.

3. The classroom incidental situation analysis and response system based on a large model according to claim 1 is characterized in that: The data preprocessing and feature engineering module adopts a dynamic threshold setting method when processing outliers, and dynamically updates the outlier judgment threshold according to the real-time distribution of data; in the process of data filling, a machine learning algorithm is used to predict missing values ​​to improve the accuracy of filled data; Data desensitization uses an irreversible encryption algorithm to encrypt sensitive information, and the encrypted data cannot be restored to the original data; in the feature extraction process, a combination of principal component analysis PCA and linear discriminant analysis LDA is used to reduce dimensionality.

4. The classroom incidental situation analysis and response system based on a large model according to claim 1 is characterized in that: During the training of the large model in the classroom incidental situation recognition module, the stochastic gradient descent SGD algorithm is combined with an adaptive learning rate strategy to optimize the model parameters. The initial value of the learning rate is set to 0.01, and is dynamically adjusted according to the number of training rounds and changes in the loss function. An adversarial training mechanism is introduced in the fine-tuning stage to enhance the robustness of the model to noisy data and abnormal conditions. The model evaluation indicators are comprehensively evaluated using accuracy, recall rate, F1 value, and confusion matrix multi-dimensional indicators. The validation set data is used regularly to evaluate the model performance, and the model retraining process is automatically triggered when the performance degrades.

5. The system for analyzing and coping with incidental situations in a classroom based on a large model according to claim 1 is characterized in that: When determining the weights by the hierarchical analysis method in the impact assessment and strategy generation module, a judgment matrix is ​​constructed by combining expert questionnaire surveys with actual teaching data analysis, and the membership function in the fuzzy comprehensive evaluation method is customized according to historical data and expert experience; the strategies in the strategy library are classified, stored and managed according to different situation types, impact levels and applicable scenarios, and support online editing, adding and deleting operations of strategies. It also has a strategy recommendation function, which automatically recommends the optimal strategy combination according to the current classroom situation.

6. A method for analyzing and coping with incidental situations in the classroom based on a large model, characterized in that: The following steps are involved: Data collection and integration step, obtaining classroom-related data, including student behavior data, teacher behavior data, classroom environment data, and teaching equipment operation data, through the multimodal data collection module; real-time verification and cleaning of the collected data to ensure the accuracy and completeness of the data, including data format check and data range verification; categorizing and storing the cleaned data according to time sequence and data type, establishing data indexes to facilitate subsequent retrieval and analysis, and backing up the data to a distributed storage system to prevent data loss, with a backup frequency of no less than once per hour; Feature extraction and model adaptation steps: Use data preprocessing and feature engineering modules to preprocess and extract features from stored data, such as using wavelet transform to remove noise from audio data and using image segmentation algorithms to extract key area features from images; input the extracted features into a pre-trained large model for identifying incidental classroom situations, and standardize the features before input to make them meet the model input requirements; select appropriate model parameter configurations or fine-tune the model according to different course types and teaching scenarios to ensure the model's adaptability to current classroom conditions; use small sample incremental learning methods during fine-tuning to reduce the impact on the original model and improve the model's ability to learn new conditions; The situation identification and judgment step analyzes the input feature data based on the big model to identify whether there are occasional situations in the classroom. If so, determine its specific type and occurrence time. In the identification process, set the confidence threshold to 0.

7. For suspected situations with a confidence lower than the threshold, further analysis is performed in combination with contextual information and historical data, such as by analyzing student behavior data in the previous and subsequent periods to assist in judgment. Visually display the identification results, and prompt abnormal situations in a striking manner on the teacher's terminal and management background. The display content includes the situation type, occurrence time, and related video clips or image screenshots. Impact assessment and strategy generation step: after identifying the occasional situation in the classroom, according to the rules and algorithms of the impact assessment and strategy generation module, comprehensively consider the current teaching progress, course objectives, individual differences of students and school resource conditions, and evaluate the impact of the situation on the teaching process and student learning; according to the impact assessment results, select or generate corresponding response strategies from the strategy library, and fully consider the feasibility, effectiveness and timeliness of the strategies when generating strategies. For example, for equipment failure conditions, give priority to strategies that can quickly restore the normal operation of the equipment; prioritize the generated strategies to ensure that important and urgent strategies are implemented first; Strategy execution and feedback collection steps: push the response strategy to relevant personnel or equipment according to the priority of the strategy, and ensure that the information is accurate and delivered in a timely manner during the push process, such as notifying teachers through SMS, pop-up windows, and voice broadcasts; during the strategy execution process, monitor the execution status in real time, and record the start time, end time, and key information of the execution results; collect feedback from teachers, students, and other relevant personnel on the effectiveness of strategy execution. Feedback methods include evaluation on terminal devices and filling out questionnaires. The collection frequency is immediately after each strategy execution; In the system optimization and knowledge updating steps, the strategy execution feedback information is collated and analyzed, and valuable information is extracted to optimize the classroom incidental situation identification model, impact assessment model, and strategy generation model; based on emerging classroom incidental situation cases and effective response strategies, the system's knowledge base is updated, including the case base, strategy base, and related model parameters and rules; the system's performance is evaluated and optimized regularly, the operating status of each module of the system is checked, and the system parameters are adjusted to ensure that the system is always in an efficient and stable operating state. The performance evaluation cycle is once a week.

7. The method for analyzing and coping with incidental situations in a classroom based on a large model according to claim 6 is characterized in that: In the data collection and integration steps, the installation location and parameter settings of the data collection equipment are optimized according to the classroom layout and personnel distribution, such as adjusting the shooting angle and focal length of the camera so that the image can clearly cover all areas of the classroom; in the data cleaning process, a data consistency check algorithm is used to ensure the logical consistency between data from different sources, such as checking whether student attendance data matches seat occupancy data.

8. The method for analyzing and coping with incidental situations in a classroom based on a large model according to claim 6 is characterized in that: In the feature extraction and model adaptation steps, the feature selection algorithm is used for high-dimensional feature data to screen out feature subsets with high correlation with occasional classroom situations, reduce data dimensions, and improve model calculation efficiency; in the model fine-tuning process, transfer learning technology is used to migrate model parameters trained in other similar classroom scenarios to the current model, speeding up fine-tuning while avoiding overfitting.

9. The method for analyzing and coping with incidental situations in a classroom based on a large model according to claim 6 is characterized in that: In the situation identification and judgment steps, multimodal information fusion technology is used to comprehensively analyze multiple data modalities such as images, voice, and text to improve the accuracy of situation identification. When a new type of sporadic situation that is not recognized by the model occurs, the sample collection and labeling process is automatically started to collect and label relevant data to expand the training data set and update the model to adapt to the new situation.

10. The method for analyzing and coping with incidental situations in a classroom based on a large model according to claim 6 is characterized in that: In the impact assessment and strategy generation steps, a user participation mechanism is introduced to allow teachers and students to propose modification suggestions to the impact assessment results and generated strategies based on actual conditions, and the system makes adjustments based on feedback. In the strategy execution and feedback collection steps, a feedback information authenticity verification mechanism is established to ensure the reliability of feedback information by comparing feedback information from different sources and analyzing the rationality of feedback data, and to filter out false or invalid feedback.

Images