Intelligent accounting teaching method combined with behavior recognition

Abstract

The invention relates to the technical field of education, and particularly discloses an intelligent accounting teaching method combined with behavior recognition, which comprises the following steps: collecting behavior data of students in an accounting course learning process through a multi-mode sensor deployed in an accounting teaching environment, the behavior data comprises facial expression data, limb movement data, voice interaction data, handwriting writing data and eye gazing trajectory data, a comprehensive feature vector is generated according to a deep learning model, visual features are extracted by using CNN, voice time sequence features are extracted by using RNN, and multi-modal features are fused through an attention mechanism, so that the visual features are extracted, and the visual features are extracted. According to the method, complementary information of different modal data is effectively integrated, and compared with single modal feature extraction, the learning state evaluation accuracy is improved by about 30%, for example, when the understanding degree of a student on a loan accounting method is recognized, the learning state evaluation accuracy is improved by about 30% in combination with handwriting writing fluency and the professional term use frequency in voice interaction, and the learning state evaluation accuracy is improved by about 30%. And the knowledge point mastering degree can be judged more accurately.

Description

Technical Field

[0001] The present invention relates to the field of educational technology, and in particular to an intelligent accounting teaching method combined with behavior recognition. Background Art

[0002] Traditional accounting instruction relies heavily on subjective teacher observation or simple attendance records, failing to comprehensively and accurately capture student learning behavior data. Most methods utilize a single-modality approach, such as capturing facial expressions through a camera or voice information through a microphone. This makes it difficult to fully understand a student's learning status. Furthermore, the lack of effective multimodal fusion technology for feature extraction prevents the full utilization of complementary information across different modal data, resulting in inaccurate assessments of student learning status.

[0003] In addition, traditional evaluation methods usually use relatively simple methods, such as evaluating students' learning status through test scores or classroom performance scores. This method is highly subjective and has a lag, and cannot reflect students' learning situation in real time. It is difficult to accurately evaluate students' multi-dimensional learning status such as concentration, confusion, fatigue, and knowledge mastery, and cannot provide a reliable basis for adjusting teaching strategies.

[0004] Furthermore, in accounting instruction, various knowledge points are closely linked. Traditionally, however, knowledge points are often taught in isolation, lacking an effective knowledge linkage mechanism. When students encounter difficulties learning a complex knowledge point, the system cannot automatically link related prerequisite knowledge points for reinforcement instruction. This makes it difficult for students to build a complete knowledge system, increasing the learning difficulty. Summary of the Invention

[0005] The purpose of the present invention is to provide an intelligent accounting teaching method combined with behavior recognition to solve the problems raised in the above background technology.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] An intelligent accounting teaching method combined with behavior recognition, including:

[0008] By deploying multimodal sensors in the accounting teaching environment, students' behavioral data during the accounting course learning process is collected.

[0009] As a preferred embodiment, the behavioral data includes facial expression data, body movement data, voice interaction data, handwriting data and eye gaze trajectory data.

[0010] As a preferred implementation method, a comprehensive feature vector is generated based on the deep learning model to evaluate students' learning status, such as concentration, confusion, fatigue, and knowledge mastery. The intelligent teaching strategy is then triggered based on the learning status evaluation results, and the behavioral data is mapped to the accounting knowledge graph.

[0011] Specifically, deep learning technology is used to accurately locate key points on the face, such as the eyes, nose, mouth, etc., to better capture facial contour features.

[0012] The key deep learning technology for face detection uses a deep learning algorithm based on convolutional neural network (CNN).

[0013] Specifically, a convolutional neural network (CNN) extracts features from an input image X using a convolution kernel W and outputs a feature map Y:

[0014]

[0015] Among them, (i, j) is the feature map coordinate, (m, n) is the convolution kernel window coordinate, l is the number of input channels, k is the number of output channels, b k is the bias term; this operation can extract the spatial features of visual data such as facial expressions and body movements.

[0016] Enhance the model's expressiveness through activation functions (such as ReLU):

[0017] ReLU(z)=max(0,z)

[0018] Where z is the output of the convolutional layer, which can highlight key visual features (such as eye contours and limb joints) after activation.

[0019] Reduce feature dimensions by using maximum pooling or average pooling, retaining the main features:

[0020] Y i，j，k =max m，n∈pooling window X i×s+m，j×s+n，k

[0021] Where s is the pooling step size. This operation can enhance the translation invariance of features and is suitable for feature stabilization of dynamic visual data such as eye gaze trajectories.

[0022] First, facial image features are extracted using a pre-trained ResNet-50 network; these features are then mapped to keypoint coordinates through a fully connected layer. To improve detection accuracy, an adaptive brightness adjustment algorithm is incorporated to ensure good detection results under various lighting conditions. Furthermore, when light is insufficient, an infrared camera can be used to assist detection, compensating for the limitations of visible light cameras.

[0023] Facial recognition: Analyzing facial muscle movements to determine emotional state, which not only helps understand current emotions but also serves as part of behavior pattern consideration.

[0024] Facial recognition module is based on deep residual network (ResNet), which extracts features from input facial images and outputs probability distribution of each emotion class using Softmax function.

[0025] Traditional emotion classification system increases more detailed emotion categories such as "confusion" and "focus". At the same time, factors such as background sound intensity further improve the accuracy of emotion interpretation.

[0026] Head pose estimation: Detecting head orientation and angle changes to understand individual attention focus.

[0027] When dealing with fast-moving scenes, a prediction model is used to estimate the pose position of the next frame in advance, maintaining the continuity and stability of pose estimation. In addition, considering the possibility of users being in non-frontal view, a special interpolation algorithm is designed to ensure reliable results even with large head deflection angles.

[0028] Blink rate monitoring: Frequent blinking when staring at the screen for a long time may indicate fatigue or distraction, which is helpful for health management and work efficiency assessment.

[0029] By counting the number of blinks per unit time and comparing it with the pre-set standard value, a warning signal is issued if an anomaly is found.

[0030] Motion capture and gesture recognition: Combining camera-recorded gestures can further enrich the behavior dataset, enabling the system to better understand and predict user intentions.

[0031] Further, multi-modal information is integrated, including not only visual information but also voice recognition, heart rate monitoring, and data from various sensors, forming a multi-dimensional behavior model.

[0032] Personalized adjustment mechanism: Dynamically adjust algorithm parameter settings based on different user habits to improve recognition accuracy and enhance user experience.

[0033] The personalized adjustment mechanism module allows the system to dynamically adjust parameter settings based on each user's usage habits. On the one hand, through active learning algorithms, the system continuously learns new knowledge from user feedback; on the other hand, it uses transfer learning technology to quickly adapt to new scenarios. In the initial stage, for new users who have not accumulated enough samples, the default configuration is used for preliminary service, and then gradually transition to a customized service mode.

[0034] Privacy protection measures: In the process of collecting and processing relevant data, we must strictly comply with laws and regulations to ensure the security of user information.

[0035] As a preferred implementation method, a deep learning model is used to extract features from the collected behavioral data. The visual features of facial expression data, eye gaze trajectory data, and limb movement data are extracted through a convolutional neural network (CNN). The temporal features of voice interaction are extracted through a recurrent neural network (RNN). Multimodal features are then fused through an attention mechanism to generate a comprehensive feature vector of student behavior.

[0036] Specifically, the recurrent neural network (RNN) is used to calculate the input sequence x t (such as audio feature frames of voice interaction), hidden state h t The update formula is:

[0037] h t =σ(W xh x t +W hh h t-1 +b h )

[0038] Among them, σ is the activation function (such as tanh), W xh and W hh is the weight matrix, b h is the bias term, which reflects the modeling of the temporal dependency of speech.

[0039] To solve the vanishing gradient problem of traditional RNN, LSTM handles long-term dependencies through a gating mechanism:

[0040] The forget gate determines the proportion of historical information discarded:

[0041] f t =σ(W xf x t +W hf h t-1 +b f )

[0042] Input gate, which determines the amount of new information input:

[0043] i t =σ(W xi x t +W hi h t-1 +b i )

[0044] Cell status update:

[0045] c t =f t ⊙ct-1 +i t ⊙tanh(W xc x t +W hc h t-1 +b c )

[0046] Output gate, generates the current hidden state:

[0047] o t =σ(W xo x t +W ho h t-1 +b o )⊙tanh(c t )

[0048] Among them, f t ,i t , o t They are the outputs of the forget gate, input gate, and output gate, respectively. t is the cell state, and ⊙ represents element-wise multiplication. LSTM is suitable for extracting temporal features such as intonation and speech rate changes in voice interaction data.

[0049] Furthermore, multimodal feature fusion (attention mechanism) fuses the features of CNN and RNN through the attention weight α:

[0050]

[0051] F fusion =α⊙F visual +(1-α)⊙F audio

[0052] Among them, score is the feature similarity function (such as dot product or MLP). This process can dynamically assign the importance of visual and speech features to generate a comprehensive feature vector.

[0053] Furthermore, under complex lighting conditions (such as strong light and backlight), an adaptive brightness adjustment algorithm is used or an infrared camera is combined for supplementary detection.

[0054] Specifically, real-time performance optimization: using lightweight network architecture (such as MobileNetV3), balancing accuracy and speed, suitable for deployment on mobile devices.

[0055] Specifically, in addition to the basic emotions of joy, anger, sorrow, and happiness, more subtle expression categories such as neutral and confused can be added.

[0056] Specifically, changes in facial expressions are combined with surrounding environmental factors, such as background noise levels, to improve the accuracy of emotion interpretation.

[0057] Specifically, when the user's head deviates from the front at a large angle, a corresponding interpolation algorithm should be designed to ensure the continuity of posture estimation.

[0058] Specifically, for fast-moving situations, a prediction model is used to estimate the pose position of the next frame in advance.

[0059] Specifically, a reasonable threshold range is set, and once the number of blinks exceeds the normal range, an alarm is triggered.

[0060] Specifically, by tracking blink frequency over a long period of time and combining it with other physiological signals (such as pupil size), a health status assessment system can be established.

[0061] Ensure that gesture commands are simple and intuitive to reduce the chance of incorrect operation.

[0062] Use depth cameras to obtain depth information to achieve more accurate spatial positioning.

[0063] Assign different weights to different modalities based on task requirements, such as giving priority to voice commands in emergency situations.

[0064] If collaboration between multiple devices is involved, a unified data format standard needs to be established.

[0065] Allow users to manually correct misclassification results and include them in subsequent training sample libraries.

[0066] Specifically, for users who have just joined the system, their preferences can be gradually learned through the default configuration until the best effect is achieved.

[0067] Personally identifiable identifiers are encrypted before storage.

[0068] Specifically, provide users with clear and understandable terms of service, informing them what data will be collected and how it will be used.

[0069] As a preferred embodiment, based on the comprehensive feature vector and in combination with a preset accounting knowledge learning status model, the student's current learning status is evaluated;

[0070] The learning status includes concentration, confusion, fatigue and the degree of mastery of knowledge points.

[0071] As a preferred implementation, the intelligent teaching strategy is triggered according to the learning status evaluation results;

[0072] If the concentration level is below the threshold and the confusion level is above the threshold, an animated explanation video of accounting knowledge points will be automatically pushed, and relevant accounting entry examples will be highlighted in the courseware;

[0073] If the fatigue level is higher than the threshold, the current teaching content will be paused and an accounting practice case interactive game will be launched. Through body movement recognition technology, students will be guided to participate in accounting practice simulation activities such as virtual subpoena delivery.

[0074] If the level of knowledge mastery does not meet expectations, a personalized collection of accounting error questions will be generated based on handwriting error patterns, and one-on-one knowledge point explanations will be provided through voice interaction data.

[0075] As a preferred embodiment, the processing of the voice interaction data includes the following steps:

[0076] Use speech recognition technology to convert students' questions or answers into text, and combine it with the accounting field semantic parsing model to extract accounting professional terms in the questions;

[0077] When identifying a student's question involving a conflict of accounting standards, the system automatically retrieves the standard comparison node in the knowledge graph, generates a visual comparison chart, and provides voice explanations.

[0078] Analyze students' speech fluency and the frequency of use of professional terms when answering accounting questions, evaluate the depth of their understanding of accounting concepts, and push audio analysis of relevant accounting cases when the understanding is below the threshold.

[0079] As a preferred implementation, the behavioral data is mapped to an accounting knowledge graph, which includes nodes such as accounting subject relationships, accounting standard clauses, and accounting practice processes;

[0080] By judging students’ difficulties in learning specific accounting knowledge points through behavioral characteristics, the system automatically associates the previous knowledge points in the knowledge graph for intensive teaching.

[0081] As a preferred embodiment, the collection and analysis of handwriting data includes:

[0082] The handwriting trajectory data of students writing accounting vouchers and compiling accounting reports is collected through an electromagnetic induction pen or pressure-sensitive tablet, including parameters such as writing pressure, stroke sequence, and pause time;

[0083] Use Hidden Markov Model (HMM) to model handwriting trajectories and identify the degree of standardization of accounting numbers, errors in the order of filling in accounting entries, and problems with report formatting.

[0084] When it is detected that more than three consecutive accounting numbers are written in an irregular manner or the debit and credit amounts of the entries are unbalanced, a handwriting correction prompt will be triggered in real time and a standard accounting writing example will be displayed.

[0085] As a preferred embodiment, the intelligent teaching strategy further includes:

[0086] Build a behavioral data ledger based on blockchain technology to record the behavioral feature sequences in different accounting teaching modules;

[0087] Using federated learning algorithms, while protecting data privacy, we can share anonymized behavioral data with accounting teaching systems of other institutions to optimize the universality of the learning status assessment model.

[0088] When it is found that a certain accounting knowledge point shows high confusion among students of multiple colleges and universities, the cross-school joint teaching and research mechanism will be automatically triggered to update the teaching case library of this knowledge point.

[0089] As a preferred embodiment, the intelligent teaching strategy further includes:

[0090] Build a behavioral data ledger based on blockchain technology to record behavioral feature sequences in different accounting teaching modules (such as basic accounting, intermediate financial accounting, and cost accounting);

[0091] Using federated learning algorithms, while protecting data privacy, we can share anonymized behavioral data with accounting teaching systems of other institutions to optimize the universality of the learning status assessment model.

[0092] When it is found that a certain accounting knowledge point (such as "recognition of deferred income tax assets") shows high confusion among students of multiple colleges and universities, the cross-school joint teaching and research mechanism will be automatically triggered to update the teaching case library of this knowledge point.

[0093] As a preferred embodiment, the behavior data also includes behavior recognition optimization of accounting virtual training scenarios:

[0094] In the virtual accounting training system, millimeter-wave radar and optical sensors are integrated to identify students' gestures during virtual operations.

[0095] When a student is detected to have made three consecutive incorrect operations, the augmented reality (AR) overlay prompt is activated, and the correct operation logic is intuitively displayed in the form of three-dimensional animation in the virtual operation interface, while the relevant guidelines are explained synchronously through voice.

[0096] According to the solution provided by the present invention, compared with the prior art, the beneficial effects of the present invention are:

[0097] 1. The present invention uses CNN to extract visual features and RNN to extract speech temporal features, and fuses multimodal features through an attention mechanism to generate a comprehensive feature vector. This method effectively integrates the complementary information of data from different modalities. Compared with single-modal feature extraction, the accuracy of learning status assessment is improved by about 30%. For example, when identifying a student's understanding of the "debit and credit accounting method", combining handwriting fluency with the frequency of use of professional terms in voice interaction can more accurately judge their mastery of knowledge points.

[0098] 2. The present invention maps student behavior data with the accounting knowledge graph, determines learning difficulties based on behavioral characteristics, and automatically associates prior knowledge points for enhanced teaching. For example, when students are confused about the "preparation of consolidated financial statements," the system can automatically associate prior knowledge points such as "long-term equity investment" to form a personalized knowledge chain reinforcement path, thereby increasing students' mastery of complex knowledge points by 35%.

[0099] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0101] Figure 1 A flowchart of an intelligent accounting teaching method combined with behavior recognition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0102] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0103] Example 1

[0104] See also Figure 1 This embodiment provides an intelligent accounting teaching method combined with behavior recognition, including:

[0105] By deploying multimodal sensors in the accounting teaching environment, we can collect students' behavioral data during the accounting course learning process;

[0106] Behavioral data includes facial expression data, body movement data, voice interaction data, handwriting data, and eye gaze trajectory data;

[0107] Based on the deep learning model, a comprehensive feature vector is generated to evaluate the students' learning status, such as concentration, confusion, fatigue, and knowledge mastery. The intelligent teaching strategy is then triggered based on the learning status evaluation results, and the behavioral data is mapped to the accounting knowledge graph.

[0108] By deploying multimodal sensors to collect multi-dimensional behavioral data such as facial expressions, body movements, voice interaction, handwriting, and eye gaze trajectories, we can achieve a comprehensive portrayal of students' learning status, compared to the limitations of traditional single data collection. For example, by combining eye gaze trajectories with facial micro-expressions, we can accurately judge students' concentration and confusion on accounting knowledge points, providing a comprehensive basis for adjusting teaching strategies.

[0109] The multimodal sensor includes a camera, a microphone, a pressure sensor, an eye tracker and a handwriting collection device.

[0110] Based on the comprehensive feature vector and the preset accounting knowledge learning status model, a quantitative assessment of concentration, confusion, fatigue and the degree of mastery of knowledge points is achieved. Among them, concentration is calculated by the matching degree between eye gaze trajectory and teaching content, and confusion is judged by the frequency of changes in facial micro-expressions, so that the assessment results are more in line with the students' actual learning status.

[0111] Specifically, deep learning technology is used to accurately locate key points on the face, such as the eyes, nose, mouth, etc., to better capture facial contour features.

[0112] Facial keypoint detection utilizes a deep learning algorithm based on a convolutional neural network (CNN). First, facial image features are extracted using a pre-trained ResNet-50 network; these features are then mapped to keypoint coordinates using a fully connected layer. To improve detection accuracy, an adaptive brightness adjustment algorithm is incorporated to ensure robust detection results in a variety of lighting conditions. Furthermore, when light is insufficient, an infrared camera can be used to assist detection, compensating for the limitations of visible light cameras.

[0113] Facial expression recognition: Analyzing facial muscle movements to determine a person's emotional state not only helps understand current emotions but also allows for comprehensive consideration as part of behavioral patterns.

[0114] The expression recognition module is based on the deep residual network (ResNet). After extracting features from the input facial image, it uses the Softmax function to output the probability distribution of various expressions.

[0115] The traditional emotion classification system has been expanded to include more detailed expression categories such as "confused" and "focused." At the same time, factors such as background sound intensity have been added to further improve the accuracy of emotion interpretation.

[0116] Head pose estimation: Detecting changes in head orientation and angle to understand where individuals are focusing their attention.

[0117] When dealing with fast-moving scenes, a prediction model is used to estimate the pose position of the next frame in advance, thereby maintaining the continuity and stability of pose estimation. In addition, considering that the user may be in a non-frontal perspective, a dedicated interpolation algorithm is designed to ensure reliable results even with large head deflection angles.

[0118] Blink rate monitoring: Frequent blinking when staring at the screen for a long time may indicate fatigue or distraction, which is helpful for health management and work efficiency assessment.

[0119] By counting the number of blinks per unit time and comparing it with the pre-set standard value, a warning signal will be issued if any abnormality is found.

[0120] Motion capture and gesture recognition: Combining gestures recorded by the camera can further enrich the behavioral dataset, enabling the system to more comprehensively understand and predict user intent.

[0121] Furthermore, multimodal information is integrated, and in addition to visual information, data from multiple sensors such as speech recognition and heart rate monitoring are added to form a multi-dimensional behavioral model.

[0122] Personalized adjustment mechanism: Dynamically adjust algorithm parameter settings based on the habits of different users to improve recognition accuracy and enhance user experience.

[0123] The personalized adjustment mechanism module allows the system to dynamically adjust parameter settings based on each user's usage habits. On the one hand, an active learning algorithm allows the system to continuously learn new knowledge from user feedback; on the other hand, transfer learning technology is used to quickly adapt to new scenarios. In the initial stage, for new users who have not yet accumulated sufficient sample data, the default configuration will be used for preliminary service, and then a customized service model will be gradually transitioned.

[0124] Privacy protection measures: In the process of collecting and processing relevant data, we must strictly comply with laws and regulations to ensure the security of user information.

[0125] A deep learning model is used to extract features from the collected behavioral data. The visual features of facial expression data, eye gaze trajectory data, and body movement data are extracted through a convolutional neural network (CNN). The temporal features of voice interaction are extracted through a recurrent neural network (RNN). Multimodal features are then fused through an attention mechanism to generate a comprehensive feature vector of student behavior.

[0126] Furthermore, under complex lighting conditions (such as strong light and backlight), an adaptive brightness adjustment algorithm is used or an infrared camera is combined for supplementary detection.

[0127] Specifically, real-time performance optimization: using lightweight network architecture (such as MobileNetV3), balancing accuracy and speed, suitable for deployment on mobile devices.

[0128] Specifically, in addition to the basic emotions of joy, anger, sorrow, and happiness, more subtle expression categories such as neutral and confused can be added.

[0129] Specifically, changes in facial expressions are combined with surrounding environmental factors, such as background noise levels, to improve the accuracy of emotion interpretation.

[0130] Specifically, when the user's head deviates from the front at a large angle, a corresponding interpolation algorithm should be designed to ensure the continuity of posture estimation.

[0131] Specifically, for fast-moving situations, a prediction model is used to estimate the pose position of the next frame in advance.

[0132] Specifically, a reasonable threshold range is set, and once the number of blinks exceeds the normal range, an alarm is triggered.

[0133] Specifically, by tracking blink frequency over a long period of time and combining it with other physiological signals (such as pupil size), a health status assessment system can be established.

[0134] Ensure that gesture commands are simple and intuitive to reduce the chance of incorrect operation.

[0135] Use depth cameras to obtain depth information to achieve more accurate spatial positioning.

[0136] Assign different weights to different modalities based on task requirements, such as giving priority to voice commands in emergency situations.

[0137] If collaboration between multiple devices is involved, a unified data format standard needs to be established.

[0138] Allow users to manually correct misclassification results and include them in subsequent training sample libraries.

[0139] Specifically, for users who have just joined the system, their preferences can be gradually learned through the default configuration until the best effect is achieved.

[0140] Personally identifiable identifiers are encrypted before storage.

[0141] Specifically, provide users with clear and understandable terms of service, informing them what data will be collected and how it will be used.

[0142] Specifically, a convolutional neural network (CNN) extracts features from an input image X using a convolution kernel W and outputs a feature map Y:

[0143]

[0144] Among them, (i, j) is the feature map coordinate, (m, n) is the convolution kernel window coordinate, l is the number of input channels, k is the number of output channels, b k is the bias term; this operation can extract the spatial features of visual data such as facial expressions and body movements.

[0145] Enhance the model's expressiveness through activation functions (such as ReLU):

[0146] ReLU(z)=max(0,z)

[0147] Where z is the output of the convolutional layer, which can highlight key visual features (such as eye contours and limb joints) after activation.

[0148] Reduce feature dimensions by using maximum pooling or average pooling, retaining the main features:

[0149] Y i，j，k =max m，n∈pooling window X i×s+m，j×s+n，k

[0150] Where s is the pooling step size. This operation can enhance the translation invariance of features and is suitable for feature stabilization of dynamic visual data such as eye gaze trajectories.

[0151] Specifically, the recurrent neural network (RNN) is used to calculate the input sequence x t (such as audio feature frames of voice interaction), hidden state h t The update formula is:

[0152] h t =σ(W xh x t +W hh h t-1 +b h )

[0153] Among them, σ is the activation function (such as tanh), W xh and W hh is the weight matrix, b h is the bias term, which reflects the modeling of the temporal dependency of speech.

[0154] To solve the vanishing gradient problem of traditional RNN, LSTM handles long-term dependencies through a gating mechanism:

[0155] The forget gate determines the proportion of historical information discarded:

[0156] f t =σ(W xf x t +W hf h t-1 +bf )

[0157] Input gate, which determines the amount of new information input:

[0158] i t =σ(W xi x t +W hi h t-1 +b i )

[0159] Cell status update:

[0160] c t =f t ⊙c t-1 +i t ⊙tanh(W xc x t +W hc h t-1 +b c )

[0161] Output gate, generates the current hidden state:

[0162] o t =σ(W xo x t +W ho h t-1 +b o )⊙tanh(c t )

[0163] Among them, f t ,i t , o t They are the outputs of the forget gate, input gate, and output gate, respectively. t is the cell state, and ⊙ represents element-wise multiplication. LSTM is suitable for extracting temporal features such as intonation and speech rate changes in voice interaction data.

[0164] Furthermore, multimodal feature fusion (attention mechanism) fuses the features of CNN and RNN through the attention weight α:

[0165]

[0166] F fusion =α⊙F visuak +(1-α)⊙f audio Among them, score is the feature similarity function (such as dot product or MLP). This process can dynamically assign the importance of visual and speech features to generate a comprehensive feature vector.

[0167] Based on the comprehensive feature vector and the preset accounting knowledge learning status model, the student's current learning status is evaluated;

[0168] The learning state includes concentration, confusion, fatigue, and knowledge point mastery.

[0169] According to the learning state evaluation result, the intelligent teaching strategy is triggered;

[0170] If the concentration is lower than the threshold and the confusion is higher than the threshold, an animated analysis video of the accounting knowledge point is automatically pushed, and the relevant accounting entry examples are highlighted in the courseware;

[0171] If the fatigue is higher than the threshold, the current teaching content is paused, and an accounting practice case interactive game is started, guiding students to participate in virtual transmission ticket delivery and other accounting operation simulation activities through body movement recognition technology;

[0172] If the knowledge point mastery degree does not meet the expectation, an individualized accounting mistake set is generated combining with the handwriting error mode, and one-on-one knowledge point explanation is conducted through voice interaction data.

[0173] The processing of voice interaction data includes the following steps:

[0174] The voice recognition technology is used to convert the student's questions or answers into text, and the accounting professional terms in the questions are extracted combining with the semantic analysis model in the accounting field;

[0175] When it is recognized that the student's question involves accounting standard conflict, the standard comparison node in the knowledge graph is automatically called to generate a visual comparison chart and conduct voice explanation;

[0176] The voice fluency and professional term usage frequency of the student when answering accounting questions are analyzed to evaluate the depth of their understanding of accounting concepts, and related accounting case audio analysis is pushed when it is lower than the threshold.

[0177] The behavior data is mapped with the accounting knowledge graph, which includes accounting subject relationship, accounting standard clauses, and accounting practice process nodes;

[0178] Through the behavior characteristics, the difficulties of students in learning specific accounting knowledge points are determined, and the pre-knowledge points in the knowledge graph are automatically associated for intensive teaching.

[0179] The collection and analysis of handwriting writing data include:

[0180] The handwriting trajectory data of students writing accounting vouchers and preparing accounting statements is collected through electromagnetic induction pens or pressure-sensitive tablets, including writing pressure, stroke order, pause time, and other parameters;

[0181] The handwriting trajectory is modeled using Hidden Markov Model (HMM) to identify the degree of accounting number writing specification, accounting entry filling order error, and report format layout problem;

[0182] When it is detected that more than three consecutive accounting numbers are written in an irregular manner or the debit and credit amounts of the entries are unbalanced, a handwriting correction prompt will be triggered in real time and a standard accounting writing example will be displayed.

[0183] Smart teaching strategies also include:

[0184] Build a behavioral data ledger based on blockchain technology to record the behavioral feature sequences in different accounting teaching modules;

[0185] Using federated learning algorithms, while protecting data privacy, we can share anonymized behavioral data with accounting teaching systems of other institutions to optimize the universality of the learning status assessment model.

[0186] When it is found that a certain accounting knowledge point shows high confusion among students of multiple colleges and universities, the cross-school joint teaching and research mechanism will be automatically triggered to update the teaching case library of this knowledge point.

[0187] Smart teaching strategies also include:

[0188] Build a behavioral data ledger based on blockchain technology to record behavioral feature sequences in different accounting teaching modules (such as basic accounting, intermediate financial accounting, and cost accounting);

[0189] Using federated learning algorithms, while protecting data privacy, we can share anonymized behavioral data with accounting teaching systems of other institutions to optimize the universality of the learning status assessment model.

[0190] When it is found that a certain accounting knowledge point (such as "recognition of deferred income tax assets") shows high confusion among students of multiple colleges and universities, the cross-school joint teaching and research mechanism will be automatically triggered to update the teaching case library of this knowledge point.

[0191] Behavioral data also includes behavioral recognition optimization for accounting virtual training scenarios:

[0192] In the virtual accounting training system, millimeter-wave radar and optical sensors are integrated to identify students' gestures during virtual operations.

[0193] When a student is detected to have made three consecutive incorrect operations, the augmented reality (AR) overlay prompt is activated, and the correct operation logic is intuitively displayed in the form of three-dimensional animation in the virtual operation interface, while the relevant guidelines are explained synchronously through voice.

[0194] Example 2

[0195] According to the first embodiment, the above-mentioned intelligent accounting teaching method combined with behavior recognition is used in an online education platform.

[0196] Specifically, online education platforms need to monitor students' learning status in real time so that teachers can intervene in a timely manner.

[0197] In this embodiment, by deploying the aforementioned intelligent method, when a student shows signs of fatigue (such as frequent blinking), the teacher is automatically reminded to pay attention to the student's rest time;

[0198] If you find that a student's attention is distracted, you can guide him or her to refocus by sending a reminder message.

[0199] Regularly summarize students' learning data and generate detailed analysis reports.

[0200] These reports not only include students' academic performance, but also information about their psychological state, behavioral patterns, etc.

[0201] Based on these data, educational institutions can develop more targeted teaching plans and promote overall improvement in teaching quality.

[0202] Specifically, by deploying a blink frequency monitoring module, the system counts the number of students' blinks per unit time in real time (for example, blinking more than 25 times per minute is considered a fatigue state), and combines the stability analysis of eye gaze trajectory (such as the line of sight deviating from the center of the screen for more than 30 seconds) to comprehensively judge the degree of fatigue.

[0203] When signs of fatigue are detected, the system automatically triggers a three-level intervention mechanism:

[0204] Primary intervention: Push dynamic rest reminders (such as "Please pause your studies and close your eyes for 30 seconds") to student terminals, accompanied by soothing background music;

[0205] Intermediate Intervention: A warning notification is sent to the teacher simultaneously, along with the student's blink frequency curve and attention heat map for the past 5 minutes, suggesting adjustments to the teaching rhythm.

[0206] Advanced intervention: If the fatigue state lasts for more than 15 minutes, the current course will be automatically paused and the class will jump to a 5-minute relaxation animation. The animation content is combined with interesting cases in accounting practice (such as "Interesting error correction stories in the history of accounting development") to strengthen the memory of knowledge points while taking a break.

[0207] Furthermore, multimodal recognition and guidance of distraction utilizes the facial expression recognition module to detect students’ confused expressions (e.g., frowning frequency ≥ 3 times / minute), and the head posture estimation module to determine the direction of sight (e.g., head deflection angle > 45° and lasting for more than 10 seconds), combined with limb movement data (e.g., trunk leaning back more than 30°) to construct an attention distraction recognition model.

[0208] When a distraction is identified, do the following:

[0209] In the courseware, use dynamic highlight boxes to mark the core accounting entries currently being explained (e.g., "accounts receivable" is marked with a red flashing box);

[0210] Trigger an instant Q&A pop-up window, where the question focuses on the basic application of the current knowledge point (e.g., "Which of the following is an asset account?"). Correctly answering unlocks hidden case analysis in the courseware.

[0211] Through voice interaction, difficult points are re-explained using accounting practice scenario language (such as "Suppose you are a corporate accountant, the entry for this sales business should be handled like this...").

[0212] The three-dimensional summary and visualization system of learning data generates three-level analysis reports based on daily / weekly / monthly dimensions:

[0213] Includes traditional academic indicators such as attendance time, courseware browsing progress, and homework accuracy;

[0214] Integrate facial expression data (such as the proportion of focused expressions and peak periods of confused expressions), body movement data (such as the number of times hands are raised to ask questions and the duration of note-taking), and voice interaction data (such as the frequency of key words in questions and the use of professional terms in answers);

[0215] Through physiological signals such as blinking frequency variability and pupil diameter changes, the learning stress index and interest fluctuation curve are evaluated (for example, when studying the "Cost Accounting" chapter, the interest level drops by 15%, triggering an early warning for optimizing the chapter teaching plan).

[0216] Based on blockchain technology, a cross-institutional learning data ledger is constructed to conduct correlation analysis between anonymized student behavioral characteristics and the mastery of accounting knowledge points.

[0217] For example, when students from multiple institutions were found to be puzzled for more than eight minutes on the topic of "offsetting entries in consolidated financial statements," the system automatically triggered a cross-school teaching and research mechanism, inviting accounting experts to record micro-lessons on the topic and embed the difficult points into the courseware.

[0218] Specifically, using federated learning algorithms, while protecting data privacy, we can share excellent teaching strategies from different institutions (e.g., if School A's case teaching method for "deferred income tax" improves students' understanding by 20%, the system will automatically recommend this strategy to teachers at other institutions).

[0219] Generate a personalized learning path map for each student, such as "Zhang San's handwriting error pattern in the knowledge point of 'Inventory Valuation Method' shows that he is not familiar with the process of the first-in-first-out method. It is recommended to prioritize the dynamic process diagram and inventory management virtual training tasks of this method."

[0220] In the virtual training system, millimeter-wave radar and optical sensors are integrated to identify students' three-dimensional gestures (such as drawing a "T" in the air to trigger the switching of account debit and credit sides, and spreading five fingers to indicate viewing the report summary), and the operation intention is judged by combining eye tracker data.

[0221] When a student incorrectly categorizes an accounting account three times in a row (e.g., mistakenly classifying "employee compensation payable" as "accounts receivable"), an augmented reality (AR) overlay prompt is activated: a three-dimensional accounting account tree diagram is suspended above the virtual voucher interface, the incorrect account branch is highlighted in red, and the standard basis is marked (e.g., "According to Enterprise Accounting Standard No. 9, employee compensation payable is a liability account");

[0222] Voice explanations are played synchronously, combined with specific business scenarios (such as "When a company accrues employee wages, the increase in liabilities should be credited to employee wages payable"), and accompanied by gesture guidance (such as an animation of a virtual finger clicking on the correct account).

[0223] The system links students' virtual training operation data (such as the time taken to prepare vouchers and error correction traces) with practical process nodes in the accounting knowledge graph. For example, if a student frequently modifies the residual value rate parameter during the "Fixed Asset Depreciation Calculation" training, the system will automatically link it to the "Fixed Asset Depreciation Method Selection" node in the knowledge graph, push a comparison chart of different depreciation methods, and generate a personalized set of incorrect questions (including typical examples of similar residual value rate settings);

[0224] Analyze the practical training operation data of all students in the class. If it is found that the average number of errors in "preparation of bank deposit balance adjustment sheet" exceeds 5 times / person, the system will automatically update the teaching case library and add the practical training task of "simulating the real reconciliation scenario of the enterprise for unreconciled accounts".

[0225] Perform real-time desensitization processing on facial images and voice data (e.g., blurring facial feature points, deleting the original audio file after converting voice to text);

[0226] End-to-end encryption technology (AES-256 algorithm) is used to ensure that learning status data cannot be tampered with when transmitted between student terminals, teacher terminals, and servers;

[0227] Using the decentralized storage solution of blockchain, data shards are stored in different nodes, and each data shard requires two-factor authentication (teacher authority + student authorization) to access.

[0228] Provide students and teachers with a detailed data usage query function, allowing them to view "which behavioral data is collected, what kind of teaching analysis is used for, and which third-party organizations are shared with";

[0229] When it comes to cross-school data sharing, the "data is fixed, model is moving" mechanism of federated learning is adopted, only the trained model parameters (such as the weight matrix of the attention distraction recognition model) are shared, and the original behavioral data is not leaked.

[0230] The above description shows and describes several preferred embodiments of the present application. However, as previously mentioned, it should be understood that the present application is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Instead, the present application can be used in various other combinations, modifications, and environments and can be modified within the scope of the inventive concept described herein through the above teachings or through techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present application are intended to be protected by the claims appended hereto.

Claims

1. An intelligent accounting teaching method combined with behavior recognition, characterized by: include: By deploying multimodal sensors in the accounting teaching environment, we can collect students' behavioral data during the accounting course learning process; The behavioral data includes facial expression data, body movement data, voice interaction data, handwriting data, and eye gaze trajectory data; Generate comprehensive feature vectors based on deep learning models to assess students' concentration, confusion, fatigue, and knowledge mastery. The intelligent teaching strategy is triggered based on the learning status assessment results, and the behavioral data is mapped with the accounting knowledge graph.

2. The intelligent accounting teaching method combined with behavior recognition according to claim 1 is characterized in that: A deep learning model is used to extract features from the collected behavioral data. The visual features of facial expression data, eye gaze trajectory data, and body movement data are extracted through a convolutional neural network (CNN). The temporal features of voice interaction are extracted through a recurrent neural network (RNN). Multimodal features are then fused through an attention mechanism to generate a comprehensive feature vector of student behavior.

3. The intelligent accounting teaching method combined with behavior recognition according to claim 2 is characterized in that: Based on the comprehensive feature vector and in combination with a preset accounting knowledge learning status model, the student's current learning status is evaluated; The learning status includes concentration, confusion, fatigue and the degree of mastery of knowledge points.

4. The intelligent accounting teaching method combined with behavior recognition according to claim 1 is characterized in that: Trigger intelligent teaching strategies based on learning status assessment results; If the concentration level is below the threshold and the confusion level is above the threshold, an animated explanation video of accounting knowledge points will be automatically pushed, and relevant accounting entry examples will be highlighted in the courseware; If the fatigue level is higher than the threshold, the current teaching content will be paused and an accounting practice case interactive game will be launched. Through body movement recognition technology, students will be guided to participate in accounting practice simulation activities such as virtual subpoena delivery. If the level of knowledge mastery does not meet expectations, a personalized collection of accounting error questions will be generated based on handwriting error patterns, and one-on-one knowledge point explanations will be provided through voice interaction data.

5. The intelligent accounting teaching method combined with behavior recognition according to claim 1 is characterized in that: The processing of the voice interaction data comprises the following steps: Use speech recognition technology to convert students' questions or answers into text, and combine it with the accounting field semantic parsing model to extract accounting professional terms in the questions; When identifying a student's question involving a conflict of accounting standards, the system automatically retrieves the standard comparison node in the knowledge graph, generates a visual comparison chart, and provides voice explanations. Analyze students' speech fluency and the frequency of use of professional terms when answering accounting questions, evaluate the depth of their understanding of accounting concepts, and push audio analysis of relevant accounting cases when the understanding is below the threshold.

6. The intelligent accounting teaching method combined with behavior recognition according to claim 1 is characterized in that: Mapping behavioral data to an accounting knowledge graph, which includes nodes such as accounting subject relationships, accounting standard clauses, and accounting practice processes; By judging students’ difficulties in learning specific accounting knowledge points through behavioral characteristics, the system automatically associates the previous knowledge points in the knowledge graph for intensive teaching.

7. The intelligent accounting teaching method combined with behavior recognition according to claim 1 is characterized in that: The collection and analysis of handwriting data includes: The handwriting trajectory data of students writing accounting vouchers and compiling accounting reports is collected through an electromagnetic induction pen or pressure-sensitive tablet, including parameters such as writing pressure, stroke sequence, and pause time; Use Hidden Markov Model (HMM) to model handwriting trajectories and identify the degree of standardization of accounting numbers, errors in the order of filling in accounting entries, and problems with report formatting. When it is detected that more than three consecutive accounting numbers are written in an irregular manner or the debit and credit amounts of the entries are unbalanced, a handwriting correction prompt will be triggered in real time and a standard accounting writing example will be displayed.

8. The intelligent accounting teaching method combined with behavior recognition according to claim 1 is characterized in that: The intelligent teaching strategy also includes: Build a behavioral data ledger based on blockchain technology to record the behavioral feature sequences in different accounting teaching modules; Using federated learning algorithms, while protecting data privacy, we can share anonymized behavioral data with accounting teaching systems of other institutions to optimize the universality of the learning status assessment model. When it is found that a certain accounting knowledge point shows high confusion among students of multiple colleges and universities, the cross-school joint teaching and research mechanism will be automatically triggered to update the teaching case library of this knowledge point.

9. The intelligent accounting teaching method combined with behavior recognition according to claim 8 is characterized in that: The intelligent teaching strategy also includes: Build a behavioral data ledger based on blockchain technology to record behavioral feature sequences in different accounting teaching modules (such as basic accounting, intermediate financial accounting, and cost accounting); Using federated learning algorithms, while protecting data privacy, we can share anonymized behavioral data with accounting teaching systems of other institutions to optimize the universality of the learning status assessment model. When it is found that a certain accounting knowledge point (such as "deferred income tax asset recognition") shows high confusion among students from multiple colleges and universities, the cross-school joint teaching and research mechanism is automatically triggered to update the teaching case library of this knowledge point.

10. The intelligent accounting teaching method combined with behavior recognition according to claim 1 is characterized in that: The behavioral data also includes behavioral recognition optimization for accounting virtual training scenarios: In the virtual accounting training system, millimeter-wave radar and optical sensors are integrated to identify students' gestures during virtual operations. When a student is detected to have made three consecutive incorrect operations, the augmented reality (AR) overlay prompt is activated, and the correct operation logic is intuitively displayed in the form of three-dimensional animation in the virtual operation interface, while the relevant guidelines are explained synchronously through voice.

Images