Teacher anxiety understanding reasoning method based on brain vision multi-mode large model representation

Abstract

The invention provides a teacher anxiety underlying reasoning method based on brain vision multi-modal large model characterization, and relates to the field of sentiment analysis, an anxiety association semantic analysis mechanism is established, and the analysis mechanism establishes an instance set according to teacher anxiety association element data under teacher brain region influence activities; processing the brain region level data to obtain a teacher anxiety state category; processing the brain region information to obtain brain vision multi-modal time sequence data representation oriented to teacher anxiety; the anxiety state of the teacher is obtained through graph convolution pyramid operation; on the basis of a long-short time multi-scale time sequence mechanism, combining a teacher anxiety causal attention calculation mechanism to quantify factors influencing the anxiety state of the teacher; and determining the influence degree of the teacher anxiety state factors on the teacher anxiety state by adopting a double-layer Bayesian network. According to the teacher anxiety understanding reasoning method based on brain vision multi-mode large model representation, scientific basis and powerful support are provided for improving teaching quality and promoting psychological health of teachers.

Description

Technical Field

[0001] The present invention relates to the field of sentiment analysis, and in particular to an embodied reasoning method for teacher anxiety based on representation of a large brain-vision multimodal model. Background Art

[0002] With the development of brain science and visual cognitive technology, multimodal learning has become a research hotspot. Brain visual multimodal technology combines the latest research results of brain science, psychology and computer science, aiming to capture and analyze human psychological states and behavioral responses in a variety of ways. By using functional magnetic resonance imaging (fMRI), electroencephalogram (EEG) and other neuroimaging techniques, this method can obtain the activity patterns of the brain in a specific situation in real time, and realize in-depth analysis of the individual's emotional state. At the same time, combining visual perception and cognitive models can provide a more comprehensive understanding of human emotional responses and cognitive processes. Brain visual multimodal technology shows broad application prospects in the fields of educational psychology, mental health assessment, etc. Teachers' visual information processing in the classroom is often accompanied by emotional regulation and cognitive decision-making. Establishing a large model based on brain visual multimodality can more accurately characterize teachers' anxiety state.

[0003] In this context, the combination of brain science and artificial intelligence technology provides a new perspective for the study of teacher anxiety. The application of a large multimodal brain visual model can reveal the psychological and physiological states of teachers in stressful situations and their temporal changes through multi-level brain area activities and their associations with teacher anxiety. Therefore, by establishing a multi-level anxiety association semantic description of "teacher-brain area-activity-anxiety", the anxiety manifestations of teachers in different situations and the brain mechanisms behind them are captured, and the multi-head self-attention mechanism is used to extract the brain response patterns under the guidance of teacher anxiety to obtain the evolution characteristics of teacher anxiety in time series. The teacher anxiety state analysis method based on the spatiotemporal graph convolution pyramid reveals the changing trend of teacher anxiety state. Through causal attention calculation supported by long-short-term multi-scale time series, combined with double-layer Bayesian network technology, embodied reasoning of teacher anxiety state is realized, which provides a scientific basis and strong support for improving teaching quality and promoting teacher mental health. Summary of the invention

[0004] The purpose of the present invention is to provide an embodied reasoning method for teacher anxiety based on the representation of a large brain visual multimodal model, capture the anxiety manifestations of teachers in different situations and the brain mechanisms behind them, and use a multi-head self-attention mechanism to extract the brain response patterns guided by teacher anxiety, and obtain the evolution characteristics of teacher anxiety in time series. A teacher anxiety state analysis method based on a spatiotemporal graph convolutional pyramid reveals the changing trend of teacher anxiety states. Through causal attention calculation supported by long-short-time multi-scale time series, combined with double-layer Bayesian network technology, embodied reasoning of teacher anxiety states is achieved, providing a scientific basis and strong support for improving teaching quality and promoting teacher mental health.

[0005] To achieve the above object, the present invention provides a teacher anxiety embodied reasoning method based on brain-visual multimodal large model representation, comprising the following steps: Establish a semantic analysis mechanism for anxiety association at four levels of "teacher-brain region-activity-anxiety". This analysis mechanism establishes a set of relevant influencing factor instances based on the teacher anxiety-related factor data under the teacher brain region influencing activities; the set of relevant influencing factor instances includes data at the teacher level, brain region level, and activity level; By processing the data at the brain region level, the categories, polarity and degree of teachers' anxiety states were obtained; The data at the brain region level is processed to obtain brain visual multimodal time series data representation for teacher anxiety; Based on the video data at the teacher level, brain region level, and activity level, the teacher's anxiety state is obtained using graph convolution pyramid operations; Based on the long-short time multi-scale temporal mechanism, the causal attention calculation mechanism of teacher anxiety is combined to quantify the factors affecting the teacher's anxiety state; Using data at the teacher level, brain region level, and activity level, a two-layer Bayesian network was used to embodiedly reason the causal relationship between the teacher's anxiety state and the factors affecting the teacher's anxiety state. The joint tree reasoning method was used to determine the degree of influence of the factors affecting the teacher's anxiety state on the teacher's anxiety state by calculating the polarity and degree of the teacher's anxiety state and the influencing factors.

[0006] The preferred four-level semantic analysis mechanism of anxiety association of "teacher-brain region-activity-anxiety" includes Teacher level teacher, attributes.collect{E,C,M}(1); Brain area level brain(visual, text),attributes.influence{E,C,M}(2); Activity level action(E,A,M,activities,t),egread:{EA,CA,MA}(3); Anxiety level (4); Among them, teacher represents teacher, attributes.collect represents attribute collection, attributes.influence represents influence, action represents activity, egread represents obtaining teacher-related activities, brain represents brain area level, visual represents visual features, text represents text features, E represents the set of instances of teacher environmental factors, A represents the generated activities, M represents the set of instances of teacher individual factors, C represents the set of instances of course factors, activities represents activities, t represents time, Indicates teacher anxiety status; The set of relevant influencing factor instances includes Teacher-level data includes the set of teacher environment factor instances E={ }, course factor instance set C = { }, the set of teacher individual factor instances M = { }; The brain region level data includes the brain region instance set B = { }; Activity level data includes teacher activity instance set D = { }.

[0007] Preferably, the processing of the brain region level data includes two processings: the first processing is to process the brain region level data in the time domain, space domain and frequency domain, and after further analysis, the power spectrum density of different frequency bands is obtained; the second processing is to process the power spectrum density by a multi-head self-attention mechanism, and different attention values ​​are used to distinguish the importance of the teacher's psychological emotions, behavioral performance and achievement level in teacher anxiety, so as to calculate and output different categories of teacher anxiety; In the time domain, short-time Fourier transform is used to analyze the instantaneous frequency characteristics of the signal. The specific implementation method is shown in formula (5): (5); In the formula, represents the instantaneous frequency of the signal, S ( t ) indicates the time t signal, w ( t ) indicates the use of window function, represents the integral budget, Represents infinity, S (\tau ) represents a non-stationary signal, w ( \tau - t ) represents the window function, Indicates the spectrum value; In the spatial domain, the EEG-based spatial filtering technology is used to perform independent component analysis. The specific implementation method is shown in formula (6): (6); In the formula, represents the observation data matrix, m Indicates the number of sensors, n Indicates the number of time frames, is the spatial filter matrix, Y represents the extracted independent components; In frequency domain analysis, the time domain signal is converted into the frequency domain signal through fast Fourier transform. S ( t ) spectrum The specific calculation method is shown in formula (7): (7); After further analysis, the power spectral density of different frequency bands is obtained. The specific implementation method is shown in formula (8): (8); In the formula, It means to find the limit, Indicates that the length of the time interval tends to infinity. frac { 1}{ T}express The decimal part of e represents a natural constant; Convert brain region level data into input sequence X=( , ···, ), the brain response data is analyzed based on multi-head self-attention. The specific method is shown in formula (9) (9).

[0008] In the formula, Indicates a query, K Indicates the key, V Represents a value.

[0009] Preferably, the data at the brain region level are subjected to latent coding and mutual information regularization processing, matched to the multi-space and multi-link dimension framework preset at the activity level according to the mutual information regularization, and then normalized to generate brain visual multimodal time series data representation for teacher anxiety; The specific calculation method of latent coding is shown in formula (10): (10); In the formula, Is The approximate posterior distribution represented by the parameterized encoder network, Is σ Parameterized decoder network , DKL It means KL Divergence, ELBO represents the variational lower bound, represents the decoder network parameterization result, represents the potential coding value; Mutual information regularization is shown in formula (11): (11); In the formula, Through the depth of the realization of some auxiliary distribution of neural networks by Parameterization, is a latent variable, The corresponding is the teacher activity instance set D , P The corresponding distribution probability is corresponds to the uncertainty in z, It represents the degree of regular fitting. Indicates that in the known Based on the uncertainty of z, It means that under given conditions z, the random variable The conditional probability distribution of It means that under the condition of given input R, the data represents the conditional probability distribution of z, represents the point mutual information between two data, Represents the normalized value between two data.

[0010] Preferably, the teacher's anxiety state is obtained based on the video data at the teacher level, brain area level, and activity level using graph convolution pyramid operations, including The obtained video data at the teacher level, brain area level, and activity level are hierarchically pooled using a pyramid hierarchical structure. Then, global average pooling is applied to the output of the last three layers, and the spatial features are reduced by a compression function. Finally, the channel dimensions are concatenated in the connection layer to output the action sequence. The action sequence is divided into segments in the time dimension, and each segment is used as an independent window of the evolution of the teacher's anxiety state; a time pooling operation is designed to generate a time feature set T; Divide the time feature set in the time dimension to obtain the short-time feature vector, and obtain the covariance matrix of the short-time feature vector, then we have 、 、 , the dimensions are [7×( S + j + h )] ×[7×( S + j + h )]; The third layer divides all frames T into 7 segments, so there are 7 covariance matrices 、 ··· , the dimensions are also [7×( S + j + h )] ×[7×( S + j + h )], and finally the covariances obtained at all levels are spliced ​​together to obtain the time series. The specific calculation method is shown in formula (12): ； ； ； (12); In the formula, L is the number of training samples, The corresponding penalty coefficient is is the classification error score, N is the number of action sequence frames, The corresponding label is the sample label, so as to obtain the covariance weight matrix. S represents the total number of short-time eigenvectors, j represents the weighted sum of short-time eigenvectors, h represents the time pyramid level, and They represent the weighted objective function constructed using the bilinear classifier, represents a constant, ( i ) represents an iterated function.

[0011] Preferably, the short-term feature vector is used to establish a long-short multi-scale temporal mechanism and a causal attention mechanism for teacher anxiety, and the factors affecting the teacher's anxiety state are identified and quantified. The factors affecting the teacher's anxiety state include teaching level pressure, colleague competition, student feedback, social relationships and emotional pressure; Long- and short-term multi-scale temporal mechanisms include For all short-time eigenvectors Perform weighted summation to finally obtain the long-term eigenvector The specific calculation method is shown in formula (13): (13); In the formula, h represents the time pyramid level, represents the short-time eigenvector The space you are in; The causal attention mechanism of teacher anxiety includes The short-term feature vector at the current moment With the long-term eigenvector Form multiple sets of temporary feature vectors OL as input for causal attention calculation ; Multiply the temporary eigenvector OL with the V matrix through matrix operation to output the eigenvector at this moment The specific calculation method is shown in formula (14), and it is sent to the stacking calculation to obtain the calculated eigenvector matrix The specific calculation method is shown in formula (15): (14); (15); In the formula, Softmax is the normalized exponential function, Expressed as matrix linear transformation weights, It represents the similarity value of the feature vector at that moment. Represents the value obtained by multiplying this set of weights by the corresponding matrix, Indicates the moment; Output causal feature vector matrix through feed-forward residual network operation , the specific calculation method is shown in formula (16), which measures the temporal causal characteristic vector matrix of the final output ( , ) (16); In the formula, ( ) represents a series of calculated eigenvector matrices.

[0012] Preferably, the data at the teacher level, brain region level, and activity level are processed into visual features and text features, including The data at the teacher level, brain region level, activity level and the brain visual multimodal time series data of teacher anxiety include text data and image data, among which the text data is mapped to the high-dimensional embedding space to generate token embedding to obtain text features; the image data is efficiently encoded using the Vision Transformer model, and the visual features of the image are seamlessly integrated into the brain visual embedding space to obtain visual features; Map text data to a high-dimensional embedding space to generate token embeddings to obtain text features. (17); In the formula, represents the i-th element in the text, Represents the embedding vector The space you are in; For image data, the Vision Transformer model is used to seamlessly integrate the visual features of the image into the brain visual embedding space to obtain the visual features. This process is shown in formula (18): (18); In the formula, is a mapping function associated with the ith image observation, Indicates j image observation, and represents the first i Features.

[0013] Preferably, a two-layer Bayesian network is used for modeling based on text features and visual features, and the causal relationship between the teacher's anxiety state and the factors affecting the teacher's anxiety state is embodied and reasoned. The association between the causal nodes and the anxiety state is constructed through the brain visual multimodal time series data representation. Finally, the joint tree reasoning method is used to determine the degree of influence of the factors affecting the teacher's anxiety state on the teacher's anxiety state through the calculation between the polarity and degree of the teacher's anxiety state and the influencing factors; the teacher's anxiety state category is subject to a fine data range definition to distinguish the polarity and degree of teacher anxiety, including The Bayesian network structure is learned using data at the teacher level, brain region level, and activity level. After the Bayesian network structure is learned, parameter learning is performed to estimate the conditional probability distribution of variables in the Bayesian network relative to their parent nodes. If the variables in the Bayesian network follow the Bayesian distribution, the posterior distribution of the parameters is formula (19): (19); In the formula, GIt is used to represent the topological structure of the Bayesian network. It is used to reflect the state number of the node. Reflects the node The number of states, = , Indicates that under a given parent node state combination k, the node The prior probability of It means node The value combination of all parent nodes of This means that when the node value is k When , the parent node status value is j The corresponding sample size, In a given network structure G In the case of The prior distribution of In a given network structure G and parameters In the case of D The probability of Representation dataset D The marginal probability of represents the prior weight of state k, represents the state probability distribution determined by the parameters of the Dirichlet distribution, Dir () represents Dirichlet distribution, Indicates the state number of the node.

[0014] Therefore, the present invention adopts the above-mentioned embodied reasoning method of teacher anxiety based on the representation of brain visual multimodal large model, captures the anxiety manifestations of teachers in different situations and the brain mechanisms behind them, and uses a multi-head self-attention mechanism to extract the brain response patterns guided by teacher anxiety, and obtains the evolution characteristics of teacher anxiety in time series. The teacher anxiety state analysis method based on the spatiotemporal graph convolution pyramid reveals the changing trend of the teacher's anxiety state. Through causal attention calculation supported by long-short-time multi-scale time series, combined with double-layer Bayesian network technology, the embodied reasoning of the teacher's anxiety state is realized, which provides a scientific basis and strong support for improving teaching quality and promoting teacher mental health. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 To map teacher anxiety representations; Figure 2 Flowchart for temporal representation and embodied reasoning of a large brain-visual multimodal model for teacher anxiety; Figure 3 Output flow chart for brain vision multimodal temporal data representation; Figure 4Flowchart for spatiotemporal graph convolutional pyramid analysis; Figure 5 A causal graph of teacher anxiety based on a two-layer Bayesian network structure. DETAILED DESCRIPTION

[0016] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0017] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0018] Embodiment 1 like Figure 2 As shown, the present invention provides a teacher anxiety embodied reasoning method based on brain-visual multimodal large model representation, comprising the following steps: Step 1: Establish a semantic analysis mechanism for anxiety association at four levels of "teacher-brain region-activity-anxiety". This analysis mechanism establishes a set of relevant influencing factor instances based on the teacher anxiety-related factor data under the teacher brain region influencing activities; the set of relevant influencing factor instances includes the teacher level, brain region level, and activity level; The data collected in this embodiment involves multiple spaces, covering relevant information of different environments such as school, family and society. The data comes from multiple links, covering multiple dimensions such as lesson preparation, class, tutoring, testing, and training. This diversified data collection method can fully reflect the comprehensive performance of teachers under the influence of anxiety, but the state and degree of teacher anxiety will be affected by different factors, and different anxiety data paths contain rich semantic information. Therefore, in order to better obtain the complex semantic relationship in anxiety-related data, based on the perspective of the impact of teacher time-series activities on teacher anxiety, with the help of the semantic mechanism that can be perceived by the teacher activity process data, the hierarchical related factors in the current teacher brain area activity data are analyzed, and a four-level anxiety-related semantic analysis mechanism of "teacher-brain area-activity-anxiety" is established.

[0019] The four-level semantic analysis mechanism of anxiety association of "teacher-brain area-activity-anxiety" includes Teacher level teacher, attributes.collect{E,C,M}(1); Brain area level brain(visual, text),attributes.influence{E,C,M}(2); Activity level action(E,A,M,activities,t),egread:{EA,CA,MA}(3); Anxiety level (4); Among them, teacher represents teacher, attributes.collect represents attribute collection, attributes.influence represents influence, action represents activity, egread represents obtaining teacher-related activities, brain represents brain area level, visual represents visual features, text represents text features, E represents the set of instances of teacher environmental factors, A represents the generated activities, M represents the set of instances of teacher individual factors, C represents the set of instances of course factors, activities represents activities, t represents time, Indicates the teacher's anxious state.

[0020] At the teacher level, teachers as subjects are closely related to teacher anxiety. Teacher anxiety is quantified through semantic analysis. Teacher anxiety is closely related to environmental factors such as society, school, and family. Teacher anxiety data is collected and quantified to construct a set of teacher environmental factor instances E={ In addition, factors such as subject courses and difficulty will also affect teacher anxiety. Collect course-related data to quantify and construct a set of course factor examples. Teacher anxiety is also closely related to individual factors such as teacher age, gender, and teaching level. The data of teacher anxiety is collected to quantify and construct a set of examples of individual factors of teachers. .

[0021] At the brain region level, different visual or text features can activate specific brain voxels, thereby affecting the teacher's emotions and psychological state. By analyzing visual features (such as student feedback, classroom interaction) and text features (such as the complexity of teaching content, the frequency of student questions, etc.), the brain activity patterns of teachers during the teaching process can be effectively quantified and predicted. For example, when teachers are faced with complex course content, the activity of their frontal lobes and hippocampus may increase significantly, reflecting their anxiety about classroom management and student understanding. Therefore, based on the existing factors at the teacher level, brain stimulation data such as heart rate and hippocampal activity affected by brain vision are collected and quantified to construct a brain region instance set B={ }.

[0022] At the activity level, the self-efficacy theory provides a basis for teacher activities. Based on the existing factors at the brain area level, teacher activities cover multiple dimensions such as lesson preparation, teaching, tutoring, testing, and training. The data of teacher activity factors are collected and quantified to construct a set of teacher activity instances D={ }.

[0023] The anxiety level is based on the teacher level, brain area level and activity level. The perceived time series teacher anxiety is affected by the aggregate data of the above three levels. Based on the teacher anxiety representation diagram, the teacher anxiety state is defined as .

[0024] Based on the above four-level anxiety-related semantic analysis mechanism, multimodal data related to teacher anxiety are analyzed to remove factors unrelated to teacher anxiety and obtain instance set data of factors related to teacher anxiety.

[0025] The set of relevant influencing factor instances includes Teacher-level data includes the set of teacher environment factor instances E={ }, Course Factor Instance Collection , the set of teacher individual factor instances M={ }; The brain region level data includes the brain region instance set B = { }; Activity level data includes teacher activity instance set D = { Step 2: Process the data at the brain region level to obtain the teacher anxiety state category; The data processing at the brain region level includes two processes: the first process is to process the brain response data in the time domain, spatial domain and frequency domain, and after further analysis, the power spectrum density of different frequency bands is obtained; the second process is to process the power spectrum density with a multi-head self-attention mechanism, and different attention values ​​are used to distinguish the importance of teachers' psychological emotions, behavioral performance and achievement levels in teacher anxiety, so as to calculate and output different categories of teacher anxiety; Under the guidance of teacher anxiety, brain response pattern analysis based on multi-head self-attention provides an advanced exploration path to deeply understand the characteristics of teachers' brain activities in different teaching situations. The multi-head self-attention mechanism allows linear transformation of queries, keys, and values ​​to achieve richer information representation by computing multiple self-attention layers in parallel. This mechanism enables the model to extract information from different subspaces in the data quantitatively represented at the teacher level, which lays the foundation for analyzing complex brain response patterns.

[0026] By acquiring response data related to brain regions such as the occipital lobe, temporal lobe, and parietal lobe, it can be obtained by collecting electroencephalogram (EEG) or functional magnetic resonance imaging (fMRI).

[0027] In the time domain, short-time Fourier transform is used to analyze the instantaneous frequency characteristics of the signal. The specific implementation method is shown in formula (5): (5); In the formula, represents the instantaneous frequency of the signal, S (t ) indicates the time t signal, w ( t ) indicates the use of window function, represents the integral budget, Represents infinity, S ( \tau ) represents a non-stationary signal, w ( \tau - t ) represents the window function, Indicates the spectrum value; In the spatial domain, the EEG-based spatial filtering technology is used to perform independent component analysis. The specific implementation method is shown in formula (6): (6); In the formula, represents the observation data matrix, m Indicates the number of sensors, n Indicates the number of time frames, is the spatial filter matrix, Y represents the extracted independent components; In frequency domain analysis, the time domain signal is converted into the frequency domain signal through fast Fourier transform. S ( t ) spectrum The specific calculation method is shown in formula (7): (7); After further analysis, the power spectrum density of different frequency bands is obtained. The specific implementation method is shown in formula (8): (8); Convert brain response data into input sequence X = ( , ···, ), the brain response data is analyzed based on multi-head self-attention. The specific method is shown in formula (9) (9).

[0028] In the formula, Indicates a query, K Indicates the key, V Indicates the value, E It's environmental factors. C It is the curriculum factor. M It is the individual factor of teachers.

[0029] Step 3: Perform latent coding and mutual information regularization on the brain region data, match them to the multi-space and multi-link dimensional framework preset at the activity level according to the mutual information regularization, and then generate brain visual multimodal time series data representation for teacher anxiety through normalization processing; The brain region level data is sequentially subjected to latent coding and mutual information regularization, including The specific calculation method of latent coding is shown in formula (10): (10); In the formula, Is The approximate posterior distribution represented by the parameterized encoder network, Is σ Parameterized decoder network , DKL It means KL Divergence, ELBO represents the variational lower bound, represents the decoder network parameterization result, represents the potential coding value, and e represents a natural constant; Mutual information regularization is shown in formula (11): (11); In the formula, Through the depth of the realization of some auxiliary distribution of neural networks by Parameterization, is a latent variable, The corresponding is the teacher activity instance set D , P The corresponding distribution probability is corresponds to the uncertainty in z, It represents the degree of regular fitting. Indicates that in the known Based on the uncertainty of z, It means that under given conditions z, the random variable The conditional probability distribution of It means that under the condition of given input R, the data represents the conditional probability distribution of z, represents the point mutual information between two data, Represents the regular value between two data; At each stage, the scale of nonlinear transformation is controlled based on input time series feature preprocessing, and the input scale of each stage is controlled by using time series sliding window strategy, and the normalized embedding of each time series slice and its position feature embedding are input into the encoder. According to the mutual information regularization matching to the multi-space and multi-link dimension framework preset at the activity level, it is found through normalization processing that teacher anxiety is closely related to factors such as teacher psychological emotions, teacher behavioral performance and teacher achievement level, thus generating brain visual multimodal time series data representation for teacher anxiety.

[0030] On the basis of constructing the specific structure of the first-level indicators, some intermediate variables were further introduced as second-level indicators for analysis. Finally, by combining the relevant research results of each second-level indicator with multimodal data, a correlation indicator system for quantitative representation of brain visual multimodal time series data for teacher anxiety was preliminarily constructed. See Table 1 for details, and the representation output method is shown in Figure 3 shown.

[0031] Table 1. The index system for quantitative representation of teacher anxiety ;

[0032] By implementing fine data scoping for teacher anxiety state categories, the polarity and degree of teacher anxiety can be distinguished.

[0033] Polarity and degree of teacher anxiety: As an important implicit characteristic, teacher anxiety has a series of complex properties, including the polarity and degree of anxiety. In order to achieve a more accurate analysis of teacher anxiety, its polarity and degree were quantified. This quantification process aims to systematically evaluate and compare different types of modal data to ensure the rigor and effectiveness of the analysis process. At the same time, by setting quantitative indicators, the correlation and difference between modal data can be better reflected, laying a solid foundation for the subsequent data analysis and model training of teacher anxiety.

[0034] On the polarity of teacher anxiety A In order to evaluate the emotional polarity of the text, we first conducted an in-depth analysis of the expression pictures and emotional words in the interactive information, and derived the association shown in formula (12). Next, we used the text attitude discriminant function O(TC) to evaluate the emotional polarity of the text, and performed a weighted summation of the text emotional polarity and the corresponding amount of expression information EN. In this process, the weight coefficients of the text discriminant function and the expression information were set as =0.8 and = 0.2 to adapt to the characteristics of the dataset.

[0035] (12); Regarding the level of teacher anxiety The calculation of is based on the method shown in formula (13). To this end, we first define the degree level set as , where the set S Each level in corresponds to a specific set of emotional levels. Furthermore, for the emotional level, Domain , the domain consists of k Independent opinion words In order to quantify the possibility of interactive text appearing at a certain intensity level, a probability calculation method is introduced. Appears at the degree level The possibilities can be used to ), the specific calculation method is shown in formula (14) for At the degree level The probability of appearing in .

[0036] (13); (14); In order to explore the teacher's anxiety in depth, a teacher anxiety representation diagram was constructed, such as Figure 1 Based on the emotions experienced during cognitive interaction, teachers’ anxiety emotions are divided into positive and negative emotion polarities. a , and five anxiety levels including no anxiety, mild anxiety, moderate anxiety, high anxiety and extreme anxiety q The value range of each dimension is set between -1 and 1, and each emotional state can be represented by a two-dimensional emotional state quantification value.

[0037] Step 4: Based on the video data at the teacher level, brain area level, and activity level, the teacher’s anxiety state and short-term feature vector are obtained using graph convolution pyramid operations, such as Figure 4 As shown; The graph convolution operation is used to obtain the changing pattern of teacher anxiety, including The obtained video data at the teacher level, brain area level, and activity level are hierarchically pooled using a pyramid hierarchical structure. Then, global average pooling is applied to the output of the last three layers, and the spatial features are reduced through a compression function, retaining only the key time and channel features. Finally, in the connection layer, the channel dimensions are spliced ​​to output the action sequence. The action sequence is carefully divided into multiple short segments of predefined size in the time dimension, and each segment is regarded as an independent window of the evolution of the teacher's anxiety state; a temporal pooling operation is designed to generate a temporal feature set T; The time feature set is divided finely in the time dimension to obtain the short-time feature vector. The covariance matrix of the short-time feature vector is calculated, and then we have 、 、 , the dimensions are [7×( S + j + h )] ×[7×( S + j + h )]; The third layer divides all frames T into 7 segments, so there are 7 covariance matrices 、 ··· , the dimensions are also [7×( S + j + h )] ×[7×( S + j + h )], and finally the covariances obtained at all levels are spliced ​​together to obtain the time series. The specific calculation method is shown in formula (15) ； ； ； (15); In the formula, L is the number of training samples, The corresponding penalty coefficient is is the classification error score, N is the number of action sequence frames, The corresponding label is the sample label, so as to obtain the covariance weight matrix. S represents the total number of short-time eigenvectors, j represents the weighted sum of short-time eigenvectors, h represents the time pyramid level, and They represent the weighted objective function constructed using the bilinear classifier, represents a constant, Represents an iterated function.

[0038] Step 5: Establish a long-short multi-scale temporal mechanism and a causal attention mechanism for teacher anxiety, which are used to capture the multi-scale variation characteristics and causal relationship characteristics of teacher anxiety, and systematically identify and quantify the factors that affect the teacher's anxiety state, including teaching level pressure, colleague competition, student feedback, social relationships and emotional pressure; Multi-scale variation features include For all short-time eigenvectors Perform weighted summation to finally obtain the long-term eigenvector The specific calculation method is shown in formula (16): (16); In the formula, S represents the total number of short-time eigenvectors, h represents the time pyramid level, represents the short-time eigenvector The space you are in; The causal attention mechanism of teacher anxiety includes The short-term feature vector at the current moment With the long-term eigenvector Form multiple sets of temporary feature vectors OL as input for causal attention calculation ; Multiply the temporary eigenvector OL with the V matrix through matrix operation to output the eigenvector at this moment , and send it to the stack calculation. The specific calculation method is shown in formula (17), and the calculated eigenvector matrix is ​​obtained. The specific calculation method is shown in formula (18): (17); (18); In the formula, Softmax is the normalized exponential function, Expressed as matrix linear transformation weights, It represents the similarity value of the feature vector at that moment. Represents the value obtained by multiplying this set of weights by the corresponding matrix, Indicates the moment; Output causal feature vector matrix through feed-forward residual network operation , the specific calculation method is shown in formula (19), which measures the temporal causal feature vector matrix of the final output ( , ) (19); In the formula, ( ) represents a series of calculated eigenvector matrices.

[0039] Step 6: Process the data at the teacher level, brain region level, and activity level into visual features and text features; based on the text features and visual features, use a two-layer Bayesian network to build a model, embodied reasoning about the causal relationship between the teacher's anxiety state and the factors that affect the teacher's anxiety state, and construct the association between the causal nodes and the anxiety state through the brain visual multimodal time series data representation. Finally, the joint tree reasoning method is used to determine the degree of influence of the factors that affect the teacher's anxiety state on the teacher's anxiety state through the calculation of the polarity and degree of the teacher's anxiety state and the influencing factors, such as Figure 5 As shown; Implement fine data scoping for teacher anxiety status categories to distinguish the polarity and degree of teacher anxiety.

[0040] Process the teacher-level, brain-region-level, and activity-level data into visual and textual features; The data at the teacher level, brain region level, activity level and the brain visual multimodal time series data of teacher anxiety include text data and image data. The text data is mapped to a high-dimensional embedding space to generate a semantically rich token embedding to obtain text features; the image data is efficiently encoded using the Vision Transformer model, and the visual features of the image are seamlessly integrated into the brain visual embedding space to obtain visual features; Map text data to a high-dimensional embedding space to generate semantically rich token embeddings. (20); In the formula, represents the i-th element in the text, Represents the embedding vector The space you are in; For image data, the Vision Transformer model is used to seamlessly integrate the visual features of the image into the brain visual embedding space. This process is shown in formula (21): (twenty one); In the formula, is a mapping function associated with the ith image observation. represents the jth image observation, and represents the i-th feature extracted from this observation.

[0041] The Bayesian network structure learning is performed using data at the teacher level, brain region level, and activity level. After completing the Bayesian network structure learning, parameter learning is performed to estimate the conditional probability distribution of variables in the network relative to their parent nodes, which will provide the necessary prior information for subsequent inference. If it is assumed that the variable follows the Bayesian distribution, the posterior distribution of its parameters is formula (23): (twenty two); In the formula, G It is used to represent the topological structure of the Bayesian network. It is used to reflect the state number of the node. Reflects the node The number of states, = , Indicates that under a given parent node state combination k, the node The prior probability of It means node The value combination of all parent nodes of This means that when the node value is k When , the parent node status value is j The corresponding sample size, Indicates that given the network structure G, the parameter The prior distribution of Indicates that given the network structure G and parameters In the case of, the probability of data set D, represents the marginal probability of the data set D, represents the prior weight of state k, represents the state probability distribution determined by the parameters of the Dirichlet distribution, Dir () represents Dirichlet distribution, Indicates the state number of the node.

[0042] Therefore, the present invention adopts the above-mentioned embodied reasoning method of teacher anxiety based on the representation of brain visual multimodal large model, captures the anxiety manifestations of teachers in different situations and the brain mechanisms behind them, and uses a multi-head self-attention mechanism to extract the brain response patterns guided by teacher anxiety, and obtain the evolution characteristics of teacher anxiety in time series. The teacher anxiety state analysis method based on the spatiotemporal graph convolution pyramid reveals the changing trend of the teacher's anxiety state. Through causal attention calculation supported by long-short-time multi-scale time series, combined with double-layer Bayesian network technology, the embodied reasoning of the teacher's anxiety state is realized, which provides a scientific basis and strong support for improving teaching quality and promoting teacher mental health.

[0043] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for embodied reasoning of teacher anxiety based on brain-visual multimodal large model representation, characterized by: The following steps are involved: Establish a semantic analysis mechanism for anxiety association at four levels of "teacher-brain region-activity-anxiety". This analysis mechanism establishes a set of relevant influencing factor instances based on the teacher anxiety-related factor data under the teacher brain region influencing activities; the set of relevant influencing factor instances includes data at the teacher level, brain region level, and activity level; By processing the data at the brain region level, the categories, polarity and degree of teachers' anxiety states were obtained; The data at the brain region level is processed to obtain brain visual multimodal time series data representation for teacher anxiety; Based on the video data at the teacher level, brain region level, and activity level, the teacher's anxiety state is obtained using graph convolution pyramid operations; Based on the long-short time multi-scale temporal mechanism, the causal attention calculation mechanism of teacher anxiety is combined to quantify the factors affecting the teacher's anxiety state; Using data at the teacher level, brain region level, and activity level, a two-layer Bayesian network was used to embodiedly reason the causal relationship between the teacher's anxiety state and the factors affecting the teacher's anxiety state. The joint tree reasoning method was used to determine the degree of influence of the factors affecting the teacher's anxiety state on the teacher's anxiety state by calculating the polarity and degree of the teacher's anxiety state and the influencing factors.

2. According to claim 1, a teacher anxiety embodied reasoning method based on brain visual multimodal large model representation is characterized by: The four-level semantic analysis mechanism of anxiety association of "teacher-brain area-activity-anxiety" includes Teacher level teacher, attributes.collect{E,C,M}(1); Brain region level brain(visual, text),attributes.influence{E,C,M}(2); Activity level action(E,A,M,activities,t),egread:{EA,CA,MA}(3); Anxiety level （4）； Among them, teacher represents teacher, attributes.collect represents attribute collection, attributes.influence represents influence, action represents activity, egread represents obtaining teacher-related activities, brain represents brain area level, visual represents visual features, text represents text features, E represents the set of instances of teacher environmental factors, A represents the generated activities, M represents the set of instances of teacher individual factors, C represents the set of instances of course factors, activities represents activities, t represents time, Indicates teacher anxiety status; The set of relevant influencing factor instances includes Teacher-level data includes the set of teacher environment factor instances E={ }, course factor instance set C = { }, the set of teacher individual factor instances M = { }; The brain region level data includes the brain region instance set B = { }; Activity level data includes teacher activity instance set D = { }.

3. According to claim 1, a teacher anxiety embodied reasoning method based on brain visual multimodal large model representation is characterized by: The data processing at the brain region level includes two processes: the first process is to process the data at the brain region level in the time domain, spatial domain and frequency domain, and after further analysis, the power spectrum density of different frequency bands is obtained; the second process is to process the power spectrum density with a multi-head self-attention mechanism, and different attention values ​​are used to distinguish the importance of teachers' psychological emotions, behavioral performance and achievement levels in teacher anxiety, so as to calculate and output different categories of teacher anxiety; In the time domain, short-time Fourier transform is used to analyze the instantaneous frequency characteristics of the signal. The specific implementation method is shown in formula (5): （5）； In the formula, represents the instantaneous frequency of the signal, S ( t ) indicates the time t signal, w ( t ) indicates the use of window function, represents the integral budget, Represents infinity, S ( \tau ) represents a non-stationary signal, w ( \tau - t ) represents the window function, Indicates the spectrum value; In the spatial domain, the EEG-based spatial filtering technology is used to perform independent component analysis. The specific implementation method is shown in formula (6): （6）； In the formula, represents the observation data matrix, m Indicates the number of sensors, n Indicates the number of time frames, is the spatial filter matrix, Y represents the extracted independent components; In frequency domain analysis, the time domain signal is converted into the frequency domain signal through fast Fourier transform. S ( t ) spectrum The specific calculation method is shown in formula (7): （7）； After further analysis, the power spectral density of different frequency bands is obtained. The specific implementation method is shown in formula (8): （8）； In the formula, It means to find the limit, Indicates that the length of the time interval tends to infinity, { 1 }{ T }express The decimal part of e represents a natural constant; Convert brain region level data into input sequence X=( , ···, ), the brain response data is analyzed based on multi-head self-attention. The specific method is shown in formula (9) （9）； In the formula, Indicates a query, K Indicates the key, V Represents a value.

4. According to claim 1, a teacher anxiety embodied reasoning method based on brain visual multimodal large model representation is characterized by: The data at the brain region level are processed with latent coding and mutual information regularization, and matched to the multi-space and multi-link dimension framework preset at the activity level according to mutual information regularization, and then normalized to generate brain visual multimodal time series data representation for teacher anxiety; The specific calculation method of latent coding is shown in formula (10): （10）； In the formula, Is The approximate posterior distribution represented by the parameterized encoder network, Is σ Parameterized decoder network , DKL It means KL Divergence, ELBO represents the variational lower bound, represents the decoder network parameterization result, represents the potential coding value; Mutual information regularization is shown in formula (11): （11）； In the formula, Through the depth of the realization of some auxiliary distribution of neural networks by Parameterization, is a latent variable, The corresponding is the teacher activity instance set D , P The corresponding distribution probability is corresponds to the uncertainty in z, It represents the degree of regular fitting. Indicates that in the known Based on the uncertainty of z, It means that under given conditions z, the random variable The conditional probability distribution of It means that under the condition of given input R, the data represents the conditional probability distribution of z, represents the point mutual information between two data, Represents the normal value between two data.

5. According to claim 1, a method for embodied reasoning of teacher anxiety based on brain-visual multimodal large model representation is characterized in that: Based on the video data at the teacher level, brain area level, and activity level, the teacher's anxiety state is obtained using graph convolution pyramid operations, including The obtained video data at the teacher level, brain area level, and activity level are hierarchically pooled using a pyramid hierarchical structure. Then, global average pooling is applied to the output of the last three layers, and the spatial features are reduced by a compression function. Finally, the channel dimensions are concatenated in the connection layer to output the action sequence. The action sequence is divided into segments in the time dimension, and each segment is used as an independent window of the evolution of the teacher's anxiety state; a time pooling operation is designed to generate a time feature set T; Divide the time feature set in the time dimension to obtain the short-time feature vector, and obtain the covariance matrix of the short-time feature vector, then we have 、 、 , the dimensions are [7×( S + j + h )] ×[7×( S + j + h )]; The third layer divides all frames T into 7 segments, so there are 7 covariance matrices 、 ··· , the dimensions are also [7×( S + j + h )] ×[7×( S + j + h )], and finally the covariances obtained at all levels are spliced ​​together to obtain the time series. The specific calculation method is shown in formula (12): ； ； ； （12）； In the formula, L is the number of training samples, The corresponding penalty coefficient is is the classification error score, N is the number of action sequence frames, The corresponding label is the sample label, so as to obtain the covariance weight matrix. S represents the total number of short-time eigenvectors, j represents the weighted sum of short-time eigenvectors, h represents the time pyramid level, and They represent the weighted objective function constructed using the bilinear classifier, represents a constant, Represents an iterated function.

6. According to claim 1, a method for embodied reasoning of teacher anxiety based on brain-visual multimodal large model representation is characterized in that: Using short-term feature vectors, we establish a long-term and short-term multi-scale temporal mechanism and a causal attention mechanism for teacher anxiety, and identify and quantify the factors that affect teacher anxiety, including teaching level pressure, competition from colleagues, student feedback, social relationships, and emotional pressure. Long- and short-term multi-scale temporal mechanisms include For all short-time eigenvectors Perform weighted summation to finally obtain the long-term eigenvector The specific calculation method is shown in formula (13): （13）； In the formula, h represents the time pyramid level, represents the short-time eigenvector The space you are in; The causal attention mechanism of teacher anxiety includes The short-term feature vector at the current moment With the long-term eigenvector Form multiple sets of temporary feature vectors OL as input for causal attention calculation ; Multiply the temporary eigenvector OL with the V matrix through matrix operation to output the eigenvector at this moment The specific calculation method is shown in formula (14), and it is sent to the stacking calculation to obtain the calculated eigenvector matrix The specific calculation method is shown in formula (15): （14）； （15）； In the formula, is the normalized exponential function, Expressed as matrix linear transformation weights, It represents the similarity value of the feature vector at that moment. Represents the value obtained by multiplying this set of weights by the corresponding matrix; Output causal feature vector matrix through feed-forward residual network operation , the specific calculation method is shown in formula (16), which measures the temporal causal characteristic vector matrix of the final output ( , ) （16）； In the formula, ( ) represents a series of calculated eigenvector matrices.

7. According to claim 1, a teacher anxiety embodied reasoning method based on brain visual multimodal large model representation is characterized by: Data at the teacher level, brain region level, and activity level are processed into visual features and text features, including The data at the teacher level, brain region level, activity level and the brain visual multimodal time series data of teacher anxiety include text data and image data, among which the text data is mapped to the high-dimensional embedding space to generate token embedding to obtain text features; the image data is efficiently encoded using the Vision Transformer model, and the visual features of the image are seamlessly integrated into the brain visual embedding space to obtain visual features; Map text data to a high-dimensional embedding space to generate token embeddings to obtain text features. （17）； In the formula, represents the i-th element in the text, Represents the embedding vector The space you are in; For image data, the Vision Transformer model is used to seamlessly integrate the visual features of the image into the brain visual embedding space to obtain the visual features. This process is shown in formula (18): （18）； In the formula, is a mapping function associated with the ith image observation, Indicates j image observation, and represents the first Features.

8. According to claim 7, a method for embodied reasoning of teacher anxiety based on brain-visual multimodal large model representation is characterized in that: Based on text features and visual features, a two-layer Bayesian network is used for modeling, and the causal relationship between the teacher's anxiety state and the factors affecting the teacher's anxiety state is embodied. The association between the causal nodes and the anxiety state is constructed through the representation of brain visual multimodal time series data. Finally, the joint tree reasoning method is used to determine the degree of influence of the factors affecting the teacher's anxiety state on the teacher's anxiety state through the calculation between the polarity and degree of the teacher's anxiety state and the influencing factors; the teacher's anxiety state category is implemented with a fine data range definition to distinguish the polarity and degree of teacher anxiety, including The Bayesian network structure is learned using data at the teacher level, brain region level, and activity level. After the Bayesian network structure is learned, parameter learning is performed to estimate the conditional probability distribution of variables in the Bayesian network relative to their parent nodes. If the variables in the Bayesian network follow the Bayesian distribution, the posterior distribution of the parameters is formula (19): （19）； In the formula, G It is used to represent the topological structure of the Bayesian network. It is used to reflect the state number of the node. Reflects the node The number of states, , Indicates that under a given parent node state combination k, the node The prior probability of It means node The value combination of all parent nodes of This means that when the node value is k When , the parent node status value is j The corresponding sample size, In a given network structure G In the case of The prior distribution of In a given network structure G and parameters In the case of D The probability of Representation dataset D The marginal probability of represents the prior weight of state k, represents the state probability distribution determined by the parameters of the Dirichlet distribution, Dir () represents Dirichlet distribution, Indicates the state number of the node.

Images