Learning state detection method based on large language model

Abstract

The embodiment of the invention discloses a learning state detection method based on a large language model, and relates to the technical field of teaching informationization, and the method comprises the steps: obtaining a first type of data and a second type of data of a target learner in a learning process; forming an overall feature vector based on the first type of data and the second type of data; and inputting the overall feature vector into a trained behavior feature detection large language model to obtain a learning behavior feature vector, and outputting learning state evaluation information of the target learner in the learning process based on the learning behavior feature vector. By adopting the method provided by the invention, not only is the behavior data of the learner utilized, but also the interaction data between the learner and the equipment is utilized, so that the complementary characteristics among the multi-source and multi-modal data can be mined, the learning state can be deeply and comprehensively detected, the condition that the learning state of the learner is poor can be found in real time, and the learning efficiency of the learner is improved. And the learning efficiency of the learner during self-learning can be improved.

Description

Technical Field

[0001] The present application relates to the field of teaching information technology, and in particular to a learning state detection method based on a large language model. Background Art

[0002] The rapid advancement of information technology and the evolving demands of education have led to the emergence of hybrid learning environments as a new trend in education. While information technology can improve teaching convenience, broaden students' knowledge, and stimulate their interest in learning, this approach generally involves merely mechanically transferring traditional instructional content to an information-based teaching platform, presenting this information electronically to each student.

[0003] The existing information-based teaching platforms often find it difficult to pay attention to the learning status of individual students. Without paying attention to the differences in students' learning styles, it is difficult to personalize learning route planning based on the learning interests of individual students. As a result, the difficulties encountered by students in the self-study process will lead to reduced learning efficiency, and the depth of actual learning is limited to the depth of the learning materials provided by the information-based teaching platform. It cannot accurately reflect the actual learning status of each student, and cannot provide targeted guidance information to students. Summary of the Invention

[0004] The present application provides a learning status detection method based on a large language model to address the shortcomings of the above-mentioned related technologies. The technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a learning status detection method based on a large language model, comprising:

[0006] Acquire first-category data and second-category data of a target learner during a learning process; wherein the first-category data is obtained based on behavioral data of the target learner, and the second-category data is obtained based on interaction data between the target learner and a learning device;

[0007] Perform feature embedding coding based on the first category of data and the second category of data to extract an overall feature vector composed of the first category of data and the second category of data;

[0008] The overall feature vector is input into a trained behavior feature detection large language model to obtain a learning behavior feature vector output by the trained behavior feature detection large language model, and learning status evaluation information of the target learner in the learning process is output based on the learning behavior feature vector.

[0009] In an optional solution of the first aspect, after outputting the learning status evaluation information of the target learner in the learning process based on the learning behavior feature vector, the method further includes:

[0010] The similarity between the learning behavior feature vector and the preset learning state adjustment scheme feature vector is compared through the adaptive learning state intervention model, and the mapping relationship between the learning state adjustment scheme and the learning state evaluation information of the target learner is determined according to the similarity value range within which the similarity falls, and the corresponding learning state adjustment scheme is output to the device end of the target learner.

[0011] In an optional solution of the first aspect, after comparing the similarity between the learning behavior feature vector and the preset learning state adjustment solution feature vector through the adaptive learning state intervention model, the method further includes:

[0012] If the similarity value is greater than the similarity threshold, the adaptive learning state intervention model is constrained by the emotional state feature vector, and a conditional probability distribution is constructed to obtain the similarity between the learning behavior feature vector and the preset learning state adjustment scheme feature vector predicted by the adaptive learning state intervention model. Based on the conditional probability distribution, it is judged whether the similarity value output by the adaptive learning state intervention model is accurate.

[0013] In an optional solution of the first aspect, obtaining the first type of data and the second type of data of the target learner during the learning process includes:

[0014] Acquire video data and audio data of the target learner during the learning process, and process them to obtain a learner head image set and a learning audio data set respectively;

[0015] Extracting a structured data set from the interaction log between the target learner and the learning device during the learning process;

[0016] The learner head image set and the learning audio data set are output as the first type of data, and the structured data set is output as the second type of data.

[0017] In an optional solution of the first aspect, performing feature embedding encoding based on the first category of data and the second category of data to extract an overall feature vector composed of the first category of data and the second category of data includes:

[0018] Extracting head image feature vectors and learning audio feature vectors based on the learner head image set and learning audio data set in the first category of data, respectively, and obtaining text feature vectors in advance based on the structured data set in the second category of data;

[0019] The head image feature vector, the learned audio feature vector and the text feature vector are combined to generate an overall feature vector of the image-audio-text.

[0020] In a second aspect, an embodiment of the present application further provides a learning state detection device based on a large language model, comprising:

[0021] A data acquisition module is used to obtain first-category data and second-category data of a target learner during the learning process; wherein the first-category data is obtained based on the behavioral data of the target learner, and the second-category data is obtained based on the interaction data between the target learner and the learning device;

[0022] A feature extraction module, configured to perform feature embedding coding based on the first category of data and the second category of data, and extract an overall feature vector composed of the first category of data and the second category of data;

[0023] A learning status detection module is used to input the overall feature vector into a trained behavior feature detection large language model, obtain the learning behavior feature vector output by the trained behavior feature detection large language model, and output the learning status evaluation information of the target learner in the learning process based on the learning behavior feature vector.

[0024] In a third aspect, an embodiment of the present application further provides an electronic device comprising a memory, a processor, and a computer program stored on the memory and runnable on the processor, wherein when the processor executes the program, the method provided in the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect is implemented.

[0025] In a fourth aspect, the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the method provided by the first aspect of the embodiment of the present application or any one of the implementation methods of the first aspect.

[0026] The beneficial effects of the technical solutions provided by some embodiments of the present application include at least:

[0027] The embodiment of the present application adopts a learning state characterization method driven by a large language model, which can monitor and analyze the learner's behavior, emotions and cognitive patterns. In view of the problem that related technologies only use single-modal analysis, which may lead to the loss of important details such as key features and insufficient generalization ability, the present application not only utilizes the learner's behavioral data, but also utilizes the interaction data between the learner and the device, and maps the obtained image, audio features and text features to the same embedding space for matching, realizes data alignment and simultaneously encodes them into feature vectors, and then inputs the feature vectors into the pre-trained large language model, so as to mine the complementary features between multi-source and multi-modal data, thereby deeply and comprehensively detecting the learner's learning state, discovering in real time when the learner's learning state is not good, and giving timely feedback, so that the learner can improve the learning efficiency during self-study and improve the level of attention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 This is a flow chart of a learning state detection method based on a large language model according to an embodiment of the present application;

[0030] Figure 2 This is a flow chart of a learning state detection method based on a large language model according to an embodiment of the present application;

[0031] Figure 3 This is a flow chart of a learning state detection method based on a large language model according to an embodiment of the present application;

[0032] Figure 4 1 is a structural diagram of a learning state detection device based on a large language model provided in an embodiment of the present application;

[0033] Figure 5 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0035] The terms "including" and "having," and any variations thereof, in the specification and claims of this application and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other steps or modules inherent to the process, method, product, or apparatus.

[0036] It should be noted that the terms "first" and "second" used in this application are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that the terms "first" and "second" may interchangeably represent a specific order or precedence, where permitted. It should be understood that the objects distinguished by "first" and "second" may interchangeably represent a specific order or precedence, where appropriate, such that the embodiments of the present application described herein can be implemented in an order other than that described or illustrated herein.

[0037] The present application is described in detail below with reference to specific embodiments.

[0038] Next, combine Figure 1 , taking the terminal executing the learning state detection method based on the large language model as an example, the learning state detection method based on the large language model provided by the embodiment of the present application is introduced. Figure 1 , Figure 1 FIG. 1 shows a flow chart of a method for detecting a learning state based on a large language model provided by an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0039] S101, obtaining the first type of data and the second type of data of the target learner during the learning process;

[0040] S102, performing feature embedding coding based on the first category of data and the second category of data, and extracting an overall feature vector composed of the first category of data and the second category of data;

[0041] S103, the feature vector is input into a trained behavior feature detection large language model to obtain a learning behavior feature vector output by the trained behavior feature detection large language model, and learning status evaluation information of the target learner in the learning process is output based on the learning behavior feature vector.

[0042] In some embodiments, in S101 , the first type of data is obtained based on behavioral data of the target learner, and the second type of data is obtained based on interaction data between the target learner and a learning device.

[0043] Specifically, the first type of data can be the learning behavior data of the learner during the self-study process, which can be the learner's behavior data collected by the camera, microphone, heart rate sensor, pulse sensor and EEG sensor of the electronic device. The second type of data can be the interactive data generated by the learner using the electronic device during the self-study process, including but not limited to the mouse, keyboard, mouse clicks, keyboard typing information, etc., which is not limited in the embodiments of the present application.

[0044] Specifically, a video stream of the student's head posture can be captured, and a fixed 2-second interval sampling can be used in advance. When the frame difference method is used to detect movement (for example, the student changes from looking at the blackboard to looking down at the notes), the sampling frequency is increased to two samples per second, and a set of learner head images with obvious features after processing is obtained: Head_Set = {P1, P2, P3, ..., P n}; You can also use a microphone device with a sampling rate of 44.1kHz and a bit depth of 16 bits to capture the learner's audio signal, and apply noise reduction and filtering technology to improve the collected audio quality to obtain the learner's interactive learning audio data set Sound_Set = {S1, S2, S3, ..., S n}; Learner interaction logs arranged in chronological order by mouse and keyboard input can be extracted and exported from electronic devices or information-based teaching platforms on electronic devices, such as the number of course clicks, teaching video viewing time, homework submission status, discussion participation status, time interval between questions, statistical accuracy rate data sets, etc., and after editing, deleting and filling missing values, handling outliers, and standardizing data preprocessing operations, the obtained structured data set is represented as Text_Set = {T1, T2, T3, ..., T n}.

[0045] Optionally, the data may be formatted and converted into JSON format and then converted into a structured dataset representation.

[0046] In some embodiments, in S102, as Figure 2 As shown, feature embedding encoding can be performed based on the first category of data and the second category of data to extract the overall feature vector composed of the first category of data and the second category of data, wherein the first category of data can include the learner's learning image data and the learner's learning audio data, and the second category of data can be online interaction log data. It can be understood that the online interaction log data is the same as the structured data set representation in S101.

[0047] Specifically, the learner's head image Head_Set and structured text data Text_Set can be used as Figure 3The input of the ViT-L / 14 image P-clip coding model is shown, and the various types of data are aligned. The image is divided into 14×14 pixel blocks, which are tiled and projected onto the dimension The shared space is represented as:

[0048]

[0049] Among them, f n (·) are pre-trained image encoding representations, which are then fed into an L-layer Transformer network.

[0050] Furthermore, the Transformer network is pre-trained and optimized through a constructed loss function. The multimodal contrastive learning objective function is:

[0051]

[0052] in, Denotes the similarity between the learner's image and text at the i-th moment, τ′ is a temperature parameter, and M is the number of samples in the batch. Paired images and texts are mapped to close positions in the embedding space through random inactivation and encoding, while mismatched images and texts are mapped to far positions, expressed as

[0053]

[0054] Different dropout masks z can be used i and z′ i , let λ represent the trade-off hyperparameter between the two objectives, and the final loss function is expressed as

[0055]

[0056] The extracted features are represented as the head image feature vector E p =(e p1 ,e p2 ,e p3 ,…,e pn );

[0057] Furthermore, the learner interaction log text data Text_Set is input into the FlanT5 fine-tuning model trained with instruction prompts, and fine-tuned to minimize the negative log-likelihood of the conditional probability of the correct output sequence given the input sequence. The log text input is converted into a format that the model can understand. The input is passed through the FlanT5 model. The model's encoder converts the input text into a series of high-dimensional vectors, and the output of the last layer of the model is selected as the feature vector output E of the text. t =(e t1 ,e t2 ,et3 ,…,e tn );

[0058] Furthermore, the learner interactive speech dataset Sound_Set is input into the audio C-Former encoding model, and the CIF alignment mechanism is used for sequence conversion. The output sequence is obtained by step-by-step integration of the information in Sound_Set as the learning audio feature vector E s =(e s1 ,e s2 ,e s3 ,…,e sn ), which is the acoustic embedding of the incoming encoding representation s k Calculate a weight a k Indicates hidden in s k The amount of sound information in the, in order to determine whether a certain boundary is located at the kth step, the weight a k Add it to the previous cumulative weight to get the current cumulative weight, as shown below:

[0059]

[0060] Use one-dimensional convolution to capture h k The local dependencies around the image are then outputted using a projection layer with sigmoid activation.

[0061] Furthermore, the tail of the feature vector composed of audio, image and text is added to the corresponding key time sequence node position and embedded in PoEb. The three data are aligned using contrastive learning. The multimodal data complement each other to capture key information. A fully connected layer is used as a projection layer (Projection Layer), which includes a weight matrix and a bias term to perform linear transformation, and the overall feature vector of image, audio and text is obtained. E = (t, e1, e2, e3, ..., e n ).

[0062] In some embodiments, in S103, as Figure 3 As shown, the embedding feature vector E=(t, e1, e2, e3, ..., e n ) is input into the trained behavior feature detection large language model, and instruction prompt fine-tuning and LoRA fine-tuning techniques are used to obtain a feature vector containing higher-level information about the learner's behavior. The specific steps include the following:

[0063] Specifically, the feature vector E obtained in step S102 is input into the pre-trained large language model for behavioral feature detection to obtain the hidden state n′ of the last layer of emotional state, which is expressed as:

[0064]

[0065] Among them, the multi-head self-attention mechanism used by Transformer_block is to obtain the query vector STU, key vector CON and value vector EMO by multiplying three different weight matrices W1, W2 and W3 respectively, which are expressed as follows:

[0066] STU=W1n, CON=W2n, EMO=W3n;

[0067] Then calculate the attention weight through the Softmax activation function, and apply the formula:

[0068]

[0069] Finally, multiple attention heads are processed in parallel, and the outputs of each attention head are spliced ​​together, and the final multi-head attention output is obtained through linear transformation:

[0070] Mu_Head(STU,CON,EMO)=Concat(head1,...,head h )W o ;

[0071] The output of Transformer_block is passed to the next layer of LLM to obtain:

[0072] n′ i-1 =LLM(e i-1 );

[0073] The final probability feature vector obtained through a linear layer and Softmax is expressed as:

[0074] R(x1|x1,x2,…,x i-1 )=softmax(W y n l-1 ′);

[0075] Among them, W y is the linear layer parameter;

[0076] Assume that the model parameter is θ. Here, we construct a large language model for behavioral feature detection and use the iterative calculation loss function L3 to update the parameter optimization model, which can be expressed as:

[0077] L3=∑ i log p(x i |x1, x2, …, x i-1 ;θ);

[0078] Therefore, the total loss L can be expressed as:

[0079] L=β1L1+β2L2+β3L3

[0080] Specifically, a preset learning state adjustment scheme feature vector can be constructed, which contains basic information of the learner, personal learning behavior, and evaluation information of the self-regulated learning level. During the training stage of the prompt fine-tuning stage, the parameters of the large language model are fixed, and the instruction prompts and learner behavior information are embedded and represented together as the initial input sequence of the large language model to obtain the global and local features of the learning behavior. The large language model outputs the learning state adjustment scheme feature vector, and the training rounds are repeated, and the parameters of the initial input sequence after updating are saved.

[0081] For example, the model design instruction prompt example is as follows:

[0082] {“Abnormal Learning Behavior Detection Instructions 1”: “Please identify abnormal behaviors in self-regulated learning by analyzing the learner’s head posture, interactive learning, and interaction log data.”

[0083] Abnormal Learning Behavior Evaluation Instructions 2: "When detecting abnormal behavior in learners, you can use criteria such as frequency of posture changes, posture stability, voice activity, speech speed and volume, interaction quality, learner operation frequency, and learning path to assign a score to the degree of abnormal behavior."

[0084] "Learning behavior data input": "Learner A's head posture learning image data, learning audio data, and interaction log data."

[0085] "Learning Behavior Anomaly Detection Output": "Learner A's head posture changes frequently, with instability or excessive tilt, low voice activity, slow interaction with the teacher, and excessive time spent on the learning page. These characteristics indicate that the learner is not interested in the learning content, lacks concentration, has low class participation, and has difficulty understanding the learning process. This indicates that learning anomalies exist. Intervention measures are recommended, and abnormal scoring data is obtained."

[0086] Specifically, multiple parallel LoRA plug-ins are embedded in the output layer of the large language model, and the parameters of the large language model are divided into fixed weights W and learnable parameters A and B through matrix low-rank decomposition. Each LoRA plug-in P i With corresponding parameter A i and B i , set parameter A i Initialized to Gaussian distribution, the initialization process can be expressed as:

[0087]

[0088] Among them, it means that the elements in A follow the mean 0 and variance σ 2 Gaussian distribution, parameter B iInitialized to all 0, LoRA adjusts the original weight matrix through these two parameter matrices. The adjusted weight matrix is ​​represented by W i ′, used for the forward propagation process of the model, is expressed as:

[0089] W i ′=W+BA;

[0090] Among them, W is the original weight matrix.

[0091] Specifically, the large language model may be used to extract the hidden state vector STP of the preset learning state adjustment scheme feature vector for rating classification.

[0092] In some embodiments, after S103, the similarity between the learning behavior feature vector and the preset learning state adjustment scheme feature vector can be compared through an adaptive learning state intervention model, and the mapping relationship between the learning state adjustment scheme and the learning state evaluation information of the target learner can be determined according to the similarity value range in which the similarity falls, and the corresponding learning state adjustment scheme can be output to the device end of the target learner.

[0093] Specifically, the adaptive intervention model can calculate the learning behavior feature vector R output by the large language model and the preset learning state adjustment plan feature vector STP through dot multiplication to obtain the learning state evaluation information of the target learner, specifically the similarity value. The prediction formula is:

[0094]

[0095] Assume that learner A uses the collected multimodal data to input the large language model to characterize the learning state, and finds that the learner has an abnormal learning state with poor plan execution, difficulty in emotional management, and low learning motivation, and obtains the similarity value. If the score is 95, the student's learning status will be classified according to the set similarity threshold, for example, into three levels: mild behavioral deviation, severe behavioral deviation, and major behavioral deviation. The learning status of learners classified as having major behavioral deviations can be intervened to prompt the learners to adjust their learning plans / learning status.

[0096] In some embodiments, after comparing the similarity between the learning behavior feature vector and the preset learning state adjustment solution feature vector through the adaptive learning state intervention model, the method further includes:

[0097] If the similarity value is greater than the similarity threshold, the adaptive learning state intervention model is constrained by the emotional state feature vector, and a conditional probability distribution is constructed to obtain the similarity between the learning behavior feature vector and the preset learning state adjustment scheme feature vector predicted by the adaptive learning state intervention model. Based on the conditional probability distribution, it is judged whether the similarity value output by the adaptive learning state intervention model is accurate.

[0098] The emotional state eigenvector n′ can be used as a constraint to constrain the adaptive intervention model, thereby reducing the probability of intervention errors and eliminating model misjudgment. From a probability perspective, the prediction conditional probability distribution is constructed as follows:

[0099]

[0100] in, is the mean stp and variance σ 2 The probability density function of the Gaussian normal distribution.

[0101] The emotional state feature vector can be composed of three parts: (1) the parameters W within the large language model; (2) the feature vector n′ of the emotional state i ; (3) Gaussian noise. The conditional probability distribution for the learner is constructed as follows:

[0102]

[0103] Using the above conditions, the maximum a posteriori probability deformation is used to construct the loss function as follows:

[0104]

[0105] The optimizer is used to optimize the model parameters to achieve the effect of using the emotional state to constrain the adaptive intervention strategy, thereby eliminating the situation where the adaptive learning state intervention model predicts inaccurate results.

[0106] In some embodiments, if the similarity value is less than or equal to a similarity threshold, for example, less than or equal to 60, there is no need to check the similarity value.

[0107] The mapping relationship between the learning state adjustment plan and the learning state evaluation information of the target learner can be determined based on the similarity value range into which the similarity falls through a table, a mapping table, etc., and the corresponding learning state adjustment plan is output to the device end of the target learner.

[0108] Specifically, if the similarity value is Then the corresponding learning status adjustment plan can be output to the target learner's device, such as providing basic time management and planning tools, such as training on how to use calendars and reminder applications, or prompting for brief deep breathing exercises and mindfulness meditation to help relieve daily learning stress; if the similarity value is This indicates that the learner has problems with emotions, attention, etc., and the learning efficiency will be negatively affected. It is necessary to provide a learning status adjustment plan, such as organizing students to join learning support groups to share experiences and encourage each other, prompting students to improve their attention, or providing a study time adjustment plan, etc. The embodiments of this application are not limited to this.

[0109] In this way, the module utilizes similarity matching methods to explore the correlation between the characteristics of abnormal learning states, such as low mood and inattention, and the learning state adjustment solutions corresponding to the preset vectors, thus matching the most appropriate learning state adjustment solution. This application addresses specific problems encountered by learners during self-study and provides learning state adjustment solutions for learners in abnormal states, helping them to promptly correct behaviors that deviate from their goals and improve their overall learning efficiency.

[0110] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0111] See next Figure 4 , is a structural diagram of a learning state detection device based on a large language model provided by an exemplary embodiment of the present application. The device can be implemented as all or part of a terminal through software, hardware, or a combination of both, and can also be integrated on a server as an independent module. The learning state detection device based on a large language model in the embodiment of the present application can be applied to a terminal or the cloud. The device 40 includes a data acquisition module 401, a feature extraction module 402, and a learning state detection module 403, wherein:

[0112] The data collection module 401 is used to obtain first-category data and second-category data of the target learner during the learning process; wherein the first-category data is obtained based on the behavioral data of the target learner, and the second-category data is obtained based on the interaction data between the target learner and the learning device;

[0113] The feature extraction module 402 is used to perform feature embedding coding based on the first category of data and the second category of data, and extract an overall feature vector composed of the first category of data and the second category of data;

[0114] The learning status detection module 403 is used to input the overall feature vector into the trained behavior feature detection large language model, obtain the learning behavior feature vector output by the trained behavior feature detection large language model, and output the learning status evaluation information of the target learner in the learning process based on the learning behavior feature vector.

[0115] It should be noted that the device 40 provided in the above embodiment, when executing the learning state detection method based on a large language model, only uses the division of the above-mentioned functional modules as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device provided in the above embodiment and the embodiment of the learning state detection method based on a large language model are based on the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.

[0116] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of any of the above-mentioned method embodiments are implemented.

[0117] See Figure 5 , is a structural block diagram of an electronic device provided in an embodiment of the present application.

[0118] like Figure 5 As shown, the electronic device 500 includes a processor 501 and a memory 502 .

[0119] In the embodiment of the present application, the processor 501 is the control center of the computer system and can be the processor of a physical machine or the processor of a virtual machine. The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 501 can be implemented in the form of at least one hardware selected from the group consisting of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array).

[0120] The processor 501 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also called a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0121] The memory 502 may include one or more computer-readable storage media, which may be non-transitory. The memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments of the present application, the non-transitory computer-readable storage medium in the memory 502 is used to store at least one instruction, which is used to be executed by the processor 501 to implement the method in the embodiment of the present application.

[0122] In some embodiments, the electronic device 500 further includes: a peripheral device interface 503 and at least one peripheral device 504. The processor 501, the memory 502, and the peripheral device interface 503 can be connected via a bus or signal lines. Each peripheral device 504 can be connected to the peripheral device interface 503 via a bus, signal lines, or a circuit board. Specifically, the peripheral devices 504 include: a display screen, a camera, and an audio circuit. The peripheral device interface 503 can be used to connect at least one I / O (Input / Output)-related peripheral device to the processor 501 and the memory 502.

[0123] In some embodiments of the present application, the processor 501, the memory 502, and the peripheral device interface 503 are integrated on the same chip or circuit board; in some other embodiments of the present application, any one or two of the processor 501, the memory 502, and the peripheral device interface 503 may be implemented on separate chips or circuit boards. This embodiment of the present application is not specifically limited to this.

[0124] The electronic device structure block diagram shown in the embodiment of the present application does not constitute a limitation on the electronic device 500. The electronic device 500 may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0125] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any of the aforementioned embodiments. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a microdrive, a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0126] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A learning state detection method based on a large language model, It is characterized in that include: Acquire first type of data and second type of data of the target learner during the learning process; wherein the first type of data is obtained based on the behavior data of the target learner, and the second type of data is obtained based on the interaction data between the target learner and the learning device; Perform feature embedding coding based on the first category of data and the second category of data to extract an overall feature vector composed of the first category of data and the second category of data; The overall feature vector is input into a trained behavior feature detection large language model to obtain a learning behavior feature vector output by the trained behavior feature detection large language model, and based on the learning behavior feature vector, learning status evaluation information of the target learner in the learning process is output.

2. The method according to claim 1, It is characterized in that After outputting the learning status evaluation information of the target learner in the learning process based on the learning behavior feature vector, the method further includes: The similarity between the learning behavior feature vector and the preset learning state adjustment scheme feature vector is compared through an adaptive learning state intervention model, and the mapping relationship between the learning state adjustment scheme and the learning state evaluation information of the target learner is determined according to the similarity value range within which the similarity falls, and the corresponding learning state adjustment scheme is output to the device end of the target learner.

3. The method according to claim 2, It is characterized in that After comparing the similarity between the learning behavior feature vector and the preset learning state adjustment scheme feature vector through the adaptive learning state intervention model, the method further includes: If the value of the similarity is greater than the similarity threshold, the adaptive learning state intervention model is constrained by the emotional state feature vector, and a conditional probability distribution is constructed to obtain the similarity between the learning behavior feature vector predicted by the adaptive learning state intervention model and the preset learning state adjustment scheme feature vector. Based on the conditional probability distribution, it is determined whether the similarity value output by the adaptive learning state intervention model is accurate.

4. The method according to claim 1, It is characterized in that The obtaining of the first type of data and the second type of data of the target learner during the learning process includes: Acquire the video data and audio data of the target learner during the learning process, and process them to obtain a learner head image set and a learning audio data set respectively; Extracting a structured data set from the interaction log between the target learner and the learning device during the learning process; The learner head image set and the learning audio data set are output as the first category of data, and the structured data set is output as the second category of data.

5. The method according to claim 4, It is characterized in that The performing feature embedding coding based on the first category of data and the second category of data to extract an overall feature vector composed of the first category of data and the second category of data includes: Based on the learner head image set and the learning audio data set in the first category of data, a head image feature vector and a learning audio feature vector are respectively extracted, and based on the structured data set in the second category of data, a text feature vector is obtained in advance; The head image feature vector, the learned audio feature vector and the text feature vector are combined to generate an overall feature vector of the image-audio-text.

6. A learning state detection device based on a large language model, It is characterized in that include: A data collection module, used to obtain first-category data and second-category data of a target learner during a learning process; wherein the first-category data is obtained based on the behavior data of the target learner, and the second-category data is obtained based on the interaction data between the target learner and a learning device; A feature extraction module, used for performing feature embedding coding based on the first category of data and the second category of data, and extracting an overall feature vector composed of the first category of data and the second category of data; A learning status detection module is used to input the overall feature vector into a trained behavior feature detection large language model, obtain the learning behavior feature vector output by the trained behavior feature detection large language model, and output the learning status evaluation information of the target learner in the learning process based on the learning behavior feature vector.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images