Data processing method and device, teaching system, and storage medium

Abstract

The application provides a data processing method and device, a teaching system and a storage medium. The method comprises determining an interest degree score based on a user's active question number, continuous follow-up question round and time length proportion for each topic in a topic sequence, and mining emotional changes in a user's learning process based on an emotional score sequence and a cognitive load value sequence in a human-computer dialogue process. Through analysis of human-computer dialogue data in a student's learning process, the user's interest and emotional fluctuation are quantified in multiple data dimensions, the interest distribution and emotional changes of the student in the process dimension are fully mined, the educational law is fitted, and the student's learning situation is comprehensively and accurately evaluated. Compared with result indexes such as examination scores and subjective evaluations of educators, the scheme in the specification automatically quantifies and identifies the real learning process of the student based on dialogue data, the evaluation result is objective and has strong timeliness, and reliable basis is provided for personalized auxiliary education of the student.

Description

Technical Field

[0001] This manual relates to the field of educational information technology, specifically to a data processing method and device, a teaching system, and a storage medium. Background Technology

[0002] In the field of education, assessing and analyzing students' learning processes and mastery can effectively help educators understand students' learning situations, optimize teaching content and methods, and is a necessary step in improving teaching quality. However, current technologies for assessing student learning are often limited in scope and subjective, lacking in-depth analysis of the learning process and failing to accurately reflect students' learning status. Summary of the Invention

[0003] To achieve comprehensive and accurate learning analysis, this specification provides a data processing method and apparatus, a teaching system, and a storage medium.

[0004] Firstly, the embodiments of this specification provide a data processing method, including: For each topic in the current topic sequence, a weighted fusion of the number of times a user actively asks questions, the number of consecutive follow-up questions, and the duration percentage of the topic is used to obtain an interest score for that topic. The topic sequence is obtained by segmenting multi-round dialogue data between the user and the teaching system, and each topic in the topic sequence includes at least one round of dialogue data. The number of active questions represents the number of times the user actively initiates a question in that topic, the number of consecutive follow-up questions represents the number of times the user asks follow-up questions on the same topic consecutively, and the duration percentage represents the ratio of the duration of that topic to the total duration of the multi-round dialogue. The weights of the number of active questions, the number of consecutive follow-up questions, and the duration percentage of the topic are positively correlated with their priorities. The interest score of any topic is used to determine topics of interest from the topic sequence. An emotion trend curve is generated based on the emotion score sequence composed of the emotion scores of each round of dialogue data in the topic sequence and the cognitive load value sequence composed of the cognitive load data of each round of dialogue data in the current topic sequence. The emotion trend curve includes a cognitive load value curve, which includes one or more high load intervals, and the high load intervals represent the time intervals in which the cognitive load value is greater than a preset threshold. Output the topic interest score or the topics of interest determined from the topic sequence based on the topic interest score, and the sentiment trend curve, to evaluate the user's learning progress.

[0005] In some possible implementations, the weight of the number of proactive questions is greater than the weight of the number of consecutive follow-up questions, and / or the weight of the number of consecutive follow-up questions is greater than the weight of the duration percentage.

[0006] In some possible implementations, generating an emotion trend curve based on the emotion score sequence composed of the emotion scores of each round of dialogue data in the topic sequence and the cognitive load value sequence composed of the cognitive load data of each round of dialogue data in the current topic sequence includes: Based on the emotional polarity of user statements in each round of dialogue data, an emotional score is determined for each round of dialogue data, and the emotional score sequence includes the emotional scores for each round of dialogue data; Based on the cognitive load data of the user in each round of dialogue, determine the sequence of cognitive load values; Align the emotion score sequence and the cognitive load value sequence to the same time axis to generate the emotion trend curve.

[0007] In some possible implementations, the data processing method further includes: For each topic in the topic sequence, a weighted fusion of the user's question round percentage, keyword round percentage, and semantic deviation score in that topic is used to obtain a comprehension deviation score. The question round percentage represents the ratio of the number of times a user asks the same question in that topic to the total number of dialogue rounds in that topic. The keyword round percentage represents the ratio of the number of times a user mentions the first type of keyword in that topic to the total number of dialogue rounds in that topic. The semantic deviation score is determined based on the semantic deviation between the user's answer and the system's expected answer in that topic. The weakness score for each topic is obtained by weighting and fusing the number of repeated questions, the number of cognitive deadlocks, and the comprehension deviation score. The number of repeated questions refers to the number of times a user asks the same question repeatedly in this topic and other topics. The number of cognitive deadlocks refers to the number of times a user experiences a cognitive deadlock during the duration of this topic.

[0008] In some possible implementations, the data processing method further includes: Feedback results are generated based on at least one of the topics of interest, topics of weakness, and emotional trend curves, and the feedback results are used to represent the user's learning status during the duration of the multi-round dialogue. The feedback result is sent to the user terminal device so that the feedback result is output on the user terminal device.

[0009] In some possible implementations, the multi-turn dialogue data represents data generated by the user during a single learning process, and the method further includes: Based on the trends of changes in points of interest, weaknesses, and emotional fluctuations from multiple feedback results, a tracking assistance result is generated and sent to the user terminal device for output on the user terminal device; The multiple feedback results are obtained based on the dialogue data of the user's continuous learning process. The trend of interest changes is obtained based on the changes in topics of interest in the multiple feedback results. The trend of weakness changes is obtained based on the changes in topics of weakness in the multiple feedback results. The trend of emotion fluctuation changes is obtained based on the changes in the emotion trend curve in the multiple feedback results.

[0010] In some possible implementations, the topic sequence is obtained by: acquiring multi-turn dialogue data between the user and the teaching system, and performing topic segmentation on the multi-turn dialogue data to obtain the topic sequence, wherein the topic segmentation process includes at least one of the following: Determine the semantic similarity between two adjacent rounds of dialogue data, and segment the two adjacent rounds of dialogue data if the semantic similarity is lower than a first threshold; If a user's speech in a round of dialogue data contains preset segmentation keywords, the dialogue data in that round is segmented from the previous round of dialogue data. If the time interval between two adjacent rounds of dialogue data exceeds a second threshold, the two adjacent rounds of dialogue data are segmented.

[0011] Secondly, embodiments of this specification provide a data processing apparatus, including: The interest point identification module is configured to, for each topic in the current topic sequence, perform a weighted fusion of the number of times a user actively asks questions, the number of consecutive follow-up questions, and the duration percentage of that topic to obtain an interest score for that topic. The topic sequence is obtained by segmenting multi-turn dialogue data between the user and the teaching system, and each topic in the topic sequence includes at least one round of dialogue data. The number of active questions represents the number of times the user actively initiates a question in that topic; the number of consecutive follow-up questions represents the number of times the user continuously asks follow-up questions about the same topic; and the duration percentage represents the ratio of the duration of that topic to the total duration of the multi-turn dialogue. The weights of the number of active questions, the number of consecutive follow-up questions, and the duration percentage of that topic are positively correlated with their respective priorities. The interest score of any topic is used to determine topics of interest from the topic sequence. The emotion recognition module is configured to generate an emotion trend curve based on the emotion score sequence composed of the emotion scores of each round of dialogue data in the topic sequence and the cognitive load value sequence composed of the cognitive load data of each round of dialogue data in the current topic sequence. The emotion trend curve includes a cognitive load value curve, and the cognitive load value curve includes one or more high load intervals, and the high load intervals represent the time intervals in which the cognitive load value is greater than a preset threshold. The results feedback module is configured to output the topic interest score or the topics of interest determined from the topic sequence based on the topic interest score, as well as the sentiment trend curve, to evaluate the user's learning progress.

[0012] In some possible implementations, the weight of the number of proactive questions is greater than the weight of the number of consecutive follow-up questions, and / or the weight of the number of consecutive follow-up questions is greater than the weight of the duration percentage.

[0013] In some possible implementations, the emotion recognition module is configured as follows: Based on the emotional polarity of user statements in each round of dialogue data, an emotional score is determined for each round of dialogue data, and the emotional score sequence includes the emotional scores for each round of dialogue data; Based on the cognitive load data of the user in each round of dialogue, determine the sequence of cognitive load values; Align the emotion score sequence and the cognitive load value sequence to the same time axis to generate the emotion trend curve.

[0014] In some possible implementations, the data processing apparatus further includes a vulnerability identification module, configured to: For each topic in the topic sequence, a weighted fusion of the user's question round percentage, keyword round percentage, and semantic deviation score in that topic is used to obtain a comprehension deviation score. The question round percentage represents the ratio of the number of times a user asks the same question in that topic to the total number of dialogue rounds in that topic. The keyword round percentage represents the ratio of the number of times a user mentions the first type of keyword in that topic to the total number of dialogue rounds in that topic. The semantic deviation score is determined based on the semantic deviation between the user's answer and the system's expected answer in that topic. The weakness score for each topic is obtained by weighting and fusing the number of repeated questions, the number of cognitive deadlocks, and the comprehension deviation score. The number of repeated questions refers to the number of times a user asks the same question repeatedly in this topic and other topics. The number of cognitive deadlocks refers to the number of times a user experiences a cognitive deadlock during the duration of this topic. Output the weakness score of the topic or determine the weak topic from the topic sequence based on the weakness score of the topic.

[0015] In some possible implementations, the result feedback module is configured as follows: Feedback results are generated based on at least one of the topics of interest, topics of weakness, and emotional trend curves, and the feedback results are used to represent the user's learning status during the duration of the multi-round dialogue. The feedback result is sent to the user terminal device so that the feedback result is output on the user terminal device.

[0016] In some possible implementations, the multi-turn dialogue data represents the data generated by the user during a single learning process, and the result feedback module is configured to: Based on the trends of changes in points of interest, weaknesses, and emotional fluctuations from multiple feedback results, a tracking assistance result is generated and sent to the user terminal device for output on the user terminal device; The multiple feedback results are obtained based on the dialogue data of the user's continuous learning process. The trend of interest changes is obtained based on the changes in topics of interest in the multiple feedback results. The trend of weakness changes is obtained based on the changes in topics of weakness in the multiple feedback results. The trend of emotion fluctuation changes is obtained based on the changes in the emotion trend curve in the multiple feedback results.

[0017] In some possible implementations, the apparatus further includes a topic segmentation module configured to: acquire multi-turn dialogue data between the user and the teaching system, and perform topic segmentation on the multi-turn dialogue data to obtain the topic sequence, wherein the topic segmentation process includes at least one of the following: Determine the semantic similarity between two adjacent rounds of dialogue data, and segment the two adjacent rounds of dialogue data if the semantic similarity is lower than a first threshold; If a user's speech in a round of dialogue data contains preset segmentation keywords, the dialogue data in that round is segmented from the previous round of dialogue data. If the time interval between two adjacent rounds of dialogue data exceeds a second threshold, the two adjacent rounds of dialogue data are segmented.

[0018] Thirdly, some embodiments of this specification provide a teaching system, including: Wearable devices are used to collect user speech data during the learning process; The server is communicatively connected to the wearable device and is used to receive user speech data sent by the wearable device and return system response data to the wearable device. The server is configured to perform the method described in any of the above embodiments.

[0019] Fourthly, some embodiments of this specification provide a storage medium storing computer instructions for implementing the methods described in any of the above embodiments.

[0020] The data processing method described in this specification includes determining an interest score for each topic in the topic sequence based on the number of times the user actively asked questions, the number of consecutive follow-up questions, and the percentage of time spent asking questions. It also mines emotional changes in the user's learning process based on the emotional score sequence and cognitive load sequence of the human-computer dialogue process. By analyzing human-computer dialogue data during students' learning process, and comprehensively quantifying user interest and emotional fluctuations across multiple data dimensions, this method can fully explore students' interest distribution and emotional changes in the process dimension, aligning with educational principles and providing a comprehensive and accurate assessment of students' learning progress. Compared to outcome indicators such as exam scores and educators' subjective evaluations, this specification's solution automatically quantifies and identifies students' actual learning process based on dialogue data. The evaluation results are objective and timely, providing a reliable basis for personalized supplementary education for students. Attached Figure Description

[0021] Figure 1a and Figure 1b This is a schematic diagram of the architecture of the teaching system in some embodiments of this specification.

[0022] Figure 2 This is a flowchart of a data processing method in an exemplary embodiment of this specification.

[0023] Figure 3 This is a flowchart of a data processing method in another exemplary embodiment of this specification.

[0024] Figure 4 This is a schematic diagram of an emotion trend curve in one exemplary embodiment of this specification.

[0025] Figure 5 This is a flowchart of a data processing method in another exemplary embodiment of this specification.

[0026] Figure 6 This is a structural block diagram of a data processing apparatus in an exemplary embodiment of this specification.

[0027] Figure 7 This is a structural block diagram of an electronic device in an exemplary embodiment of this specification. Detailed Implementation

[0028] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws and standards of the relevant regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0029] In related technologies, the assessment and analysis of students' learning mainly includes the following methods: assessing students' knowledge mastery through regular test scores; evaluating students' learning process based on user behavior data statistics from a learning management system (LMS), such as recording students' online learning time, video viewing progress, and homework submission rate; assessing students' mastery of knowledge points by statistically analyzing students' accuracy in answering questions; and evaluating students' learning based on manual observation by educators (such as teachers and parents).

[0030] However, the aforementioned methods of learning assessment are rather simplistic. Test scores and accuracy rates only reflect learning outcomes and fail to reveal the thought processes, points of confusion, and areas of interest during the learning process, and are also subject to chance. Statistical data on online learning also fail to reflect the depth of students' cognitive understanding; for example, prolonged viewing of instructional videos does not necessarily indicate a deeper level of knowledge acquisition. These statistical data lack in-depth analysis of the learning process. Furthermore, manual assessment relies entirely on the educator's subjective perception and cannot objectively evaluate students' learning progress.

[0031] Based on this, the embodiments of this specification provide a dialogue data processing method and apparatus, a teaching system, a storage medium, and a computer program product, which aim to deeply explore students' interests and emotional changes during the learning process based on the human-computer dialogue data of the teaching system, and comprehensively and accurately analyze the learning situation from the process dimension.

[0032] Understandably, with the development of artificial intelligence technology, dialogue systems based on Large Language Models (LLMs) are becoming widely used in various fields, such as intelligent customer service, intelligent education, and psychological counseling. Taking intelligent teaching-assisted systems as an example, by leveraging the breadth of knowledge and reasoning ability of large language models, the system can simulate teacher-student interactive dialogues to assist students' learning process.

[0033] During the teaching support process, the continuous dialogue between students and the teaching system contains a wealth of information such as learning thinking, knowledge confusion, interests and preferences, and real-time emotions. This information can fully reflect the students' true learning status and thinking process, and can achieve a more accurate assessment of the entire learning process than outcome indicators such as exam scores.

[0034] Figure 1a This specification shows an architecture diagram of the teaching system in some embodiments. Figure 1b A schematic diagram of a teaching system in an exemplary embodiment is shown below. Figure 1a and Figure 1b Please provide an explanation.

[0035] The teaching system may include a wearable device 100, a server 200, and a network 400. In some embodiments, the teaching system may also include a user terminal device 300.

[0036] The wearable device 100 can be worn by students during study time. The device form of the wearable device 100 can be any suitable form, and the part of the student's body worn will vary depending on the form of the wearable device 100. In some embodiments, the wearable device 100 can be implemented as glasses, a watch, a bracelet, a wristband, an armband, a ring, a headband, headphones, clothing, or an accessory. For example... Figure 1b As shown, the wearable device 100, taking glasses as an example, can be worn on the eyes during the student's study period.

[0037] Wearable device 100 includes processor 110, memory 120, one or more sensors 130, and communication module 140. Memory 120 may store application programs or other computer-executable instructions, and processor 110 includes one or more computing cores that can read the executable instructions from memory 120 to perform one or more method steps in this specification.

[0038] One or more sensors 130 can acquire relevant data from the wearable device 100. For example, in one embodiment, the one or more sensors 130 may include one or more physiological sensors, including but not limited to heart rate sensors, blood oxygen sensors, blood pressure sensors, and body temperature sensors. In another embodiment, the one or more sensors 130 may include one or more wearable sensors, including but not limited to infrared sensors and capacitance sensors.

[0039] For example, in another example, one or more sensors 130 may include image sensors, including but not limited to RGB cameras, monochrome cameras, depth cameras, etc. Furthermore, the camera can be used to acquire environmental images or to acquire images of the user's eyes. For example, in one example, combined with... Figure 1b Taking the glasses shown as an example, at least one camera can be installed on the outside of the glasses to capture environmental images; at the same time, at least one camera can be installed on the inside of the glasses to capture images of the student's eyes.

[0040] One or more sensors 130 may also include at least one microphone, which can collect ambient audio data. For example, during a user's learning process while wearing the wearable device 100, the user can speak directly, and the wearable device 100 can collect the audio data of the user's speech through the microphone. One or more sensors 130 may also include at least one speaker, which can convert the audio data into a speech waveform and play it as a sound that humans can hear. Of course, it is understood that the wearable device 100 may also include other types and numbers of sensors 130, such as an ambient temperature sensor, etc., and this specification does not limit this.

[0041] Communication module 140 refers to the module in wearable device 100 used to establish network connections with other devices or gateways. Communication module 140 can be a wired communication module or a wireless communication module. For example, in some embodiments, communication module 140 is implemented as a wired communication module, which can be a USB (Universal Serial Bus) interface and data cable. In other embodiments, communication module 140 is implemented as a wireless communication module, which can be a Bluetooth, WiFi, or other communication module.

[0042] Wearable device 100 establishes a communicable connection with server 200 through network 400, which can be a local area network or the Internet. For example, server 200 can be a cloud server corresponding to wearable device 100. Wearable device 100 can connect to the Internet through gateway devices such as routers to achieve a communication connection with server 200.

[0043] Server 200 can be a single server or a server cluster. In some implementations, server 200 is a cloud server corresponding to wearable device 100. For example, server 200 can be a cloud platform maintained and operated by the manufacturer of wearable device 100, and wearable device 100 can access the cloud platform network to establish a data communication connection with server 200.

[0044] Server 200 may run server program 210 and database 220. Server program 210 refers to a computer program that implements one or more method steps described below in this specification. For example, server 200 can implement the data processing method described below in this specification through server program 210. Database 220 can be used to store relevant data on the server 200 side, such as user-system dialogue data during the learning process and intermediate data during the operation of server 200.

[0045] User-side device 300 refers to the device used by the client. User-side device 300 can run client programs provided by server 200, thereby accessing server 200. Client programs can be, for example, web pages, client applications, etc., and there are no restrictions on this.

[0046] In the teaching system described in this manual, server 200 is equipped with a dialogue model. The dialogue model is a neural network model based on a large language model, and its fundamental function is to enable human-computer dialogue. In the teaching system, student responses are used as input to the dialogue model. The dialogue model generates corresponding system responses based on the student's responses and outputs them to the student, thereby assisting in a continuous question-and-answer learning process.

[0047] Below, in conjunction with Figure 1b The network architecture of the teaching system shown provides a detailed explanation of the multi-turn dialogue generation process during student learning.

[0048] First, students wear the wearable device 100 before the learning process begins, and then proceed with the learning process. When students encounter problems and need help, they can speak directly: Q: How do I calculate the area of ​​a rectangle? I always forget the area formula. At this time, the wearable device 100 collects the student's audio through a microphone, converts the audio into text data through the processor 110, and then sends the text data to the server 200 via the network 400. On the server 200 side, the text data is used as input to the dialogue model, and the output response of the dialogue model is obtained, for example: {A: The formula for the area of ​​a rectangle is length multiplied by width. For example, a rectangle with a length of 2 meters and a width of 3 meters has an area of ​​2 meters multiplied by 3 meters, which equals 6 square meters.} Then, server 200 returns the response data to wearable device 100 via network 400. After receiving the response data, wearable device 100 converts the text data into audio data and broadcasts it to the student through a speaker, thus completing one round of dialogue.

[0049] In the embodiments described in this specification, a round of dialogue includes one user statement and one system response. For example, the text of a round of dialogue in the previous example is as follows: Q: How do I calculate the area of ​​a rectangle? I always forget the area formula.

[0050] A: The formula for the area of ​​a rectangle is length multiplied by width. For example, if a rectangle is 2 meters long and 3 meters wide, then its area is 2 meters multiplied by 3 meters, which equals 6 square meters. In this example, Q represents a user's message, and A represents a system response. A user's message combined with a system response constitutes one round of dialogue.

[0051] It is understandable that during a learning process, a user may engage in multiple rounds of human-computer dialogue with the teaching system, resulting in multiple rounds of dialogue data. Each round of dialogue data includes a user's question (Q) and a system's response (A).

[0052] Server 200 can store dialogue data from all rounds of the entire learning process. For example, in some implementations, server 200 can store the first session data corresponding to the learning process, which includes the following aspects: (1) Multi-turn dialogue data generated during this learning process. Multi-turn dialogue data can be stored in text form to save storage space.

[0053] (2) Metadata of multi-turn dialogue data. Metadata is data used to describe the attribute information of dialogue data, such as the start / end time of each round of dialogue, the sorting round, the total duration of multi-turn dialogue, etc.

[0054] (3) Interactive events during the learning process. For example, the teaching system can use the sensor 130 of the wearable device 100 and combine it with the corresponding algorithm to detect interactive events of students during the learning process, such as events such as removing the wearable device or events that cause cognitive deadlock.

[0055] Of course, those skilled in the art will understand that the data from a student's first session during a learning process is not limited to the data in the examples above, and may include more or fewer types of data, which this specification does not limit.

[0056] As mentioned above, the dialogue data packets generated by students during the learning process contain a wealth of information, including their learning strategies, knowledge gaps, interests, and real-time emotions. In some embodiments of this specification, data processing methods can be used to mine and extract the distribution of students' interests from the first conversation data of the learning process, identifying the knowledge points and topics that students are interested in, and accurately assessing students' learning status from a process perspective.

[0057] In the embodiments described in this specification, the data processing method can be executed by the server 200, and the data source of the method is the first session data generated during a student's learning process.

[0058] For example, in one exemplary scenario, a student starts learning by wearing wearable device 100 at 18:00 and engages in multiple rounds of dialogue with the teaching system during the learning process until turning off wearable device 100 at 19:30, thus ending the learning process.

[0059] In this example, server 200 can continuously record and store the data generated during the learning process until the learning process ends. The data stored by server 200 during the 90-minute learning process is the first session data described in this specification.

[0060] Figure 2 Flowcharts of data processing methods according to some embodiments of this specification are shown below. Figure 2 Please provide an explanation.

[0061] like Figure 2 As shown, in some embodiments, the data processing methods exemplified in this specification include: S210. For each topic in the current topic sequence, the number of times users actively ask questions, the number of consecutive follow-up questions, and the duration of the topic are weighted and integrated to obtain the interest score of the topic.

[0062] It's understandable that during a single learning session, the student's dialogue with the system might cover different topics. For example, in one instance, the student might ask English questions in the first three rounds, while in the fourth round, the student might ask math questions, thus the first three rounds and the fourth round belong to different subjects. In another example, the student might ask geometry questions in the fourth round, while in the fifth round, the student might ask algebraic equation questions, thus the fourth round and the fifth round belong to different knowledge points within the same subject.

[0063] In the embodiments described in this specification, considering that the first conversation data may contain multiple topics in the multi-turn dialogues, in order to accurately discover the distribution of students' interests, the first conversation data can first be segmented into topics to obtain a topic sequence.

[0064] For example, in one sample, the first session data includes multiple rounds of dialogue, as shown in Table 1 below: Table 1: Multi-turn Dialogue Data In the example in Table 1 above, the first and second rounds are dialogues on the English "past tense" knowledge point, and semantically, they belong to the same topic. The third and fourth rounds are dialogues on the mathematical "area calculation" knowledge point, and semantically, they belong to the same topic. The fifth round is a dialogue on the mathematical "sum of the interior angles of a triangle," and the sixth round is a dialogue on the Chinese "rhetorical devices" knowledge point, thus the fifth and sixth rounds each correspond to different topics.

[0065] After performing topic segmentation on the multi-turn dialogue data in the example in Table 1 above, the resulting topic sequence is shown in Table 2 below: Table 2 Topic Sequence As shown in Table 2, topic segmentation can separate identical or similar dialogue content in multi-turn conversations into the same topic. In the embodiments described in this specification, topic segmentation can be implemented using various algorithms. For example, in some embodiments, it can be based on one or more methods such as semantic jump detection, displaying switching markers, and time interval judgment. These will be explained later in this specification and will not be detailed here.

[0066] In some implementations, the first session data may be preprocessed before topic segmentation of the multi-turn dialogue data included in the first session data, as will be described later in this specification.

[0067] In other implementations, after topic segmentation of the multi-turn dialogue data included in the first session data, a dialogue model can be used to add topic tags to each topic, which are used to describe the knowledge points corresponding to the topic.

[0068] For example, in one exemplary implementation, as shown in Table 2 above, the topic tag “English - Past Tense Grammar” is added to Topic_1, the topic tag “Mathematics - Area Calculation” is added to Topic_2, the topic tag “Mathematics - Interior Angles of a Triangle” is added to Topic_3, and the topic tag “Chinese - Rhetoric” is added to Topic_4 through the dialogue model.

[0069] In this example, by adding topic tags to each topic, a mapping relationship between the dialogue text and standard subject knowledge points can be established, which facilitates subsequent statistical analysis of the interest level corresponding to each knowledge point by tag classification, and provides a standard index basis for cross-conversation knowledge point tracking.

[0070] In the embodiments described in this specification, the distribution of knowledge points of interest to students is identified by analyzing multiple rounds of dialogue during the learning process. In some embodiments, the degree of student interest in a particular knowledge point can be quantified from the following three dimensions: the number of times a student actively asks questions, the number of consecutive follow-up questions, and the percentage of time spent on that topic.

[0071] The number of proactive questions indicates the number of times a user actively asks questions within a given topic. It can be understood that the more questions a student proactively asks within a topic, the higher their initiative and level of interest. Proactive questions are determined when the user's question in that round of dialogue is not prompted by a system response, but rather a new or extended question voluntarily raised by the user. For example, in one instance, if the previous system response was without guidance, and a student proactively asks, "What rhetorical devices are used in this poem?", this counts as one proactive question. Conversely, in another instance, if the previous system response was, "Do you know what rhetorical device is used in the line 'The waterfall plunges three thousand feet'?", and a student proactively asks, "Is it an exaggeration?", this does not count as a proactive question.

[0072] The number of consecutive follow-up questions represents the number of times a user asks follow-up questions on the same topic. It can be understood that the more times a student asks follow-up questions on the same topic or knowledge point, the higher their willingness to explore that topic, and thus the higher their level of interest in it. The method for determining consecutive follow-up questions is that the current round of questions is an extension or refinement of the previous question. For example, in one instance, after the system has finished outputting the previous round, if the user continues to ask, "Are there other solutions to this problem?", it indicates that the user is willing to continue exploring the current topic further, thus counting as one round of consecutive follow-up questions.

[0073] The duration percentage of a topic represents the ratio of its duration to the total duration of the first session data. This means that the duration of a topic can reflect, to some extent, the amount of time students spend learning on that topic; a longer duration indicates more time spent on it, and thus a higher probability of interest in that topic or knowledge point. Combining this with the topic sequence shown in Table 2, we know that each topic has a start and end time, allowing us to calculate its duration. Simultaneously, the metadata of the first session data records the start and end times of the entire learning process, allowing us to calculate the total duration of the entire learning process. Then, by calculating the ratio of each topic's duration to the total duration, we can obtain the duration percentage of each topic.

[0074] In the embodiments described in this specification, the weighted fusion process of the interest score Interest_i for each Topic_i in the topic sequence is expressed as follows: (1) In formula (1), This represents the number of times the i-th topic was asked proactively. This indicates the number of consecutive follow-up questions on the i-th topic. This represents the percentage of time spent on the i-th topic.

[0075] in, These are the weights corresponding to the number of proactive questions, the number of consecutive follow-up questions, and the percentage of time spent asking questions. In some implementations, the weight of each parameter is positively correlated with its priority; that is, the higher the priority, the larger the weight.

[0076] For example, based on pedagogical principles, students' proactive questioning behavior reflects genuine interest more effectively than passive engagement, and continuous questioning is a direct manifestation of curiosity-driven learning. Therefore, in some implementation methods, a weight can be set for the number of proactive questions. The weight of the number of consecutive follow-up questions is greater than the weight of the number of consecutive follow-up questions. Furthermore, the weights of consecutive questioning rounds... Weight greater than the percentage of time ,Right now, .

[0077] For example, in one example, The sum is 1, and the values ​​are respectively... , , In this example, the influence of students' initiative in asking questions on interest is set to the highest level, accurately identifying students' perceived active interest; the influence of the depth of follow-up questions on interest is set to the second highest level, reflecting students' willingness to explore; and the influence of time spent on interest is set to the lowest level, serving as a supplementary reference. This approach effectively improves the identification and analysis of topics and knowledge points that students are interested in.

[0078] In one exemplary implementation, suppose the number of proactive questions for the i-th topic is... Continuous follow-up questions Duration percentage Substituting into the aforementioned formula (1), the interest score for the i-th topic can be calculated. .

[0079] For each topic included in the topic sequence, repeat the above S220~S230 process in sequence to calculate the interest score corresponding to each topic.

[0080] S220. An emotion trend curve is generated based on the emotion score sequence composed of the emotion scores of each round of dialogue data in the topic sequence and the cognitive load value sequence composed of the cognitive load data of each round of dialogue data in the current topic sequence.

[0081] It is worth noting that students' emotional trends during the learning process can reflect emotional changes and help understand the fluctuations in students' learning status. Therefore, in the implementation method of this specification, further data analysis can be performed based on the multi-turn dialogue data included in the first conversation data to explore the emotional trends of students during the learning process. The following section will combine... Figure 3 Please provide an explanation.

[0082] S221. Determine the sentiment score for each round of dialogue data based on the sentiment polarity of user statements in each round of dialogue data.

[0083] In some implementations, the multi-turn dialogues included in the first session data can be described with reference to the example in Table 1 above. First, the sentiment polarity of user statements Q in each round of dialogue data can be calculated to determine the sentiment score for each round of dialogue.

[0084] Emotional polarity refers to whether a student's user statement Q is more positive or negative. In some implementations, emotional polarity can be represented by an emotional score e_text, which ranges from -1 to 1, where -1 represents strong negative emotion, 0 represents neutral emotion, and 1 represents strong positive emotion.

[0085] In some implementations, the quantification of the emotional polarity of user statements can be achieved using an emotional dictionary adapted to educational scenarios.

[0086] For example, a general Chinese sentiment dictionary could be used as a base, including common positive and negative sentiment words. Positive sentiment words could include words like "happy," "great," and "very interesting," while negative sentiment words could include words like "difficult," "annoying," and "disgusting." Each word would also be pre-labeled with its sentiment score and intensity.

[0087] Furthermore, the emotional dictionary can be expanded to include emotional expressions specific to educational scenarios. For example, it could include positive expressions related to learning achievements, including but not limited to "Oh, I see," "I understand," and "That's so easy!" These positive words could have an emotional score of 0.6 to 1.0. Or, it could include negative expressions related to learning frustration, including but not limited to "Too difficult," "Can't do it," "Don't want to do it anymore," and "Can't learn it!" These negative words could have an emotional score of -0.6 to -1.0. Finally, it could include neutral to slightly negative expressions related to cognitive confusion, including but not limited to "Convoluted," "A bit confusing," and "Something's not right," these words could have an emotional score of -0.2 to -0.4.

[0088] In some implementations, context disambiguation rules may be further set to process polysemous words. For example, the phrase "so difficult" in "so difficult but so fun" should be comprehensively determined to be neutral to positive. For another example, the word "cannot" should be processed differently in "I cannot learn it" and "Isn't it too easy?" (neutral rhetorical question). The disambiguation rules may be determined based on the context of multiple preceding and following words, which can be understood by those skilled in the art, and will not be repeated in this specification.

[0089] When calculating the sentiment score for each round of dialogue data through the sentiment lexicon constructed above, the user utterance Q in each round of dialogue may be matched with the words in the sentiment lexicon, and the sentiment score of this round of dialogue is determined according to the sentiment score corresponding to the matched word. For example, in an example, the user utterance Q is "I still cannot learn the quadratic equation with one unknown". Through matching with the sentiment lexicon, it is determined that the user's sentiment polarity tends to be negative, and the corresponding sentiment score may be -0.8.

[0090] According to the above process, the sentiment score e_text of each round of dialogue in the first session data may be determined, then all the sentiment scores of each round of dialogue are collected in the order of rounds, and a sentiment score sequence in the entire learning process is constructed, which is expressed as E_text = {e_text_1, e_text_2, ..., e_text_N}.

[0091] S222. Determine a cognitive load value sequence according to the cognitive load data of the user in each round of dialogue data.

[0092] Cognitive load refers to the total amount of cognitive processing resources that the brain needs to consume during the learning process of a student. The higher the cognitive load, the heavier the brain burden of the student, and the time period with high cognitive load corresponds to the period when the student is anxious or confused. Therefore, by recording the cognitive load data throughout the learning process, the emotional changes of the student during the learning process may also be reflected.

[0093] It can be understood that human cognitive load may be represented by physiological features, such as heart rate, electrocardiogram, eye movement frequency, etc. A wearable device may be used to monitor the physiological feature parameters of the student in real time during the learning process, so as to calculate the student's cognitive load data in real time. The value range of the cognitive load data may be [0,100], and the cognitive load value sequence may be obtained by counting the cognitive load data at a plurality of discrete time points during the learning process. For the algorithm process of calculating the cognitive load data, those skilled in the art can understand and fully implement it by referring to the related art, which will not be repeated in this specification.

[0094] S223. Align the sentiment score sequence and the cognitive load value sequence to the same time axis to generate an emotion trend curve.

[0095] It can be understood that the emotional score sequence represents the changes in students' emotional polarity during the learning process, while the cognitive load sequence represents the changes in students' cognitive load during the learning process. Therefore, the two correspond to data from the same time period. By aligning the emotional score sequence and the cognitive load sequence along the time axis, we can obtain the emotional trend curve of students throughout the learning process.

[0096] In some implementations, the emotion trend curve includes an emotion score curve and a cognitive load score curve. For example, in one example, the emotion trend curve might look like this: Figure 4 As shown, Figure 4 The middle horizontal axis represents time T, which ranges from the start time to the end time of the learning process; the left vertical axis represents the emotional score E, which ranges from [-1,1]; the right vertical axis represents the cognitive load value CL, which ranges from [0,100].

[0097] See Figure 4 The diagram shows the emotion score curve represented by a solid line. The peak of the curve represents the moment when students exhibit positive emotions, and the trough represents the moment when students exhibit negative emotions. Each inflection point on the curve represents a change in the student's emotion. The cognitive load curve is represented by a dashed line. The peak represents the moment when students have a high cognitive load, and the trough represents the moment when students have a low cognitive load.

[0098] In some implementations, high cognitive load intervals can be marked on the emotion trend curve. These intervals represent the time range during which a student's cognitive load exceeds a preset threshold, i.e., the time range that triggers cognitive deadlock. The preset threshold can be set according to specific needs and is not limited thereto. For example, in one example, the preset threshold could be 80, indicating that a cognitive load value greater than 80 indicates that a student has triggered cognitive deadlock.

[0099] S230, Output the topic interest score or the topics of interest determined from the topic sequence based on the topic interest score, and the sentiment trend curve.

[0100] As mentioned above, the interest score reflects the student's level of interest in each topic; a higher interest score indicates a greater level of interest. Therefore, in this embodiment, a preset number of topics of interest can be selected from the multiple topics included in the topic sequence based on the interest score for each topic.

[0101] For example, in some implementations, all topics can be sorted in descending order of interest score, and then the top K topics are selected as the topics of interest to students. It is worth noting that the value of K can be set according to specific scenario requirements, and this specification does not impose any restrictions on it.

[0102] For example, in one example, K can be a fixed value, such as K=3, which means selecting the top 3 topics with the highest interest scores as topics of interest. In another example, the value of K can be dynamically determined based on the number of topics. For example, when the total number of topics N≤5, K=min(3,N), and when the total number of topics N>5, K=min(5,ceil(N×0.3)), that is, at most 30% of the total number of topics and no more than 5 topics.

[0103] In some implementations, after obtaining topics of interest, the server can output them to a local database for storage, awaiting subsequent method calls. For example, the server can further generate an analysis report reflecting the learning process based on the topics of interest stored in the database, which includes the topics of interest for the student during the learning process. In other implementations, after obtaining topics of interest, the server can directly output them to the user's device, allowing teachers and / or parents to view the distribution of students' interests during the learning process. In still other implementations, the server can also directly output the interest score for each topic to a database for storage.

[0104] Meanwhile, in the embodiments of this specification, after generating the emotion trend curve, the server can output the emotion trend curve to a local database for storage, awaiting subsequent method calls. For example, the server can further generate an analysis report reflecting the learning process based on the emotion trend curve stored in the database, which includes the emotion trend curve during this learning process. In other embodiments, after obtaining the emotion trend curve, the server can directly output the emotion trend curve to the user's terminal device, allowing teachers and / or parents on the user's terminal device to view the student's emotional changes during this learning process. As can be seen from the above, in the embodiments of this specification, by analyzing the human-computer dialogue data during the student's learning process and quantifying user interests and emotional fluctuations across multiple data dimensions, it is possible to fully explore the distribution of students' interests and emotional changes in the process dimension. Moreover, by segmenting multi-turn dialogues into topics, scattered dialogues are effectively summarized into topic units, facilitating accurate quantification and identification of user interest in different knowledge points. Combining multiple dimensions such as proactive questioning, continuous follow-up questions, and duration proportions to quantify topic interest aligns with educational principles, comprehensively and accurately assessing the distribution of students' interests. By analyzing emotional trends across the dual dimensions of cognitive load and affective polarity, this approach accurately captures students' latent states of learning frustration, confusion, and cognitive impasses, providing educators with a direct and intuitive understanding of students' real-time psychological states and periods of difficulty. Compared to outcome indicators such as exam scores and subjective evaluations by educators, this solution automatically quantifies and identifies students' genuine interests based on dialogue data. The assessment results are objective and timely, providing a reliable basis for personalized supplementary education.

[0105] In some implementations, the server can trigger the execution of the above-described method flow at the end of each learning process, thereby analyzing and evaluating the learning process in real time.

[0106] For example, in one possible implementation, when the user finishes learning and turns off the wearable device, the wearable device can send a trigger signal to the server. When the server receives the trigger signal, it can confirm that the current learning process has ended, and then execute the above-described method flow based on the first session data corresponding to the recorded learning process.

[0107] For example, in another possible implementation, the wearable device can detect the wearing status. If it detects that the user has removed the wearable device and has not worn it for a period of time (which can be set, such as 15 minutes), it can determine that the user has ended the learning process and send a trigger signal to the server.

[0108] For example, in another possible implementation, the server can determine the end of the learning process when it detects an explicit end instruction from the user during the dialogue. For example, if the user says, "Today's learning is over, thank you for your help.", the server can determine the end of the current learning process based on the user's statement, and then begin to execute the above-described method flow.

[0109] In the embodiments described in this specification, the above-mentioned data processing method can be automatically triggered at the end of the session. Through the asynchronous architecture of the learning process and the data analysis process, the data analysis and teaching assistance are decoupled, without affecting the user's learning process. Moreover, the learning process can be evaluated and analyzed in a timely manner at the end of the learning process, improving timeliness and helping educators to identify problems and adjust teaching strategies in a timely manner.

[0110] In some implementations, after acquiring the first session data of the learning process, the server can preprocess the first session data. For example, in one instance, the server can perform text cleaning on the multi-turn dialogue data included in the first session data, such as removing irrelevant content like system instruction markers, format control characters, and greetings from the dialogue text. Data preprocessing can filter out noise and duplicate data, improving data accuracy.

[0111] After preprocessing, topic segmentation can be performed on the multi-turn dialogue data included in the first session data. For example, in some implementations, the topic segmentation process includes at least one of the following: Determine the semantic similarity between two adjacent rounds of dialogue data, and segment the two adjacent rounds of dialogue data if the semantic similarity is lower than a first threshold; If a user's speech in a round of dialogue data contains preset segmentation keywords, the dialogue data in that round is segmented from the previous round of dialogue data. If the time interval between two adjacent rounds of dialogue data exceeds a second threshold, the two adjacent rounds of dialogue data are segmented.

[0112] For example, in one exemplary implementation, the semantic similarity of adjacent dialogue data can be calculated separately, and the semantic similarity can be represented using cosine similarity. If the cosine similarity of adjacent dialogues is greater than or equal to a first threshold, it indicates that the content of the adjacent dialogues is highly related and belongs to the same topic. Conversely, if the cosine similarity of adjacent dialogues is lower than the first threshold, it indicates that the content of the adjacent dialogues is less related, thus allowing the two dialogue data to be segmented. The specific value of the first threshold can be selected according to requirements; for example, in one example, the first threshold can be 0.8.

[0113] For example, in another exemplary implementation, it is possible to detect whether the user's speech Q in each round of dialogue data contains preset segmentation keywords. Preset segmentation keywords represent keywords for topic switching, such as "Let's try a different question" or "Let's skip math and look at English instead." Through semantic recognition, when the system recognizes preset segmentation keywords, it can perform topic segmentation between the current round of dialogue and the previous round of dialogue.

[0114] For example, in another exemplary implementation, topic segmentation can be determined based on the time interval between two adjacent rounds of dialogue. The time interval between two rounds of dialogue refers to the duration between the end time of the previous round and the start time of the current round. If the time interval is less than or equal to a second threshold, it indicates that the two rounds of dialogue are short and belong to the same topic, thus topic segmentation is unnecessary. Conversely, if the time interval exceeds the second threshold, it indicates that the two rounds of dialogue are long and belong to different topics, thus topic segmentation can be performed. The specific value of the second threshold can be selected according to requirements and is not limited thereto.

[0115] It is understood that the methods of topic segmentation are not limited to the three examples above, and those skilled in the art can use other segmentation methods. Moreover, the various topic segmentation methods described above can be used individually or in combination, and this specification does not impose any restrictions on this.

[0116] Combining the examples in Tables 1 and 2 above, a topic sequence can be obtained by segmenting the multi-turn dialogue data included in the first session data into topics. The topic sequence includes multiple topics, each containing one or more turns of dialogue data. After obtaining the topic sequence, refer to the aforementioned... Figure 2 The method and process shown are used to quantitatively analyze the distribution of students' interests and changes in their emotions.

[0117] In some implementations, after obtaining the topic sequence, data analysis can be performed on the topic sequence to identify weak points in the learning process. Weak points refer to topics and knowledge points that students encounter difficulties understanding during the learning process. The following section will combine... Figure 5 Please provide an explanation.

[0118] like Figure 5 As shown, in some embodiments, the data processing method exemplified in this specification includes: S510. For each topic in the topic sequence, the weighted fusion of the user's question rounds, keyword rounds, and semantic deviation scores in that topic yields a comprehension deviation score.

[0119] S520. The number of repeated questions, the number of cognitive deadlocks, and the aforementioned comprehension deviation scores are weighted and integrated to obtain the weakness score for each topic.

[0120] S530, Output the weakness score of the topic or determine the weak topic from the topic sequence based on the weakness score of the topic.

[0121] In this embodiment, by analyzing multiple rounds of dialogue during the learning process, the weak points in students' understanding are identified. In some embodiments, the student's weak points in a certain knowledge point can be quantified from the following three dimensions: the number of repeated questions, the number of cognitive deadlocks, and the score for comprehension deviation.

[0122] The number of repeated questions indicates the number of times a user asks the same or similar questions repeatedly in multiple rounds of conversation. For example, in one round of conversation, a user asks, "How do you calculate the area of ​​a rectangle that is 2 meters long and 3 meters wide?", and in another round, the user asks, "How do you calculate the area of ​​a rectangle that is 1.5 centimeters long and 3 centimeters wide?". Since the questions asked by the user in these two rounds are similar, it can be counted as one repeated question.

[0123] In some embodiments of this specification, the semantic similarity between user statements in two rounds of dialogue can be calculated to determine whether the user statements constitute a duplicate question. For example, in one example, cosine similarity can be used to represent the semantic similarity between user statements in two rounds of dialogue. If the cosine similarity is less than or equal to a third threshold, it indicates that the semantic similarity between the user statements in two rounds of dialogue is low, and therefore it is not counted as a duplicate question. Conversely, if the cosine similarity is greater than the third threshold, it indicates that the semantic similarity between the user statements in two rounds of dialogue is high, constituting a duplicate question, and is therefore counted as one duplicate question. The specific value of the second threshold can be selected according to requirements and is not limited thereto.

[0124] Using the above method, the number of repeated questions for each topic can be calculated. For example, the number of repeated questions for the i-th topic is represented as... .

[0125] Cognitive impasse refers to a situation where students encounter problems they find difficult to understand but do not actively seek help. The occurrence of cognitive impasse indicates that students have difficulty understanding a certain knowledge point, reflecting their weakness in that knowledge point.

[0126] In the embodiments described in this specification, the teaching system can monitor the student's cognitive load status in real time during the learning process, thereby determining whether the student has entered a cognitive deadlock, and recording the time and number of times a cognitive deadlock occurs during the learning process. For example, the number of times a cognitive deadlock occurs for the i-th topic can be represented as... It is understood that cognitive impasses exhibit relatively obvious physiological characteristics, such as increased heart rate and accelerated eye movements. In some implementations, wearable devices can be used to detect changes in physiological parameters such as heart rate and eye movement frequency in students, thereby achieving real-time monitoring of cognitive impasses. Those skilled in the art can understand and fully implement this by referring to relevant technologies, and this specification will not elaborate further.

[0127] The comprehension deviation score indicates the degree of deviation in a student's understanding of a particular topic or knowledge point; a higher score indicates a lower level of understanding. In some implementations, the comprehension deviation score can be quantified from three dimensions: the percentage of question rounds, the percentage of keyword rounds, and the semantic deviation score.

[0128] The question-repeat ratio represents the ratio of the number of times a student repeatedly asks the same question within a topic to the total number of dialogue rounds for that topic. For example, in the i-th topic, in the first round of dialogue, a user asks, "Why can't the denominator of a fraction be zero?" After the system responds, the user continues in the next round, "I still don't understand why a denominator of zero is meaningless?" In these two rounds, the user asks the same question, so the number of times the user asks the same question is counted as 2. If the topic includes a total of 4 rounds of dialogue, then the question-repeat ratio R_repeat_i = 2 / 4 = 0.5.

[0129] The keyword round percentage represents the ratio of the number of times a user mentions Category I keywords in a topic to the total number of dialogue rounds in that topic. Category I keywords are markers of user confusion directly contained in the user's statements. For example, in one scenario, Category I keywords include, but are not limited to, "still don't understand," "don't get it," and "what do you mean?" For instance, in a scenario where, in the i-th topic, the number of rounds in which the user's statements contain Category I keywords is 1, and the topic includes a total of 4 dialogue rounds, then the keyword round percentage R_confusion_i = 1 / 4 = 0.25.

[0130] The semantic deviation score represents the semantic deviation between the user's answer and the system's expected answer after the system responds to a question. For example, in a previous dialogue, the system responded, "Now that you understand how to calculate the area of ​​a rectangle, let's try it: What is the area of ​​a rectangle that is 2 meters long and 3 meters wide?" In this dialogue, the system responded with the question, and simultaneously generated the expected answer, "The area is 2 meters multiplied by 3 meters, which equals 6 square meters." In the current dialogue, the user's response is based on the question posed in the previous dialogue. The system then obtains the user's response as the answer, calculates the semantic similarity between the user's answer and the expected answer, and determines the semantic deviation score based on this similarity. In some implementations, the cosine similarity between the user's answer and the expected answer can be calculated, and then 1 can be subtracted from the cosine similarity to obtain the semantic deviation score, denoted as R_deviation_i.

[0131] In some embodiments of this specification, the process of weighted fusion of the proportion of question rounds R_repeat_i, the proportion of keyword rounds R_confusion_i, and the semantic deviation score R_deviation_i is expressed as follows: (2) In formula (2), This indicates the score for comprehension deviation. These represent the weights corresponding to the percentage of question rounds (R_repeat_i), the percentage of keyword rounds (R_confusion_i), and the semantic deviation score (R_deviation_i), respectively. The weight of each parameter is positively correlated with its priority; that is, the higher the priority, the larger the weight. For example, in one example... The sum is 1, where, , , .

[0132] The number of repeated questions is determined through the above method. Number of cognitive deadlocks And the score for understanding the deviation Then, by weighting and fusing them, we can obtain the weakness score of the i-th topic, which is represented as: (3) In formula (3), This represents the score for the weakness in the i-th topic. The number of repeated questions Number of cognitive deadlocks And the score for understanding the deviation The weights can be predetermined. For example, in some implementations, cognitive deadlock is the strongest signal of difficulty in understanding, and therefore can be given a high weight. Repeated questioning receives the next highest weight, while the comprehension deviation score, as a supplementary dimension, has the lowest weight. For example, in one example… The sum is 1, where, , , .

[0133] Through the above process, the weakness score is calculated for each topic in the topic sequence. For example, in one example, assume the number of times the i-th topic is asked repeatedly. Number of cognitive deadlocks Comprehension deviation score Then, the weakness score of the topic can be calculated according to formula (3). =0.3×2+0.45×1+0.25×0.7=1.225.

[0134] As mentioned above, the weakness score reflects a student's level of understanding of each topic; a higher weakness score indicates a weaker understanding of the topic. Therefore, in the implementation method of this specification, a preset number of weakness topics can be selected from multiple topics included in the topic sequence based on the weakness score of each topic.

[0135] For example, in one scenario, all topics can be sorted in descending order of their weakness scores, and then the top K topics can be selected as the students' weak points. The value of K can be determined by referring to the process for selecting topics of interest mentioned earlier, and will not be repeated here.

[0136] In some implementations, after obtaining the weakness topics, the server can output similar weakness topics to a local database for storage, awaiting subsequent method calls. For example, the server can further generate an analysis report reflecting the learning process based on the weakness topics stored in the database, which includes the student's weakness topics during this learning process. In other implementations, after obtaining the weakness topics, the server can directly output them to the user's device, allowing teachers and / or parents to view the distribution of the student's weaknesses during this learning process. In still other implementations, the server can also directly output the weakness score for each topic to the database for storage.

[0137] As can be seen from the above, in the embodiments of this specification, by analyzing the human-computer dialogue data during the student's learning process, it is possible not only to discover the distribution of students' interests and emotional trends during the learning process, but also to effectively identify weak knowledge points. Furthermore, by combining multiple dimensions such as repeated questioning, cognitive deadlock monitoring, and quantification of comprehension level, weak knowledge points that students find difficult to understand can be discovered, thus comprehensively and accurately assessing the student's learning process and providing a reliable basis for targeted auxiliary education.

[0138] In some implementations, after identifying the topics of interest and weaknesses in the learning process through the above-described methods, a corresponding natural language description can be generated for each topic. For example, for a topic of interest, the generated natural language description might be, "Today I showed great interest in insect-related topics, actively asked questions 5 times, and conducted 3 rounds of in-depth follow-up questions on 'the structure of butterfly wings'." For a topic of weakness, the generated natural language description might be, "I repeatedly encountered difficulties understanding fraction division-related problems, asked questions twice, triggered a cognitive deadlock once, and it is recommended to focus on reviewing these topics." In one possible implementation, the server can utilize a dialogue model to generate natural language descriptions for each topic of interest and / or topic of weakness, which will not be elaborated further.

[0139] In some implementations, after generating the emotion trend curve through the above-described process, a natural language description of the emotion trend curve can be further generated. For example, in one example, the natural language description generated based on the emotion trend curve could be: "The emotions were positive in the first half of this learning session, but the emotions dropped significantly when discussing mathematical fraction division (triggering cognitive deadlock), and the emotions gradually recovered after being inspired and guided by the system."

[0140] In some implementations, the server can generate feedback results based on at least one of the aforementioned topics of interest, topics of weakness, and sentiment trend curves, and send the feedback results to the user's terminal device so that educators (such as teachers and parents) can view the feedback results on the user's terminal device.

[0141] In some possible implementations, the feedback can be a structured analysis report, which is used to summarize and analyze the learning process.

[0142] In some implementations, a structured analysis report may include at least one of the following components: (1) Session Summary This can include the date, duration (such as start time, end time, etc.), and number of dialogue rounds for this learning process.

[0143] It can also include the knowledge areas and the number of topics involved in this learning process. Knowledge areas can be represented by the aforementioned topic tags.

[0144] It can also include a comprehensive evaluation, such as a sentence or paragraph summarizing the learning status. The comprehensive evaluation can be given by the dialogue model based on the data analysis results mentioned above.

[0145] (2) Analysis of points of interest This can include topics of interest during this learning process, as well as natural language descriptions for each topic. Multiple topics of interest can be sorted from highest to lowest according to their interest scores.

[0146] It can also generate radar charts for multiple topics of interest based on interest scores, and display the radar charts in the analysis report, providing visualization of interest distribution.

[0147] (3) Weakness Analysis This can include the weak points in this learning process, as well as the natural language descriptions for each weak point. Multiple weak points can be sorted from highest to lowest according to their scores.

[0148] It can also include an assessment of the mastery of knowledge points corresponding to each weak point topic. For example, a mastery rating can be given for the knowledge points corresponding to each weak point topic, and the rating can include multiple levels such as good, average, and weak.

[0149] (4) Sentiment Trend Analysis This can include the aforementioned sentiment trend curve.

[0150] You can also mark key nodes on the sentiment trend curve, such as sentiment highs, lows, turning points, and high-load intervals.

[0151] It can also include natural language descriptions corresponding to the sentiment trend curve.

[0152] (5) Personalized suggestions Based on the analysis of topics of interest, weak points, and sentiment trends, the system can use a dialogue model to generate specific learning suggestions, such as recommending knowledge points to be strengthened, directions to be explored in depth, and suggestions on learning methods.

[0153] As can be seen from the above, the implementation method of this manual provides users with an intuitive and comprehensive analysis report by integrating data on topics of interest, weak points, and emotional trends. This allows teachers and parents to fully understand the duration of the student's learning process, the knowledge points involved, the distribution of interests, knowledge gaps, and emotional fluctuations. At the same time, it provides targeted suggestions and learning plans, providing comprehensive quantitative data support for personalized tutoring.

[0154] In some implementations, the server can store the feedback results of each learning process and generate continuous tracking assistance results based on multiple consecutive feedback results. For example, in one example, tracking assistance results can be generated based on the changing trends of points of interest, weaknesses, and emotional fluctuations in multiple feedback results.

[0155] It is understandable that after each learning process, the server obtains the feedback result corresponding to this learning process based on the aforementioned method and stores it in the database. After accumulating the feedback results of M learning processes in the database, a comprehensive analysis can be performed based on these M feedback results to further explore the student's changing trends over these M learning processes. The value of M can be set according to requirements, and this manual does not impose any restrictions on it. For example, in one example, M=5.

[0156] In some implementations, the changing trend of a user's interests can be determined based on the topics of interest in multiple feedback results. For example, in one instance, the topics of interest included in each feedback result can be extracted, and the changing process of the student's topics of interest can be analyzed based on the generation time of the feedback results, and a corresponding trend of interest change can be generated based on the changing process. For instance, in multiple learning processes, the student's topics of interest gradually changed from mathematical knowledge points to insect-related topics, so the trend of interest change can be described as "the student's interests gradually shifted from mathematical topics to insect-related topics during this period".

[0157] In some implementations, it's possible to determine whether a user's weaknesses have improved based on the weakness topics identified in multiple feedback results. For example, in one instance, the weakness topics included in each feedback result can be extracted, and the same weakness topics across multiple feedback results can be tracked. Based on the changes in the weakness scores for these topics, it can be determined whether the student's weaknesses have improved, thus generating a weakness trend. For instance, over multiple learning sessions, if a student's weakness score for "fraction division" decreases from 0.82 to 0.45, the weakness trend could be described as "Compared to last week, the student's weakness score for fraction division decreased from 0.82 to 0.45, showing significant improvement."

[0158] In some implementations, the trend of a user's emotional fluctuations can be determined based on the emotional trend curves in multiple feedback results. For example, in one instance, the emotional changes during each learning process can be analyzed based on the emotional trend curve in each feedback result to obtain the emotional fluctuation trend. For instance, if the student's average emotional value and fluctuation amplitude show significant improvement during multiple learning processes, the emotional fluctuation trend can be described as "Compared to last week, the student's average emotional value increased by 30% and the fluctuation amplitude decreased by 20% during the learning period, indicating a significant improvement in emotional state."

[0159] By combining the trends in changes in interests, areas of weakness, and emotional fluctuations mentioned above, tracking auxiliary results can be generated. These results can be trend description reports, used to describe the student's changing trends over multiple learning sessions. For example, one trend description might include: "Compared to last week, the student's score on fraction division decreased significantly from 0.82 to 0.45. Interest in natural science topics continued to grow, especially in insect-related topics. Moreover, the student's mood improved significantly during the learning period, and their learning state was good."

[0160] Combination Figure 1b As shown, after generating the tracking assistance results, the server can send the tracking assistance results to the user's terminal device via the network. This allows teachers or parents on the user's terminal device to view the feedback results of a single learning process, as well as the tracking assistance results of multiple consecutive learning processes, and to understand the student's learning progress trends in a timely manner.

[0161] As can be seen from the above, in the embodiments of this specification, by tracking students' multiple learning processes and generating tracking assistance results across conversational data, educators can intuitively and timely observe the long-term changing trends of students' learning processes, such as trends in changes in interest points, trends in improvement of weak points, and trends in changes in emotions during the learning process, rather than only seeing isolated single assessment results, thus providing a continuous and reliable basis for long-term personalized teaching assistance.

[0162] In some embodiments, this specification provides a data processing apparatus, such as... Figure 6 As shown, the device includes: The interest point identification module 10 is configured to, for each topic in the current topic sequence, perform a weighted fusion of the number of times a user actively asks questions, the number of consecutive follow-up questions, and the duration percentage of the topic in that topic to obtain an interest score for that topic; the topic sequence is obtained by segmenting multi-round dialogue data between the user and the teaching system, and each topic in the topic sequence includes at least one round of dialogue data; the number of times a user actively asks questions represents the number of times the user actively initiates questions in that topic, the number of consecutive follow-up questions represents the number of times the user continuously asks follow-up questions on the same topic, and the duration percentage represents the ratio of the duration of that topic to the total duration of the multi-round dialogue; wherein, the weights of the number of times a user actively asks questions, the number of consecutive follow-up questions, and the duration percentage of the topic are positively correlated with the priorities of the number of times a user actively asks questions, the number of consecutive follow-up questions, and the duration percentage, respectively; the interest score of any topic is used to determine topics of interest from the topic sequence. The emotion recognition module 20 is configured to generate an emotion trend curve based on an emotion score sequence composed of emotion scores of each round of dialogue data in the topic sequence and a cognitive load value sequence composed of cognitive load data of each round of dialogue data in the current topic sequence; the emotion trend curve includes a cognitive load value curve, the cognitive load value curve includes one or more high load intervals, and the high load intervals represent the time intervals in which the cognitive load value is greater than a preset threshold. The result feedback module 30 is configured to output the interest score of the topic or the topics of interest determined from the topic sequence based on the interest score of the topic, as well as the emotion trend curve, in order to evaluate the user's learning situation.

[0163] In some possible implementations, the weight of the number of proactive questions is greater than the weight of the number of consecutive follow-up questions, and / or the weight of the number of consecutive follow-up questions is greater than the weight of the duration percentage.

[0164] In some possible implementations, the emotion recognition module is configured as follows: Based on the emotional polarity of user statements in each round of dialogue data, an emotional score is determined for each round of dialogue data, and the emotional score sequence includes the emotional scores for each round of dialogue data; Based on the cognitive load data of the user in each round of dialogue, determine the sequence of cognitive load values; Align the emotion score sequence and the cognitive load value sequence to the same time axis to generate the emotion trend curve.

[0165] In some possible implementations, the data processing apparatus further includes a vulnerability identification module, configured to: For each topic in the topic sequence, a weighted fusion of the user's question round percentage, keyword round percentage, and semantic deviation score in that topic is used to obtain a comprehension deviation score. The question round percentage represents the ratio of the number of times a user asks the same question in that topic to the total number of dialogue rounds in that topic. The keyword round percentage represents the ratio of the number of times a user mentions the first type of keyword in that topic to the total number of dialogue rounds in that topic. The semantic deviation score is determined based on the semantic deviation between the user's answer and the system's expected answer in that topic. The weakness score for each topic is obtained by weighting and fusing the number of repeated questions, the number of cognitive deadlocks, and the comprehension deviation score. The number of repeated questions refers to the number of times a user asks the same question repeatedly in this topic and other topics. The number of cognitive deadlocks refers to the number of times a user experiences a cognitive deadlock during the duration of this topic. Output the weakness score of the topic or determine the weak topic from the topic sequence based on the weakness score of the topic.

[0166] In some possible implementations, the result feedback module is configured as follows: Feedback results are generated based on at least one of the topics of interest, topics of weakness, and emotional trend curves, and the feedback results are used to represent the user's learning status during the duration of the multi-round dialogue. The feedback result is sent to the user terminal device so that the feedback result is output on the user terminal device.

[0167] In some possible implementations, the multi-turn dialogue data represents the data generated by the user during a single learning process, and the result feedback module is configured to: Based on the trends of changes in points of interest, weaknesses, and emotional fluctuations from multiple feedback results, a tracking assistance result is generated and sent to the user terminal device for output on the user terminal device; The multiple feedback results are obtained based on the dialogue data of the user's continuous learning process. The trend of interest changes is obtained based on the changes in topics of interest in the multiple feedback results. The trend of weakness changes is obtained based on the changes in topics of weakness in the multiple feedback results. The trend of emotion fluctuation changes is obtained based on the changes in the emotion trend curve in the multiple feedback results.

[0168] In some possible implementations, the apparatus further includes a topic segmentation module configured to: acquire multi-turn dialogue data between the user and the teaching system, and perform topic segmentation on the multi-turn dialogue data to obtain the topic sequence, wherein the topic segmentation process includes at least one of the following: Determine the semantic similarity between two adjacent rounds of dialogue data, and segment the two adjacent rounds of dialogue data if the semantic similarity is lower than a first threshold; If a user's speech in a round of dialogue data contains preset segmentation keywords, the dialogue data in that round is segmented from the previous round of dialogue data. If the time interval between two adjacent rounds of dialogue data exceeds a second threshold, the two adjacent rounds of dialogue data are segmented.

[0169] In some embodiments, this specification provides a teaching system, including: Wearable devices are used to collect user speech data during the learning process; A server, communicatively connected to the wearable device, is used to receive user speech data sent by the wearable device and return system response data to the wearable device, wherein the server is configured to perform the method described in any of the above embodiments.

[0170] In some embodiments, this specification provides a storage medium storing computer instructions for implementing the methods described in any of the above embodiments.

[0171] In some embodiments, this specification provides a computer program product for causing a computer to perform the methods described in any of the above embodiments.

[0172] Figure 7 A schematic diagram of the structure of an electronic device 600 suitable for implementing the methods of this specification is shown. Figure 7 The architecture shown can realize the corresponding functions of the above-mentioned server and storage media.

[0173] like Figure 7 As shown, the electronic device 600 includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a memory 602 or a program loaded into the memory 602 from a storage section 608. The memory 602 also stores various programs and data required for the operation of the electronic device 600. The processor 601 and the memory 602 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0174] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0175] In particular, according to embodiments of this disclosure, the above-described method process can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing the above-described methods. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611.

[0176] Obviously, the above embodiments are merely examples for clear illustration and are not intended to limit the embodiments. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all embodiments here. However, obvious variations or modifications derived therefrom remain within the scope of protection created by this specification.

Claims

1. A data processing method, characterized in that, include: For each topic in the current topic sequence, the number of times the user actively asks questions, the number of consecutive follow-up questions, and the duration of the topic are weighted and integrated to obtain the interest score of the topic; the topic sequence is obtained by segmenting the multi-round dialogue data between the user and the teaching system, and each topic in the topic sequence includes at least one round of dialogue data. The number of proactive questions indicates the number of times the user proactively asks questions in this topic; the number of consecutive follow-up questions indicates the number of times the user continuously asks follow-up questions on the same topic; and the duration percentage indicates the ratio of the duration of this topic to the total duration of the multi-round dialogue. The weights of the number of proactive questions, the number of consecutive follow-up questions, and the duration percentage are positively correlated with their respective priorities. The interest score for any topic is used to determine topics of interest from the topic sequence. An emotion trend curve is generated based on the emotion score sequence composed of the emotion scores of each round of dialogue data in the topic sequence and the cognitive load value sequence composed of the cognitive load data of each round of dialogue data in the current topic sequence. The emotion trend curve includes a cognitive load value curve, which includes one or more high load intervals, and the high load intervals represent the time intervals in which the cognitive load value is greater than a preset threshold. Output the topic interest score or the topics of interest determined from the topic sequence based on the topic interest score, and the sentiment trend curve, to evaluate the user's learning progress.

2. The data processing method according to claim 1, characterized in that, The weight of the number of proactive questions is greater than the weight of the number of consecutive follow-up questions, and / or the weight of the number of consecutive follow-up questions is greater than the weight of the duration percentage.

3. The data processing method according to claim 1, characterized in that, The generation of an emotion trend curve, based on the emotion score sequence composed of the emotion scores of each round of dialogue data in the topic sequence and the cognitive load value sequence composed of the cognitive load data of each round of dialogue data in the current topic sequence, includes: Based on the emotional polarity of user statements in each round of dialogue data, an emotional score is determined for each round of dialogue data, and the emotional score sequence includes the emotional scores for each round of dialogue data; Based on the cognitive load data of the user in each round of dialogue, determine the sequence of cognitive load values; Align the emotion score sequence and the cognitive load value sequence to the same time axis to generate the emotion trend curve.

4. The data processing method according to claim 1, characterized in that, Also includes: For each topic in the topic sequence, a weighted fusion of the user's question round percentage, keyword round percentage, and semantic deviation score in that topic is used to obtain a comprehension deviation score. The question round percentage represents the ratio of the number of times a user asks the same question in that topic to the total number of dialogue rounds in that topic. The keyword round percentage represents the ratio of the number of times a user mentions the first type of keyword in that topic to the total number of dialogue rounds in that topic. The semantic deviation score is determined based on the semantic deviation between the user's answer and the system's expected answer in that topic. The weakness score for each topic is obtained by weighting and fusing the number of repeated questions, the number of cognitive deadlocks, and the comprehension deviation score. The number of repeated questions refers to the number of times a user asks the same question repeatedly in this topic and other topics. The number of cognitive deadlocks refers to the number of times a user experiences a cognitive deadlock during the duration of this topic. Output the weakness score of the topic or determine the weak topic from the topic sequence based on the weakness score of the topic.

5. The data processing method according to any one of claims 1 to 4, characterized in that, Also includes: Feedback results are generated based on at least one of the topics of interest, topics of weakness, and emotional trend curves, and the feedback results are used to represent the user's learning status during the duration of the multi-round dialogue. The feedback result is sent to the user terminal device so that the feedback result is output on the user terminal device.

6. The data processing method according to claim 5, characterized in that, The multi-turn dialogue data represents the data generated by the user during a single learning process, and the method further includes: Based on the trends of changes in points of interest, weaknesses, and emotional fluctuations from multiple feedback results, a tracking assistance result is generated and sent to the user terminal device for output on the user terminal device; The multiple feedback results are obtained based on the dialogue data of the user's continuous learning process. The trend of interest changes is obtained based on the changes in topics of interest in the multiple feedback results. The trend of weakness changes is obtained based on the changes in topics of weakness in the multiple feedback results. The trend of emotion fluctuation changes is obtained based on the changes in the emotion trend curve in the multiple feedback results.

7. The data processing method according to claim 1, characterized in that, The topic sequence is obtained by acquiring multi-turn dialogue data between the user and the teaching system, and performing topic segmentation on the multi-turn dialogue data to obtain the topic sequence, wherein the topic segmentation process includes at least one of the following: Determine the semantic similarity between two adjacent rounds of dialogue data, and segment the two adjacent rounds of dialogue data if the semantic similarity is lower than a first threshold; If a user's speech in a round of dialogue data contains preset segmentation keywords, the dialogue data in that round is segmented from the previous round of dialogue data. If the time interval between two adjacent rounds of dialogue data exceeds a second threshold, the two adjacent rounds of dialogue data are segmented.

8. A data processing apparatus, characterized in that, include: The interest point identification module is configured to perform a weighted fusion of the number of times the user actively asks questions, the number of consecutive follow-up questions, and the duration of the topic for each topic in the current topic sequence, to obtain the interest score for that topic; the topic sequence is obtained by segmenting the multi-round dialogue data between the user and the teaching system, and each topic in the topic sequence includes at least one round of dialogue data. The number of proactive questions indicates the number of times the user proactively asks questions in this topic; the number of consecutive follow-up questions indicates the number of times the user continuously asks follow-up questions on the same topic; and the duration percentage indicates the ratio of the duration of this topic to the total duration of the multi-round dialogue. The weights of the number of proactive questions, the number of consecutive follow-up questions, and the duration percentage are positively correlated with their respective priorities. The interest score for any topic is used to determine topics of interest from the topic sequence. The emotion recognition module is configured to generate an emotion trend curve based on the emotion score sequence composed of the emotion scores of each round of dialogue data in the topic sequence and the cognitive load value sequence composed of the cognitive load data of each round of dialogue data in the current topic sequence. The emotion trend curve includes a cognitive load value curve, and the cognitive load value curve includes one or more high load intervals, and the high load intervals represent the time intervals in which the cognitive load value is greater than a preset threshold. The results feedback module is configured to output the topic interest score or the topics of interest determined from the topic sequence based on the topic interest score, as well as the sentiment trend curve, to evaluate the user's learning progress.

9. A teaching system, characterized in that, include: Wearable devices are used to collect user speech data during the learning process; The server is communicatively connected to the wearable device and is used to receive user speech data sent by the wearable device and return system response data to the wearable device. The server is configured to perform the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, The device stores computer instructions for implementing the method as described in any one of claims 1 to 7.

Images