Auxiliary learning method and device, related equipment and computer program product

Abstract

The invention discloses an auxiliary learning method and device, related equipment and a computer program product. User state information is obtained. According to the method, question information answered by a user, user answering information and correction information for user answering are obtained, auxiliary learning global information is formed by the obtained multi-dimensional information, the large model capability is called, the auxiliary learning global information is understood and perceived, and thus auxiliary learning prompt information is generated. The multi-dimensional information including the user state, the answer question, the answer information, the answer correction information and the like is obtained, so that a large model can conveniently understand the corresponding answer conditions of the user in different states, actual questions encountered in the answer process of the user, such as answer errors caused by carelessness, inattention and the like, can be found in time, and the user experience is improved. By means of the multi-dimensional understanding and sensing ability of the large model, the auxiliary learning global information can be processed, and the auxiliary line prompt information is generated, so that more auxiliary learning guidance schemes can be provided for the user, and the auxiliary learning effect is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a learning assistance method, apparatus, related equipment, and computer program product. Background Art

[0002] With the widespread adoption of smart education products like learning machines, students are using them for independent learning in a growing number of scenarios and for a greater length of time. Current smart education products generally include a grading system that assesses students' responses and provides feedback, such as the accuracy of their answers. However, these grading systems are ineffective in assisting students' learning, failing to identify the real problems students encounter during their responses and providing targeted guidance. Summary of the Invention

[0003] In view of the above problems, this application is proposed to provide a learning assistance method, device, related equipment and computer program product to improve the effect of learning assistance for students. The specific solution is as follows:

[0004] In a first aspect, the present application provides a learning assistance method, comprising:

[0005] Get user status information during the answering process;

[0006] Obtain information about questions answered by users, information about user answers, and correction information about user answers;

[0007] The user status information, the question information, the user answer information and the correction information are combined into auxiliary learning global information, and the big model is called to instruct the big model to combine the auxiliary learning global information to generate auxiliary learning prompt information.

[0008] In one possible design, in another implementation of the first aspect of the embodiments of the present application, a process of calling the large model to instruct the large model to generate auxiliary learning prompt information in combination with the auxiliary learning global information includes:

[0009] Obtain a prompt format template, wherein the prompt format template includes a task instruction and an auxiliary learning information slot, wherein the task instruction is used to instruct the macro model to generate auxiliary learning prompt information in combination with the auxiliary learning global information in the auxiliary learning information slot;

[0010] The auxiliary learning global information is filled into the auxiliary learning information slot to obtain a first prompt instruction prompt, and the first prompt instruction prompt is input into the large model to obtain the auxiliary learning prompt information generated by the large model.

[0011] In one possible design, in another implementation of the first aspect of the embodiment of the present application, the task instruction is specifically used to: instruct the large model to align the auxiliary learning global information in the time dimension, and generate auxiliary learning prompt information in combination with the aligned auxiliary learning global information.

[0012] In one possible design, in another implementation of the first aspect of the embodiments of the present application, before calling the large model to instruct the large model to generate the auxiliary learning prompt information in combination with the auxiliary learning global information, the method further includes:

[0013] The auxiliary learning global information is aligned in the time dimension, and the aligned auxiliary learning global information is used as the input of the large model.

[0014] In one possible design, in another implementation of the first aspect of the embodiments of the present application, the supplementary learning prompt information includes a combination of one or more of the following:

[0015] A first prompt message for prompting the user to concentrate on answering the question;

[0016] A second prompt message for guiding the user to manage emotions;

[0017] A third prompt message for recommending specialized training resources;

[0018] A fourth prompt message for indicating errors in key steps and the degree of mastery of knowledge points;

[0019] The fifth prompt information is used for interactive guidance of key steps or knowledge points.

[0020] In one possible design, in another implementation of the first aspect of the embodiments of the present application, the supplementary learning prompt information further includes:

[0021] The question difficulty adjustment information is used for the configured question recommendation module to adjust the difficulty of subsequent recommended questions according to the question difficulty adjustment information.

[0022] In one possible design, in another implementation of the first aspect of the embodiments of the present application, the user status information includes a combination of one or more of the following:

[0023] Emotions, heart rate, sitting posture, eye gaze position.

[0024] In one possible design, in another implementation of the first aspect of the embodiments of the present application, the topic information includes a combination of one or more of the following:

[0025] Question difficulty, answer length, number of sub-questions contained in the question, knowledge points involved in the question, and global knowledge graph information.

[0026] In one possible design, in another implementation of the first aspect of the embodiments of the present application, the user answer information includes answer information in a spatial dimension and answer information in a temporal dimension, and the answer information in the spatial dimension includes a combination of one or more of the following:

[0027] Question answer location, draft area, circled content, and erased content;

[0028] The answer information of the time dimension includes one or more of the following combinations:

[0029] The duration of answering the entire question, the duration of answering key steps, the duration of pauses, and the writing timestamp of each answer segment.

[0030] In one possible design, in another implementation of the first aspect of the embodiments of the present application, the correction information includes a combination of one or more of the following:

[0031] Step-level grading scores, error analysis, and answer time analysis.

[0032] In a second aspect of the present application, a learning assistance device is provided, comprising:

[0033] A user status acquisition unit, used to acquire user status information during the answering process;

[0034] The answer-related data acquisition unit is used to obtain the question information answered by the user, the user's answer information and the correction information for the user's answer;

[0035] The auxiliary learning prompt generating unit is used to combine the user status information, the question information, the user answer information and the correction information into auxiliary learning global information, and call the big model to instruct the big model to combine the auxiliary learning global information to generate auxiliary learning prompt information.

[0036] In a third aspect of the present application, an electronic device is provided, comprising: a memory and a processor;

[0037] The memory is used to store programs;

[0038] The processor is used to execute the program to implement each step of the auxiliary learning method described in any one of the first aspects of this application.

[0039] In a fourth aspect of the present application, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the various steps of the auxiliary learning method described in any one of the aforementioned first aspects of the present application are implemented.

[0040] In a fifth aspect of the present application, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the various steps of the auxiliary learning method described in any one of the aforementioned first aspects of the present application.

[0041] By means of the above technical solution, this application obtains user status information for the answering process in the user's autonomous learning scenario, such as emotional changes, heart rate data, sitting posture, etc. during the answering process. Furthermore, the question information answered by the user, the user's answer information and the correction information for the user's answer are obtained, and the obtained multi-dimensional information constitutes the auxiliary learning global information. On this basis, the large model capability can be called to understand and perceive the auxiliary learning global information, thereby generating auxiliary learning prompt information. Since this application obtains multi-dimensional information including user status, answer questions, answer information, answer correction information, etc., it is convenient for the large model to understand the user's corresponding answer situation in different states, and can promptly discover the actual problems encountered by the user in the answering process, such as carelessness, answer errors caused by lack of concentration, answer errors caused by too high difficulty of the question, etc. With the help of the multi-dimensional understanding and perception capabilities of the large model, the auxiliary learning global information can be processed to generate auxiliary line prompt information, so that users can be given more auxiliary learning guidance plans and improve the auxiliary learning effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0043] Figure 1 A schematic diagram of an implementation system architecture of the learning assistance method provided in an embodiment of the present application;

[0044] Figure 2 A flowchart of a learning assistance method provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of the implementation flow of a learning assistance method provided in an embodiment of the present application;

[0046] Figure 4 A schematic diagram of a learning assistance scenario provided in an embodiment of the present application;

[0047] Figure 5 A schematic diagram of another auxiliary learning scenario provided in an embodiment of the present application;

[0048] Figure 6 A schematic diagram of the structure of a learning assistance device provided in an embodiment of the present application;

[0049] Figure 7A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0051] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, usage scenarios, etc. of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0052] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. Thus, based on the prompt message, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operation of the technical solution of this application.

[0053] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0054] It is understandable that the above notification and user authorization process is merely illustrative and does not limit the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present invention.

[0055] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0056] Currently, there are more and more scenarios and durations for students’ independent learning, but a more comprehensive auxiliary system is needed to achieve real-time perception of the answering process and emotions, so as to provide personalized and targeted guidance plans for students’ independent learning.

[0057] This application proposes a concrete perception assistance system for autonomous learning scenarios, which can obtain status information of students during the problem-solving process, such as emotions, heart rate, sitting posture, attention, etc., as well as question information, answer information, correction information, etc., to provide multi-dimensional, personalized large-scale model assistance capabilities in the entire answering process, and provide auxiliary learning prompts, such as guiding students to manage their emotions, discover weaknesses and difficulties, carelessness, identify key issues and propose targeted guidance plans, etc.

[0058] This application provides a learning assistance method that can be applied to Figure 1 The system architecture shown in FIG. 1 may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 (This section includes a server as an example).

[0059] The terminal 100 or the server 200 can be used alone to execute the assisted learning method provided in the embodiment of the present application. In addition, the terminal 100 and the server 200 can also be used in conjunction to execute the assisted learning method provided in the embodiment of the present application.

[0060] Next describe Figure 1 The product form of the mid-terminal 100;

[0061] The terminal 100 in the embodiment of the present application can be a learning machine, a mobile phone, a tablet computer, a teaching screen, a wearable device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiment of the present application does not impose any restrictions on this.

[0062] The embodiment of the present application provides a method for assisting learning, taking the method as an example of applying the method to a computer device, which can be specifically Figure 1 The terminal 100 or the system consisting of the terminal 100 and the server 200. Figure 2 , the auxiliary learning method specifically includes the following steps:

[0063] Step S100: Obtain user status information during the answering process.

[0064] Specifically, in the user's self-learning scenario, the user's status information during the answering process can be obtained. The user status information includes but is not limited to: emotions, heart rate, sitting posture, eye gaze position, etc.

[0065] The process of obtaining user status information in this step can be based on image data captured by a camera, and one or more types of user status information can be obtained through image or video analysis. In addition, user status information can also be collected through other types of sensors, such as collecting heart rate data during the user's answering process.

[0066] In this embodiment, the user status information is obtained based on the image data captured by the camera. During the user's answering process, the user can be photographed by the camera, and then various types of user status information can be determined based on the captured image data, for example:

[0067] 1. Emotion prediction:

[0068] The emotion prediction model models the spatial and temporal information of the user's facial expressions, extracts features, and then predicts emotion classification results, such as happiness, sadness, and other emotion types. Based on the configured emotion prediction model, the image data captured by the camera can be fed into the emotion prediction model to determine the user's emotion type at different moments during the answer process.

[0069] 2. Heart rate prediction:

[0070] Considering that changes in blood flow at different heart rates can cause changes in vascular volume, which in turn leads to changes in diffuse light intensity, it is possible to analyze changes in the user's heart rate by capturing changes in the user's facial vascular patterns in camera-captured image data. In one possible implementation, a heart rate prediction model can be pre-trained, with its input being the user's image data and its output being the user's heart rate data at different moments in the image data. Based on this, the camera-captured image data can be fed into the heart rate prediction model to obtain heart rate data at different moments during the user's response.

[0071] 3. Sitting posture detection:

[0072] The user's sitting posture movements include but are not limited to supporting the face, scratching the head, playing with electronic products, shaking the head, etc. The user's image data captured by the camera can be processed using image detection technology to obtain the user's sitting posture movements at different times during the answering process.

[0073] 4. Eye tracking:

[0074] Through the eye tracking algorithm, the user's eye gaze position can be mapped to a specific area on the screen based on the user image data captured by the camera, thereby obtaining the user's eye gaze position information at different times during the answering process.

[0075] The above examples only illustrate several types of user status information acquisition methods. In addition, other types of user status information can also be set, and this embodiment will not be repeated one by one.

[0076] In this step, by obtaining the status information of the user's answering process, we can fully understand the user's status in the answering process, which is convenient for analyzing the actual problems encountered or encountered in the user's answering process in subsequent steps, so as to provide targeted guidance solutions.

[0077] Step S110: Obtain the question information answered by the user, the user's answer information, and the correction information for the user's answer.

[0078] Among them, question information refers to information related to the questions answered by the user, including but not limited to: question difficulty, answer length, number of sub-questions contained in the question, knowledge points involved in the question, global knowledge graph information, etc.

[0079] The user answer information is the spatiotemporal information of the writing track points in the user's answer process. Exemplarily, the user answer information may include answer information of the spatial dimension and answer information of the temporal dimension. The answer information of the spatial dimension is information related to the position of the writing track points, examples of which include the answer position of the small question, the draft area, the circled content, the corrected content, etc. The answer information of the temporal dimension is information related to the time of the writing track points, examples of which include the answer time of the whole question, the answer time of the key steps, the pause time, the writing timestamp of each answer segment, etc. Among them, the key steps can be pre-defined important steps. The answer segments can be divided according to behavioral units, or according to other units.

[0080] Furthermore, this step can also obtain correction information for the user's answer. This correction information can include correction scores, error analysis, and answer time analysis at the step granularity level. The correction information obtained in this step can be the correction information obtained after the user's answer is corrected by the answer correction module. The answer correction module has the ability to correct user answers, and it can use a rule-based correction method or a deep neural network model-based correction method.

[0081] In some possible implementations, the process of obtaining correction information for the user's answer in this step may also be obtaining the correction results for the user's answer submitted by the correction subject (such as a teacher, parent, etc.).

[0082] It should be noted that the execution order of the above steps S100 and S110 can be executed in parallel or in any order. Figure 2 This is just an example of an alternative.

[0083] Step S120: The user status information, the question information, the user answer information and the correction information are combined into auxiliary learning global information, and the large model is called to instruct the large model to combine the auxiliary learning global information to generate auxiliary learning prompt information.

[0084] Among them, the auxiliary learning global information can be packaged according to the set format, so that it is convenient to input it into the large model for processing in the subsequent steps.

[0085] In this step, the auxiliary learning global information covers multi-dimensional information in the user's autonomous learning scenario, providing rich data support for comprehensive analysis of the large model in subsequent steps.

[0086] Step S130: calling the big model to instruct the big model to generate auxiliary learning prompt information in combination with the auxiliary learning global information.

[0087] The large model can use a general large model, or can use supplementary learning task training data to pre-train the general large model to obtain a pre-trained large model for generating supplementary learning prompt information. Supplementary learning task training data may include user answer information, correction information, status information of the user's answer process for sample questions, and manually annotated supplementary learning prompt information. By using supplementary learning task training data to pre-train the general large model, the instruction-following ability of the large model can be improved, making it more suitable for supplementary learning scenarios, and the supplementary learning prompt information generated is also more accurate.

[0088] The learning assistance and improvement information generated in this step is prompt information related to assisting users in learning, such as prompting users not to be distracted, prompting users to improve their attention, and prompting users to pay attention to points that are prone to mistakes.

[0089] An example of a possible scenario is: in the time dimension corresponding to the user's incorrect answer, the user's answer time is short and his emotions and heart rate are stable, and his movements and sitting posture are standard and serious, which can be defined as the perception category of carelessness. While the large model gives the specific error, it can guide the child to strengthen his concentration on calculation and recommend special training resources.

[0090] The auxiliary learning method provided in this embodiment obtains multi-dimensional information including user status, answer questions, answer information, answer correction information, etc., which is convenient for the big model to understand the user's corresponding answer situation in different states, and can promptly discover the actual problems encountered by the user in the answering process, such as answer errors caused by carelessness and lack of concentration, answer errors caused by the difficulty of the questions being too high, etc. With the help of the big model's multi-dimensional understanding and perception capabilities, the global auxiliary learning information can be processed and auxiliary line prompt information can be generated, so that the user can be given more auxiliary learning guidance plans and the auxiliary learning effect can be improved.

[0091] In this embodiment, an optional implementation scheme for calling a large model to generate supplementary learning prompt information is introduced.

[0092] This application can pre-configure a prompt format template, which includes a task instruction and a supplementary learning information slot. The task instruction is used to instruct the large model to combine the supplementary learning global information in the supplementary learning information slot to generate supplementary learning prompt information.

[0093] On this basis, in the process of calling the big model to generate auxiliary learning prompt information, you can obtain the configured prompt format template, fill the auxiliary learning global information obtained in the previous steps into the auxiliary learning information slot, and get the first prompt instruction prompt. Then, you can input the first prompt instruction prompt into the big model to get the auxiliary learning prompt information generated by the big model.

[0094] In this embodiment, by configuring the prompt format template, the large model can be called quickly and in a standardized manner, thereby improving the efficiency and standardization of the large model call.

[0095] In the auxiliary learning global information obtained in the aforementioned steps of this application, information of some dimensions has time series characteristics. For example, user status information is information related to time, which includes the changes in user status over time during the entire answering process. User answer information is also information related to time, which includes the user's answer information at each moment in the entire answering process. In some possible implementations, the big model has the ability to understand and process multi-dimensional information. In the above steps, in the process of generating auxiliary learning prompt information by combining the big model with the auxiliary learning global information, the big model can autonomously analyze the time relationship of each dimensional information in the auxiliary learning global information, perform time alignment on the auxiliary learning global information, and then generate auxiliary learning prompt information based on the aligned auxiliary learning global information.

[0096] Furthermore, the global supplementary learning information also contains spatial characteristics, such as the location of the user's answer, the location of the draft area, the labeled data in the question stem, and the answer data of the sub-questions. The large model can use its inherent spatiotemporal perception capabilities to align the global supplementary learning information in time and space, and then generate supplementary learning prompts based on this aligned global supplementary learning information.

[0097] In some possible implementations, the task instructions contained in the prompt format template can be specifically used to instruct the large model to align the global supplementary learning information in the time dimension, and generate supplementary learning prompt information based on the aligned global supplementary learning information. Alternatively, the task instructions can be specifically used to instruct the large model to align the global supplementary learning information in the time and space dimensions, and generate supplementary learning prompt information based on the aligned global supplementary learning information.

[0098] In this embodiment, by explicitly adding the time (time and space) dimension alignment processing operation of the auxiliary learning global information in the prompt task instruction, the large model can be more clearly guided to perform the alignment operation. By aligning the auxiliary learning global information, the large model can better understand the relationship between the aligned auxiliary learning global information, thereby improving the accuracy of the generated auxiliary learning prompt information.

[0099] In some other possible implementations, before calling the large model in step S120 of the aforementioned embodiment to instruct the large model to generate auxiliary learning prompt information in combination with the auxiliary learning global information, the following processing steps may be added:

[0100] The auxiliary learning global information is aligned in the time dimension and the aligned auxiliary learning global information is used as the input of the large model.

[0101] Alternatively, the auxiliary learning global information is aligned in the time and space dimensions, and the aligned auxiliary learning global information is used as the input of the large model.

[0102] By aligning the acquired auxiliary learning global information in terms of time (time and space) before calling the large model, the aligned auxiliary learning global information can be obtained. Then, in step S120, the large model is called to instruct the large model to generate auxiliary learning prompt information based on the aligned auxiliary learning global information.

[0103] In this embodiment, by aligning the auxiliary learning global information before calling the big model and then using it as the input of the big model, the big model can better understand the relationship between the various dimensions of data in the auxiliary learning global information and improve the accuracy of the generated auxiliary learning prompt information.

[0104] In some embodiments of the present application, some possible compositions of supplementary learning prompt information are introduced.

[0105] In this embodiment, the supplementary learning reminder information may include one or more of the following combinations:

[0106] A first prompt message for prompting the user to concentrate on answering the question;

[0107] A second prompt message for guiding the user to manage emotions;

[0108] A third prompt message for recommending specialized training resources;

[0109] A fourth prompt message for indicating errors in key steps and the degree of mastery of knowledge points;

[0110] The fifth prompt information is used for interactive guidance of key steps or knowledge points.

[0111] For example, if the user is distracted (e.g., playing with a mobile phone) during the time dimension corresponding to an incorrect answer, the large model can output a first prompt message to remind the user to concentrate on answering. For example, the first prompt message may be "Children, please pay attention when doing your homework~"

[0112] For example, if the user shows abnormal emotions in the time dimension corresponding to the user's incorrect answer, the large model can output a second prompt message to prompt the user to manage his emotions. An example of the second prompt message is "It is detected that your little universe is a little fluctuating~ Put down the pen first and take three deep breaths with me? Inhale - exhale -".

[0113] For example, the large model analyzes the target knowledge points corresponding to the user's incorrect answers and can further output a third prompt message to recommend specialized training resources related to the target knowledge points. An example of the third prompt message is "The following is the specialized computing training I recommend for you. Let's strengthen our training together."

[0114] For example, the large model detects that the step where the user answered incorrectly is a key step and obtains the user's mastery of the knowledge point corresponding to the key step. It can then output a fourth prompt message to inform the user of the incorrect answer to the key step and the user's mastery of the knowledge point. An example of the fourth prompt message might be, "An error occurred in the calculation process of the second step. This step is a key step in solving the entire problem. We detect that you have a strong mastery of the current knowledge point. Please do not be careless."

[0115] For example, the large model detects that a step in which the user's answer is incorrect is a key step, and further obtains knowledge points related to this key step. Based on this knowledge point, it outputs a fifth prompt message to interactively guide the user. An example of the interactive guidance process corresponding to the fifth prompt message is: "We detect that you are not proficient in calculating 'linear equations in two variables'. Let's do some intensive training together, okay?" ...

[0116] Further optionally, the supplementary learning prompt information may also include: question difficulty adjustment information.

[0117] The question difficulty adjustment information is used by a question recommendation module configured in the auxiliary learning system to adjust the difficulty of subsequent recommended questions according to the question difficulty adjustment information.

[0118] Specifically, the large-scale model analyzes global information about the learning assistance system and outputs difficulty adjustment information when the analysis determines that the current question is too difficult (for example, if the large-scale model detects symptoms such as scratching the user's head or pausing when answering an incorrect answer, it can be determined that the current question is too difficult). The question recommendation module in the learning assistance system can use this difficulty adjustment information to adjust the difficulty of subsequent questions recommended to the user, thereby automatically adjusting the difficulty of the questions based on the user's current level and improving the learning assistance effect.

[0119] The above embodiments illustrate only a few possible examples of supplementary learning prompt information. In actual applications, supplementary learning prompt information may include any one or more of the above prompt information. In addition, other types of supplementary learning prompt information can be set according to actual business needs. Corresponding task prompt information can be added to the task instruction to instruct the large model to generate the corresponding type of supplementary learning prompt information.

[0120] Combine Figure 3 , which provides a specific application process of a supplementary learning method.

[0121] The supplementary learning global information obtained in this embodiment includes three types of information:

[0122] First, based on the visual understanding ability of the camera, the image data captured by the camera can be analyzed and processed to obtain status information of the user's answering process, including but not limited to: emotion detection results, heart rate detection results, sitting posture detection results, and eye tracking results.

[0123] Second, obtain the questions answered by the user and their answer correction information. Examples of question information include: question difficulty, number of sub-questions, answer length, knowledge points involved, and the global knowledge graph. Examples of correction information include: correction score, error analysis, and answer time analysis.

[0124] Third, the user's response information, specifically the spatiotemporal information of the user's writing trajectory, is obtained. Examples of spatial information about the writing trajectory include the location of the answer, the control area, the image content, and any corrections. Examples of temporal information about the writing trajectory include the duration of the entire answer, the duration of key steps, pauses, and the writing timestamp of each answer segment.

[0125] The three types of input are combined into global learning information, which is then assembled into prompts and fed into the large model. The large model aligns the input global learning information with the timeline and, based on this aligned global learning information, generates learning prompts according to the task instructions.

[0126] Among them, the task instructions can pre-specify various types of tasks to be analyzed by the large model, including but not limited to the analysis task of the first prompt information, the analysis task of the second prompt information, the analysis task of the third prompt information, the analysis task of the fourth prompt information, the analysis task of the fifth prompt information, the question difficulty analysis task, etc. introduced in the above embodiments.

[0127] Figure 3 The following examples illustrate several possible outputs of the large model, such as carelessness reminders, key step reminders (such as reminding users of incorrect answers to key steps and their mastery of relevant knowledge points of key steps), emotional guidance, difficulty adjustment information, etc.

[0128] Reference Figure 4 , Figure 4 An example diagram of a supplementary learning scenario is provided.

[0129] The global information of auxiliary learning includes user status information, user answer information, question and correction information.

[0130] The user status information includes the set type status of the user at each moment detected in chronological order from the beginning of answering to the end of answering and submitting for correction. Figure 4 As shown in the figure, "doubted expression", "distracted actions: playing with mobile phone", "eyes away from the answering area", and "high heart rate".

[0131] The user's answer information records the trajectory information of the user's answer. In chronological order, it includes the first to fifth lines of the answer.

[0132] The question information includes: the current question is of moderate difficulty, the knowledge point involved is: linear equation of two variables, the user's mastery of this knowledge point is: general, and the user still has some knowledge points in this chapter that he has not mastered in the global knowledge graph.

[0133] Correction information includes: Error Analysis: Method error in the third line, Correction Step Scoring: 1 / 1 / 0 / 0 / 0. Answer Time Analysis: Too long.

[0134] From the timeline alignment, we can see that the user displayed a puzzled expression during the third key step of the answer. Combined with the question information, we can see that the user's understanding of the knowledge points involved in the question is average, indicating that the user did not truly grasp the relevant knowledge points, leading to incorrect answers. Furthermore, the user displayed distractions during the answering process: playing with their phone and looking away from the answer area, indicating that the user was not paying attention.

[0135] The above-mentioned global information of supplementary learning is fed into the big model. The big model is used to align and analyze the global information of supplementary learning in timeline, and supplementary learning prompt information can be obtained. For example:

[0136] "Children, this question is a high-frequency test point, so you must concentrate while answering it. Although we didn't get it right this time, I recommend a special video on the "Substitution Elimination Method" to you. If you don't understand something during the learning process, you can interrupt and ask me at any time."

[0137] The aforementioned learning assistance prompts demonstrate that the large-scale model can detect inattention based on the user's status during the answering process and provide reminders. Furthermore, it can identify errors in the user's answers and provide targeted video exercises, effectively improving learning assistance.

[0138] Reference Figure 5 , Figure 5 Another diagram of a supplementary learning scenario is illustrated.

[0139] The global information of auxiliary learning includes user status information, user answer information, question and correction information.

[0140] The user status information includes the set type status of the user at each moment detected in chronological order from the beginning of answering to the end of answering and submitting for correction. Figure 5 "Emotionally stable", "no distracting movements", "normal eye tracking", and "normal heart rate" are shown in the .

[0141] The user's answer information records the trajectory information of the user's answer. In chronological order, it includes the first to third rows of the answer.

[0142] The question information includes: the current question is of moderate difficulty, the knowledge point involved is: addition and subtraction of three-digit decimals, the user's mastery of this knowledge point is: proficient, and the user has no unmastered knowledge points in this chapter in the global knowledge graph.

[0143] Correction information includes: Error Analysis: Calculation error in the third row, Correction Step Scoring: 1 / 0 / 1 / 1 / 1. Answer Time Analysis: Normal.

[0144] From the perspective of timeline alignment, it can be seen that there is no abnormal state in the user's answering process, and combined with the question information, it can be seen that the user is proficient in the knowledge points involved in the question, and the calculation error is most likely due to carelessness.

[0145] The above-mentioned global information of supplementary learning is fed into the big model. The big model is used to align and analyze the global information of supplementary learning in timeline, and supplementary learning prompt information can be obtained. For example:

[0146] "Children, although you have mastered the knowledge points, you still need to calculate carefully. The following are special calculation trainings that I recommend to you. Let's strengthen our training together."

[0147] The above-mentioned learning assistance prompts show that the large-scale model can identify the cause of user calculation errors due to carelessness based on the overall learning information and provide reminders. At the same time, it can also identify where the user made mistakes in the answer and provide targeted special training, effectively improving the learning assistance effect.

[0148] The following describes the auxiliary learning device provided in the embodiment of the present application. The auxiliary learning device described below and the auxiliary learning method described above can be referenced to each other.

[0149] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of a learning assistance device disclosed in an embodiment of the present application.

[0150] like Figure 6 As shown, the device may include:

[0151] A user status acquisition unit 11 is used to acquire user status information during the answering process;

[0152] The answer-related data acquisition unit 12 is used to obtain the question information answered by the user, the user's answer information and the correction information for the user's answer;

[0153] The auxiliary learning prompt generating unit 13 is used to combine the user status information, the question information, the user answer information and the correction information into auxiliary learning global information, and call the big model to instruct the big model to generate auxiliary learning prompt information in combination with the auxiliary learning global information.

[0154] In one possible implementation, the auxiliary learning prompt generating unit calls the large model to instruct the large model to generate the auxiliary learning prompt information in combination with the auxiliary learning global information, including:

[0155] Obtain a prompt format template, wherein the prompt format template includes a task instruction and an auxiliary learning information slot, wherein the task instruction is used to instruct the macro model to generate auxiliary learning prompt information in combination with the auxiliary learning global information in the auxiliary learning information slot;

[0156] The auxiliary learning global information is filled into the auxiliary learning information slot to obtain a first prompt instruction prompt, and the first prompt instruction prompt is input into the large model to obtain the auxiliary learning prompt information generated by the large model.

[0157] In a possible implementation, the task instruction is specifically used to: instruct the large model to align the auxiliary learning global information in the time dimension, and generate auxiliary learning prompt information in combination with the aligned auxiliary learning global information.

[0158] In a possible implementation, before calling the large model to instruct the large model to generate the auxiliary learning prompt information in combination with the auxiliary learning global information, the auxiliary learning prompt generating unit is further configured to:

[0159] The auxiliary learning global information is aligned in the time dimension, and the aligned auxiliary learning global information is used as the input of the large model.

[0160] In one possible implementation, the supplementary learning prompt information generated by the supplementary learning prompt generating unit includes a combination of one or more of the following:

[0161] A first prompt message for prompting the user to concentrate on answering the question;

[0162] A second prompt message for guiding the user to manage emotions;

[0163] A third prompt message for recommending specialized training resources;

[0164] A fourth prompt message for indicating errors in key steps and the degree of mastery of knowledge points;

[0165] The fifth prompt information is used for interactive guidance of key steps or knowledge points.

[0166] Further optionally, the supplementary learning prompt information generated by the supplementary learning prompt generating unit further includes:

[0167] The question difficulty adjustment information is used for the configured question recommendation module to adjust the difficulty of subsequent recommended questions according to the question difficulty adjustment information.

[0168] In one possible implementation, the user status information acquired by the user status acquisition unit includes a combination of one or more of the following:

[0169] Emotions, heart rate, sitting posture, eye gaze position.

[0170] In one possible implementation, the question information acquired by the answer-related data acquisition unit includes a combination of one or more of the following:

[0171] Question difficulty, answer length, number of sub-questions contained in the question, knowledge points involved in the question, and global knowledge graph information.

[0172] In one possible implementation, the user answer information acquired by the answer-related data acquisition unit includes answer information in a spatial dimension and answer information in a temporal dimension, where the answer information in the spatial dimension includes a combination of one or more of the following:

[0173] Question answer location, draft area, circled content, and erased content;

[0174] The answer information of the time dimension includes one or more of the following combinations:

[0175] The duration of answering the entire question, the duration of answering key steps, the duration of pauses, and the writing timestamp of each answer segment.

[0176] In one possible implementation, the correction information acquired by the answer-related data acquisition unit includes a combination of one or more of the following:

[0177] Step-level grading scores, error analysis, and answer time analysis.

[0178] Each unit in the above-mentioned auxiliary learning device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the above-mentioned units can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each of the above-mentioned units.

[0179] An electronic device is also provided in an embodiment of the present application. Figure 7 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to terminals such as learning machines, mobile phones, tablet computers, teaching large screens, wearable devices, etc. Figure 7 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0180] like Figure 7 As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage device 608 into the random access memory (RAM) 603 to implement the auxiliary learning method of the aforementioned embodiment of the present application. When the electronic device is powered on, the RAM 603 also stores various programs and data required for the operation of the electronic device. The processing device 601, ROM 602 and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0181] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Figure 7 The electronic device is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0182] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the auxiliary learning methods provided in the embodiments of the present application.

[0183] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any one of the auxiliary learning methods provided in the embodiment of the present application.

[0184] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0185] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0186] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0187] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website, a computer, a training device or a data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0188] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

Claims

1. A learning assistance method, characterized in that: include: Get user status information during the answering process; Obtain information about questions answered by users, information about user answers, and correction information about user answers; The user status information, the question information, the user answer information and the correction information are combined into auxiliary learning global information, and the big model is called to instruct the big model to combine the auxiliary learning global information to generate auxiliary learning prompt information.

2. The method according to claim 1, characterized in that The process of calling the big model to instruct the big model to generate auxiliary learning prompt information in combination with the auxiliary learning global information includes: Obtain a prompt format template, wherein the prompt format template includes a task instruction and an auxiliary learning information slot, wherein the task instruction is used to instruct the macro model to generate auxiliary learning prompt information in combination with the auxiliary learning global information in the auxiliary learning information slot; The auxiliary learning global information is filled into the auxiliary learning information slot to obtain a first prompt instruction prompt, and the first prompt instruction prompt is input into the large model to obtain the auxiliary learning prompt information generated by the large model.

3. The method according to claim 2, characterized in that The task instruction is specifically used to instruct the large model to align the auxiliary learning global information in the time dimension, and generate auxiliary learning prompt information in combination with the aligned auxiliary learning global information.

4. The method according to claim 1, wherein Before calling the large model to instruct the large model to combine the global auxiliary learning information to generate auxiliary learning prompt information, the method further includes: The auxiliary learning global information is aligned in the time dimension, and the aligned auxiliary learning global information is used as the input of the large model.

5. The method according to any one of claims 1 to 4, characterized in that The supplementary learning reminder information includes one or more of the following combinations: A first prompt message for prompting the user to concentrate on answering the question; A second prompt message for guiding the user to manage emotions; A third prompt message for recommending specialized training resources; A fourth prompt message for indicating errors in key steps and the degree of mastery of knowledge points; The fifth prompt information is used for interactive guidance of key steps or knowledge points.

6. The method according to claim 5, characterized in that The auxiliary learning prompt information also includes: The question difficulty adjustment information is used for the configured question recommendation module to adjust the difficulty of subsequent recommended questions according to the question difficulty adjustment information.

7. The method according to any one of claims 1 to 4, characterized in that The user status information includes one or more of the following: Emotions, heart rate, sitting posture, eye gaze position.

8. The method according to any one of claims 1 to 4, characterized in that The topic information includes one or more of the following: Question difficulty, answer length, number of sub-questions contained in the question, knowledge points involved in the question, and global knowledge graph information.

9. The method according to any one of claims 1 to 4, characterized in that The user answer information includes answer information in the spatial dimension and answer information in the temporal dimension. The answer information in the spatial dimension includes one or more of the following: Question answer location, draft area, circled content, and erased content; The answer information of the time dimension includes one or more of the following combinations: The duration of answering the entire question, the duration of answering key steps, the duration of pauses, and the writing timestamp of each answer segment.

10. The method according to any one of claims 1 to 4, characterized in that The correction information includes one or more of the following: Step-level grading scores, error analysis, and answer time analysis.

11. A learning aid device, characterized in that: include: A user status acquisition unit, used to acquire user status information during the answering process; The answer-related data acquisition unit is used to obtain the question information answered by the user, the user's answer information and the correction information for the user's answer; The auxiliary learning prompt generating unit is used to combine the user status information, the question information, the user answer information and the correction information into auxiliary learning global information, and call the big model to instruct the big model to combine the auxiliary learning global information to generate auxiliary learning prompt information.

12. An electronic device, characterized in that: include: memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the auxiliary learning method according to any one of claims 1 to 10.

13. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the auxiliary learning method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, each step of the auxiliary learning method according to any one of claims 1 to 10 is implemented.

Images