Teacher reflection text multi-dimensional evaluation method and system based on document theme perception

Abstract

The invention relates to a teacher reflection text multi-dimensional evaluation method and system based on document theme perception. The method comprises the following steps: obtaining teacher reflection text data to be evaluated; converting the teacher reflection text data into bag-of-word representation, and obtaining theme distribution of a teacher reflection text by using a pre-trained theme embedding model; and embedding the teacher reflection text data into a preset scoring instruction template, taking the topic distribution as a distribution route, and extracting local sentences corresponding to the topics from the global teacher reflection text data by using a large model comprising a LoRA block to obtain scores corresponding to the topics so as to realize evaluation of the teacher reflection text. Compared with the prior art, the method has the advantages of good evaluation effect, low resource consumption and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of text data analysis and evaluation, and in particular to a multi-dimensional evaluation method and system for teacher reflection texts based on document theme perception. Background Art

[0002] The automatic scoring technology based on artificial intelligence is relatively mature, but it still has certain deficiencies in the evaluation of different dimensions of text. In addition, the data sets currently used to train the evaluation model are mostly student essays, and the evaluation dimensions are coarse-grained information such as language, content, organizational structure, and theme consistency. These methods often do not work well when applied to other specific fields, especially in the evaluation of teacher reflection texts.

[0003] The particularity of the evaluation of teacher reflection texts is that it not only involves the expression quality of the text, but also requires the evaluation of more fine-grained content. For example: whether there is a sense of problem, that is, whether problems in the teaching process can be discovered; whether the perspective of reflection is diverse, that is, whether the problem is analyzed in depth from multiple angles; whether personalized insights and specific improvement methods are proposed to solve the problems reflected; and whether the awareness of continuous growth is reflected. Therefore, the existing artificial intelligence scoring methods, especially the coarse-grained evaluation methods for student essays, are often difficult to adapt to these fine-grained, multi-dimensional evaluation needs in teacher reflection texts.

[0004] In the current technological context, large model technology has emerged with powerful semantic understanding capabilities due to its huge number of parameters, enabling it to achieve good performance in various natural language processing (NLP) downstream tasks. However, large models are often trained using corpora in general fields, and there are still certain difficulties in migrating to the field of education. In addition, multi-dimensional text evaluation is often regarded as a scenario for multi-task training, but most of the current multi-task training frameworks are designed based on non-homologous data (i.e., task data from different sources). In actual applications, the data source for multi-dimensional text evaluation usually comes from the same article, so the existing framework cannot effectively build proprietary domain knowledge for each evaluation dimension. This limitation makes it difficult for existing frameworks to comprehensively and accurately evaluate the performance of text in different dimensions. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a multi-dimensional evaluation method and system for teacher reflection text based on document topic awareness. By combining the document topic awareness ability of the topic model with the strategy of efficient parameter fine-tuning and hybrid expert model, accurate evaluation of teacher reflection text in multiple dimensions can be achieved.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] One aspect of the present invention provides a multi-dimensional evaluation method for teacher reflection text based on document theme perception, comprising the following steps:

[0008] Obtain the teacher reflection text data to be evaluated;

[0009] The teacher reflection text data is converted into a bag-of-words representation, and a pre-trained topic embedding model is used to obtain the topic distribution of the teacher reflection text;

[0010] The teacher reflection text data is embedded in a preset scoring instruction template, and the large model is used to extract sentences corresponding to each topic. The topic distribution is used as the distribution routing. A large model including LoRA blocks is used to extract local sentences corresponding to each topic from the global teacher reflection text data, and the scores corresponding to each topic are obtained to realize the evaluation of the teacher reflection text.

[0011] As a preferred technical solution, the training process of the topic embedding model includes:

[0012] The annotated sentences are obtained and divided according to the topic to obtain a set of sentences corresponding to the topic;

[0013] For each dimension, randomly sample a batch of sentences from the corresponding sentence set, embed the sentences into a preset keyword extraction instruction template, and use the large model to generate keywords that represent the batch of sentences;

[0014] The extracted keywords are used as the input of the large model to extract the hidden state representation, and the semantic embedding vectors corresponding to each topic are obtained through pooling operations. The semantic embedding vectors corresponding to each topic are concatenated to obtain the topic embedding prior distribution;

[0015] The topic embedding model is trained using the topic embedding prior distribution as supervision.

[0016] As a preferred technical solution, the training of the topic embedding model using the topic embedding prior distribution as supervision includes:

[0017] The teacher reflection text data to be evaluated is converted into a bag-of-words representation, and the topic embedding prior distribution is used as a supervisory signal. The topic embedding model is trained with the goal of minimizing the difference between the generated topic embedding distribution and the topic embedding prior distribution to generate topics and their distributions related to the evaluation dimensions.

[0018] As a preferred technical solution, the construction process of the LoRA block includes the following steps:

[0019] A plurality of LoRA blocks are constructed in each dense layer of the large model, and the LoRA blocks are trained during the forward propagation process of the large model in response to the scoring instruction template.

[0020] As a preferred technical solution, the number of LoRA blocks matches the number of topics.

[0021] As a preferred technical solution, the process of extracting local sentences corresponding to each topic from the global teacher reflection text data includes the following steps:

[0022] The teacher reflection text data embedded in the scoring instruction template, the description of the scoring dimension corresponding to the topic and the instruction text are used as the input of the large model, and the sentences corresponding to the topic are extracted from the teacher reflection text data.

[0023] As a preferred technical solution, the process of obtaining the scores corresponding to each topic by using the pre-built evaluation network corresponding to the topic includes the following steps:

[0024] The task instructions in the form of multi-round dialogues are constructed as the architecture. After the big model gives answers to the sentences corresponding to each topic, scoring instructions are continued to be given to let the big model output the scores corresponding to each topic.

[0025] As a preferred technical solution, the themes include text expression, problem meaning, comprehensive thinking, method construction and continuous growth, which are the five dimensions of multi-dimensional evaluation.

[0026] Another aspect of the present invention provides a system for multi-dimensional evaluation of teacher reflection texts based on document topic perception, comprising:

[0027] The data acquisition module is used to obtain the teacher reflection text data to be evaluated;

[0028] A topic distribution extraction module is used to convert the teacher reflection text data into a bag-of-words representation, and obtain the topic distribution of the teacher reflection text using a pre-trained topic embedding model;

[0029] The reflective text evaluation module is used to embed the teacher reflection text data into a preset scoring instruction template, use the topic distribution as the distribution route, and utilize a large model including LoRA blocks to extract local sentences corresponding to each topic from the global teacher reflection text data, obtain the scores corresponding to each topic, and realize the evaluation of the teacher reflection text.

[0030] Another aspect of the present invention provides an electronic device, comprising: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the aforementioned multi-dimensional evaluation method of teacher reflection text based on document topic perception.

[0031] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0032] (1) Realizing supervised topic model training: The present invention uses a large model to complete the designed topic keyword extraction task, constructs a topic embedding prior, and introduces a supervisory signal to generate a specific scoring dimension topic for the unsupervised topic model.

[0033] (2) High evaluation accuracy: The present invention uses multiple scoring dimensions (corresponding to multiple topics) for evaluation, and uses the document topic distribution driving model generated by the topic model to select appropriate evaluation experts for processing, thereby improving the evaluation effect.

[0034] (3) Reducing computing resource consumption: The present invention combines a hybrid expert model framework with a scoring instruction template to solve the resource consumption problem caused by combining the same text input with different task instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flowchart of a multi-dimensional evaluation method of teacher reflection text based on document theme perception in an embodiment;

[0036] Figure 2 A schematic diagram of the evaluation process in the embodiment;

[0037] Figure 3 It is a schematic diagram of a multi-dimensional evaluation system for teacher reflection texts based on document theme perception in an embodiment;

[0038] Figure 4 Schematic diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] Example 1

[0041] In response to the problems existing in the aforementioned prior art, the present embodiment provides a multi-dimensional evaluation method for teacher reflection texts based on document topic awareness, which aims to achieve accurate evaluation of teacher reflection texts in multiple dimensions by combining the document topic awareness ability of the topic model with efficient parameter fine-tuning and the strategy of a hybrid expert model.

[0042] See also Figure 1 and Figure 2, the method comprises the following steps:

[0043] Step S1: annotate the collected teaching reflection texts of new teachers and calculate the topic embedding prior distribution.

[0044] Preferably, themes (i.e., evaluation dimensions) may include text expression, problem meaning, comprehensive thinking, method construction, and continuous growth.

[0045] In the process of data annotation, we not only annotated the scores for each evaluation dimension, but also annotated the specific sentences that can reflect each dimension. However, since most evaluation dimensions are relatively vague, it is difficult to directly find the corresponding sentences, resulting in a large imbalance in the annotated sentences between the dimensions.

[0046] To address the above issues, this embodiment designs a task of finding keywords from a sentence set corresponding to the evaluation dimension to construct representation information of the evaluation dimension. The specific construction steps are as follows:

[0047] Step 1: Summarize the annotated sentences according to each evaluation dimension to form corresponding sets. Each set contains all sentences related to the dimension.

[0048] Step 2: Sample the sentence set corresponding to each dimension. Specifically, five sentences are randomly sampled from a set of sentences in a specific dimension in each round, and these sentences are inserted into the preset "find keywords" instruction template to form the input of the large model. With the powerful semantic understanding ability of the large model, keywords that can represent these five sentences are generated. This process will continue for 30 rounds for each dimension to ensure that the details and feature information of each evaluation dimension are accurately captured.

[0049] Step 3: Input the obtained keywords of each dimension into the large model, extract the representation of the hidden state of the last five layers, and finally obtain the overall deep semantic embedding vector representing the dimension through average pooling operation. This step is performed once for each dimension, and finally the feature vectors of each dimension are spliced ​​to form the topic embedding prior distribution.

[0050] Step S2: The training input is the bag-of-words representation, and the output is the embedded topic model that predicts the topic distribution.

[0051] Train an embedded topic model. Since the topic model is an unsupervised model, the document topics it generates are automatically learned based on the latent semantic structure of the document, without relying on manually annotated labels or supervisory signals. However, this does not meet the requirements of generating topics with specified evaluation dimensions.

[0052] To address this problem, this embodiment uses the topic embedding prior distribution obtained in the previous step as a supervisory signal to guide the topic model to generate topics of the specified evaluation dimension. The specific training steps are as follows:

[0053] Step 1, define each evaluation dimension as a different topic.

[0054] Step 2: Filter the teacher reflection text according to word frequency to find the word set that best represents the document, and convert it into a bag-of-words model representation.

[0055] Step 3, design a loss function to measure the similarity between the generated topic embedding distribution and the prior topic embedding distribution. In order to quantify the similarity between the generated topic distribution and the prior distribution, cosine similarity is used here. Specifically, the topic embedding prior distribution is introduced into the latent space of VAE (variational autoencoder), the backbone of the embedded topic model, and the designed loss function is used to make the latent space topic embedding distribution generated by the model during the training process as close as possible to the prior topic embedding distribution, so as to generate topics that meet specific evaluation dimensions.

[0056] Step S3: Combining the document topic perception ability of the topic model with efficient parameter fine-tuning and hybrid expert model to achieve multi-dimensional and accurate evaluation of teaching reflection texts.

[0057] This embodiment uses the LoRA (Low-rank Adaption) method in efficient parameter fine-tuning, and combines it with the mixed expert model framework (Mixture of experts). Each expert is set as a LoRA block, and the routing mechanism for multi-task selection of different experts is determined by the document topic distribution generated by the topic model trained in the previous step.

[0058] When fine-tuning the large model, we use the efficient fine-tuning technology Lora. To adapt to the task setting of multi-dimensional evaluation, we design a Lora block as an expert. Each dense layer of the large model has such a group of expert modules (the number of experts in the group depends on the evaluation dimension). In this way, during the training process, the document embedded with the evaluation instruction template is input. During the overall forward propagation process, each dense layer of the large model ( Figure 2 The weight parameters of the LLM in the Figure 2The weight parameters of LoRA1 to LoRAN in the training are trained and initialized uniformly. The contribution of experts to the evaluation task is controlled by the document topic distribution inferred by the topic model trained in the previous stage, that is, a specific topic corresponds to a specific expert. The training goal is to enable the large model to perceive the content of a specific topic and accurately evaluate the score of the corresponding topic text content. In other words, it is an evaluation expert in a specific dimension. Therefore, during the training process, as the model fits the score of each dimension, its expert (LoRA block) will also highlight the expert nature with this fitting process, and become a specialized module for the evaluation task of a specific dimension.

[0059] During the evaluation, the input is the teacher's reflection text, but it will be embedded into the scoring instruction template and converted into a bag-of-words model representation. The former is used as the input for fine-tuning the large model to guide the model to generate evaluation scores for each dimension, while the latter is used as the input for the topic model to generate the topic distribution of the document.

[0060] The evaluation of each dimension of the scoring instruction template combines two tasks: finding the corresponding sentence for the dimension and scoring the dimension. This instruction template is based on a multi-round dialogue structure. First, the description, original text and instruction (find the corresponding sentence for the dimension) of a scoring dimension are used as the input of one round, and the model is asked to find the corresponding sentence for the dimension from the original text. Then, the scoring instruction is given to let the model evaluate the score of the dimension. Then, the description of another scoring dimension and the instruction to find the corresponding sentence are used as the input of another round for another round of scoring tasks. (The instruction template is described later)

[0061] In summary, the document topic distribution can reflect the distribution of different contents in the article, and this method constructs topics related to the evaluation dimensions, so that the document topic distribution output by the topic model in the framework can naturally represent the distribution of articles on different evaluation dimensions, helping the model to automatically select suitable experts for processing according to the proportion of different evaluation dimensions, thereby achieving more accurate and efficient task allocation.

[0062] This method has the following characteristics:

[0063] (1) Design a topic keyword extraction task to construct a topic embedding prior, thereby supervising the topic model to generate relevant topics: Use a large model to complete the designed topic keyword extraction task, construct a topic embedding prior, and introduce supervision signals for the unsupervised topic model to generate specific scoring dimension topics.

[0064] (2) Construct a hybrid expert model architecture with topic model as routing decision to realize multi-dimensional evaluation of teaching reflection text: The document topic distribution driven model generated by the topic model selects appropriate evaluation experts for processing to improve the evaluation effect.

[0065] (3) Build an efficient scoring instruction template: By combining this architecture with the designed scoring instruction template, the problem of resource consumption caused by combining the same text input with different task instructions is solved.

[0066] Example of keyword extraction instruction template:

[0067] from user: <Task instruction>Please find 5 most relevant keywords from the following set of sentences according to the following evaluation observation points. These keywords should accurately summarize and describe the core content of this dimension. Each keyword should be concise and clear, and can represent the main features or attributes of this dimension. Please output the answer strictly according to the example output format, and the number of keywords should not exceed 5< / Task instruction>.

[0068] <Observation point description>Has the teacher found his own problems through reflection?< / Observation point description>

[0069] <Set of sentences>· Present the logical relationship of asking questions from big questions to small questions, rather than presenting a piece of historical material to reflect a knowledge point and then converging into a big concept· Most of the time, I am still giving lectures or playing videos, with little interaction with students, and the dominant position of students is not fully reflected· However, there are no students actively answering questions, and basically all rely on the teacher to call on students· The new curriculum emphasizes the importance of process teaching, but in some teachers' classroom teaching, the whole class is just a class for exploring formulas· The state of the font is quite bad, which may affect students' note-taking on the blackboard< / Set of sentences>

[0070] <Task example>: Set of sentences: {'In primary school information, teachers need to demonstrate each step of the operation as clearly as possible, and the gap between the two is very large', 'The number of primary school information courses is small', 'The questions raised will also be diverse. Sometimes they just fit the teaching tasks of this lesson, and sometimes they are about the content before or after', 'In classroom teaching, the teaching progress needs to be slowed down and students' feedback should be paid attention to in a timely manner.', 'Therefore, most teaching teachers will spend time on how to solve the difficult points', 'But if the teacher does not explain, there will still be many students operating in the small window interface in the classroom'}

[0071] <Example Output>Keywords: · 'Demonstrate clearly' (from "In primary school information, teachers need to demonstrate every step of the operation as clearly as possible, and the gap between the two is very large.") · 'Diverse problems' (from "The problems raised will also be diverse. Sometimes they just fit the teaching tasks of this class, and sometimes they are about previous or subsequent content.") · 'Teaching progress' (from "In classroom teaching, it is necessary to slow down the teaching progress and pay attention to students' feedback in a timely manner.") · 'Solve difficult points' (from "Therefore, most teaching teachers will spend time on how to solve difficult points.") · 'Students' feedback' (from "In classroom teaching, it is necessary to slow down the teaching progress and pay attention to students' feedback in a timely manner.")< / Example Output>< / Task Example>"

[0072] from assistant: Keywords: · 'Reflect on problems' (from "Did the teacher find their own problems through reflection?") · 'Students' dominant position' (from "Most of the time, I was still explaining or playing videos, with little interaction with students, and the dominant position of students was not fully reflected.") · 'Lack of interaction' (from "But there were no students actively answering questions, and basically the teacher had to call on students by name.") · 'Process teaching' (from "The new curriculum emphasizes paying attention to process teaching, but in some teachers' classroom teaching, the whole class is just about exploring formulas.") · 'Blackboard writing record' (from "The state of the font is very poor, which may affect students' blackboard writing records.")

[0073] Note: Since the keyword extraction task is a large - model inference, the content in 'from assistant' should not be regarded as the real labels in the supervised task

[0074] Scoring instruction template:

[0075] from user:There are many knowledge points in this lesson, which are easy to be confused. Students have some difficulties in understanding factors, prime factors, common factors and the greatest common factor. They also need to practice more on the process and methods of finding the greatest common factor. In class, through situational questions and diagrams, students were helped to better understand and summarize the concepts of common factors and the greatest common factor. Three methods of finding the greatest common factor were learned through specific examples. Among them, the concept of relatively prime numbers was also studied and judged, and three summaries of relatively prime numbers were supplemented. In the second class, the summary and sorting of the three methods were lacking in the method of finding the greatest common factor, and the awareness of cultivating students to find the greatest common factor according to the required method was not developed. This should be summarized and consolidated to pave the way for finding the least common multiple and the least common multiple.

[0076] from assistant:PA-1: There are many knowledge points in this lesson, which are easy to be confused. Students have some difficulties in understanding factors, prime factors, common factors and the greatest common factor. The awareness of cultivating students to find the greatest common factor according to the required method was not developed.

[0077] from user:Please score the problem awareness dimension according to the sentences corresponding to the observation points you found above! Note that the score range is from 0 to 6.

[0078] from assistant: The score for the problem awareness dimension is: 4

[0079] Form user: Comprehensive Thinking: CT-1: Whether the perspective of the reflector's reflection on the problem is single or multiple (analyze and reflect on the problem from different dimensions such as teachers, teaching, students, environment, culture, technology, interpersonal relationships, ethics, etc., more than 3 points) CT-2: Whether the reflector can do more than just focus on teaching and the problem itself CT-3: Whether the reflector can reflect on specific problems in the overall system of subject teaching CT-4: Whether the reflector demonstrates a holistic analysis and improvement thinking. Please strictly comply with the scoring criteria to find the sentences corresponding to the observation points from the original text!

[0080] From assistant: CT-1: No obvious sentences corresponding to the observation points were found. CT-2: No obvious sentences corresponding to the observation points were found. CT-3: No obvious sentences corresponding to the observation points were found. CT-4: No obvious sentences corresponding to the observation points were found.

[0081] ……

[0082] Note: From user is the input, and from assistant is the expected answer from the large model, that is, the label.

[0083] Conventional multi-task evaluation instruction template:

[0084] Task id: 1

[0085] from: user: <Original text>There are many knowledge points in this lesson and they are easy to be confused. Students have certain difficulties in understanding factors, prime factors, common factors and the greatest common factor. They also need to practice more on the process and methods of finding the greatest common factor. In class, through situational questions and diagrams, students were helped to better understand and summarize the concepts of common factors and the greatest common factor. Through specific examples, three methods of finding the greatest common factor were learned, among which the concept of relatively prime numbers was also studied and judged, and three generalizations of relatively prime numbers were supplemented. In the second class, in the method of finding the greatest common factor, the summary and clarification of the three methods were lacking, and the awareness of cultivating students to find the greatest common factor according to the required method was not developed. This should be summarized and consolidated to lay a foundation and make preparations for finding the least common multiple and the least common multiple.< / Original text><Scoring criteria>Problem awareness: PA-1: Whether the teacher found their own problems through reflection; PA-2: Whether the problems found by the teacher are accurate, reasonable and valuable, such as paying attention to the confusion of students in classroom teaching, the deficiencies of teaching design, the rationality of teaching content, the suitability of teaching methods, teaching evaluation, etc.; PA-3: Whether the teacher clearly expressed the problems found in the reflection text; PA-4: Whether the reflector analyzed the problems necessary; PA-5: Regarding the analysis of the problems, whether the reflection is superficial or touches on the essence of the problems; PA-6: Whether the reflector thought about the feasible directions and measures to solve the problems< / Scoring criteria> Please try your best to find the sentences corresponding to the observation points from the original text in strict accordance with the scoring criteria!"

[0086] from assistant: PA-1: There are many knowledge points in this lesson and they are easy to be confused. Students have certain difficulties in understanding factors, prime factors, common factors and the greatest common factor. The awareness of cultivating students to find the greatest common factor according to the required method was not developed; PA-2: The awareness of cultivating students to find the greatest common factor according to the required method was not developed; PA-3: The awareness of cultivating students to find the greatest common factor according to the required method was not developed; PA-4: They also need to practice more on the process and methods of finding the greatest common factor. This should be summarized and consolidated to lay a foundation and make preparations for finding the least common multiple and the least common multiple. The summary and clarification of the three methods were lacking; PA-5: No obvious sentences corresponding to the observation points were found; PA-6: This should be summarized and consolidated to lay a foundation and make preparations for finding the least common multiple and the least common multiple.< /

[0087] from user: Please score for the problem awareness dimension according to the sentences corresponding to the observation points you found above! Note that the score range is from 0 to 6!!! It must be in this form and only an integer should be output!

[0088] from assistant: Score for the problem awareness dimension: 4

[0089] task id: 2

[0090] from: user: <Original text> There are many knowledge points in this lesson and they are easy to be confused. Students have certain difficulties in understanding factors, prime factors, common factors and the greatest common factor. They also need to practice more on the process and methods of finding the greatest common factor. In class, through situational questions and diagrams, students were helped to better understand and summarize the concepts of common factors and the greatest common factor. Through specific examples, three methods of finding the greatest common factor were learned. Among them, relatively prime numbers were also studied and judged, and three summaries of relatively prime numbers were supplemented. In the second class, in the method of finding the greatest common factor, the summary and sorting of the three methods were lacking, and the awareness of students to find the greatest common factor according to the required method was not cultivated. This should be summarized and consolidated to pave the way for finding the least common multiple and the least common multiple. < / Original text><Scoring criteria for comprehensive thinking: CT-1: Whether the perspective of the reflector's reflection on the problem is single or multiple (analyze and reflect on the problem from different dimensions such as teachers, teaching, students, environment, culture, technology, interpersonal relationships, ethics, etc., more than 3 points) CT-2: Whether the reflector not only pays attention to the teaching and the problem itself CT-3: Whether the reflector can place the specific problem in the overall system of subject teaching for reflection CT-4: Whether the reflector demonstrates a holistic analysis and improvement thinking< / Scoring criteria> Please try your best to find the sentences corresponding to the observation points from the original text according to the scoring criteria! "

[0091] from assistant: CT-1: No obvious sentences corresponding to the observation points were found. CT-2: No obvious sentences corresponding to the observation points were found. CT-3: No obvious sentences corresponding to the observation points were found. CT-4: No obvious sentences corresponding to the observation points were found. "

[0092] from user: Please score for the comprehensive thinking dimension according to the sentences corresponding to the observation points you found above! Note that the score range is from 0 to 4!!! It must be output in this form and only an integer, be sure to output it like this! "

[0093] from assistant: "The score for the comprehensive thinking dimension is: 0"

[0094] Note: Under the conventional multi-task setting, the task id is required as an identifier between different tasks. Therefore, although the above is clearly a document, it becomes two samples when evaluating two dimensions, that is, the document to be evaluated will be repeatedly combined with evaluation instructions of different dimensions, resulting in resource consumption. "

[0095] Example 2

[0096] Based on Example 1, refer to Figure 3 and this example provides a system for multi-dimensional evaluation of teachers' reflection texts based on document theme perception, including:

[0097] A data acquisition module is used to obtain the teacher reflection text data to be evaluated;

[0098] The topic distribution extraction module is used to convert the teacher reflection text data into bag-of-words representation and obtain the topic distribution of the teacher reflection text using the pre-trained topic embedding model;

[0099] The reflective text evaluation module is used to embed the teacher reflection text data into a preset scoring instruction template, use the topic distribution as the distribution route, and utilize a large model including LoRA blocks to extract local sentences corresponding to each topic from the global teacher reflection text data, obtain the scores corresponding to each topic, and realize the evaluation of the teacher reflection text.

[0100] Example 3

[0101] See also Figure 4 This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the multi-dimensional evaluation method of teacher reflection text based on document theme perception as described in Example 1.

[0102] like Figure 4 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The multi-dimensional evaluation method of teacher reflection text based on document theme perception. Of course, in addition to the software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0103] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0104] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0105] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A multi-dimensional evaluation method for teacher reflection text based on document theme perception, characterized in that: The steps include: Obtain the teacher reflection text data to be evaluated; The teacher reflection text data is converted into a bag-of-words representation, and a pre-trained topic embedding model is used to obtain the topic distribution of the teacher reflection text; The teacher reflection text data is embedded in a preset scoring instruction template, and the topic distribution is used as the distribution route. A large model including LoRA blocks is used to extract local sentences corresponding to each topic from the global teacher reflection text data, and the scores corresponding to each topic are obtained to realize the evaluation of the teacher reflection text.

2. According to claim 1, a multi-dimensional evaluation method for teacher reflection text based on document theme perception is characterized in that: The training process of the topic embedding model includes: The annotated sentences are obtained and divided according to the topic to obtain a set of sentences corresponding to the topic; For each dimension, randomly sample a batch of sentences from the corresponding sentence set, embed the sentences into a preset keyword extraction instruction template, and use the large model to generate keywords that represent the batch of sentences; The extracted keywords are used as the input of the large model to extract the hidden state representation, and the semantic embedding vectors corresponding to each topic are obtained through pooling operations. The semantic embedding vectors corresponding to each topic are concatenated to obtain the topic embedding prior distribution; The topic embedding model is trained using the topic embedding prior distribution as supervision.

3. According to claim 2, a multi-dimensional evaluation method for teacher reflection text based on document theme perception is characterized in that: The training of the topic embedding model using the topic embedding prior distribution as supervision includes: The teacher reflection text data to be evaluated is converted into a bag-of-words representation, and the topic embedding prior distribution is used as a supervisory signal. The topic embedding model is supervised trained with the goal of minimizing the difference between the generated topic embedding distribution and the topic embedding prior distribution to generate topics and their distribution related to the evaluation dimensions.

4. According to claim 1, a multi-dimensional evaluation method for teacher reflection text based on document theme perception is characterized in that: The construction process of the LoRA block includes the following steps: A plurality of LoRA blocks are constructed in each dense layer of the large model, and the LoRA blocks are trained during the forward propagation process of the large model in response to the scoring instruction template.

5. According to claim 4, a multi-dimensional evaluation method for teacher reflection text based on document theme perception is characterized in that: The number of LoRA blocks matches the number of topics.

6. According to the method of claim 1, the multi-dimensional evaluation method of teacher reflection text based on document theme perception is characterized in that: The process of extracting local sentences corresponding to each topic from the global teacher reflection text data includes the following steps: The teacher reflection text data embedded in the scoring instruction template, the description of the scoring dimension corresponding to the topic and the instruction text are used as the input of the large model, and the sentences corresponding to the topic are extracted from the teacher reflection text data.

7. The method for multi-dimensional evaluation of teacher reflection text based on document theme perception according to claim 1 is characterized in that: The process of obtaining the scores corresponding to each topic includes the following steps: The task instructions in the form of multi-round dialogues are constructed as the architecture. After the big model gives answers to the sentences corresponding to each topic, scoring instructions are continued to be given, and the big model is used to output the scores corresponding to each topic.

8. According to claim 1, a multi-dimensional evaluation method for teacher reflection text based on document theme perception is characterized in that: The themes described include text expression, problem meaning, integrated thinking, method construction, and continuous growth as the five dimensions of the multidimensional evaluation.

9. A system for multi-dimensional evaluation of teacher reflection texts based on document theme perception, characterized in that: include: The data acquisition module is used to obtain the teacher reflection text data to be evaluated; A topic distribution extraction module is used to convert the teacher reflection text data into a bag-of-words representation, and obtain the topic distribution of the teacher reflection text using a pre-trained topic embedding model; The reflective text evaluation module is used to embed the teacher reflection text data into a preset scoring instruction template, use the topic distribution as the distribution route, and utilize a large model including LoRA blocks to extract local sentences corresponding to each topic from the global teacher reflection text data, obtain the scores corresponding to each topic, and realize the evaluation of the teacher reflection text.

10. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory stores one or more programs, wherein the one or more programs include instructions for executing the multi-dimensional evaluation method of teacher reflection text based on document theme perception as described in any one of claims 1-8.

Images