Physical, chemical and biological experiment report scoring method and system based on multi-modal large model

Abstract

The invention relates to the technical field of artificial intelligence and experiment teaching, in particular to a physicochemical and student experiment report scoring method and system based on a multi-modal large model.The method comprises the steps that questions and answers input by a user are obtained, and question data of the questions are retrieved and recalled through a scoring standard knowledge base; wherein the questions comprise blank filling questions and drawing questions; the question data comprises questions, reference answers, scoring standards and question scores; the reference answers of the drawing questions comprise correct pictures, character descriptions of the correct pictures and character interpretations of element symbols; integrating the answer input by the user and the question data of the question into the input content of a multi-modal large model by using the thinking chain cue word template; the input content is scored and answered through the multi-modal large model, and the score and the scoring basis of the question are obtained; according to the invention, through more accurate score feedback, the education quality can be improved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and experimental teaching technology, and in particular to a physical, chemical and biological experiment report scoring method and system based on a multimodal large model. Background Art

[0002] With the continuous development of educational technology, the teaching and scoring methods of physics, chemistry and biology experiments are also gradually improving. The traditional experimental examination scoring mainly relies on the on-site manual scoring of teachers. This method is not only time-consuming and laborious, but also easily affected by subjective factors, resulting in inconsistent scoring standards, affecting the fairness and accuracy of scoring. In recent years, with the advancement of educational informatization, a large number of automatic scoring systems for physics, chemistry and biology experimental examinations have emerged, but these systems are still not satisfactory in the automatic scoring of experimental reports, especially when dealing with open-ended short-answer fill-in-the-blank questions and drawing questions. Summary of the invention

[0003] The object of the present invention is to provide a method and system for scoring physical, chemical and biological experiment reports based on a multimodal large model to solve at least one technical problem in the background technology.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] In a first aspect, an embodiment of the present invention provides a method for scoring a physical, chemical, and biological experiment report based on a multimodal large model, the method comprising the following steps:

[0006] Obtain the question and the answer input by the user, and retrieve the question data of the question through the scoring standard knowledge base; wherein the question includes fill-in-the-blank questions and drawing questions; the question data includes the question, reference answer, scoring standard and question score; the reference answer of the drawing question includes the correct picture, the text description of the correct picture and the text explanation of the component symbol;

[0007] Apply the thinking chain prompt word template to integrate the user's input answer and the question data into the input content of the multimodal large model;

[0008] The input content is scored and answered through a multimodal large model to obtain the score and scoring basis for the question.

[0009] Optionally, the method further comprises:

[0010] Creating a structured scoring document, wherein the scoring document includes a plurality of question data;

[0011] Preprocess each of the question data, and vectorize the preprocessed question data using an embedding encoder to obtain an embedding vector corresponding to each of the question data;

[0012] The embedded vector corresponding to each question data is stored in the vector database to form a scoring standard knowledge base for lab report scoring.

[0013] Optionally, obtaining the question and the answer input by the user, and retrieving the question data of the question through the scoring standard knowledge base, includes:

[0014] Obtain the answer entered by the user when answering the question, and form a query word with the answer entered by the user and the corresponding question;

[0015] The query word is vectorized using an embedding encoder to obtain a semantic feature vector, an embedding vector that is most similar to the semantic feature vector is retrieved from a scoring standard knowledge base, and the question data corresponding to the embedding vector is used as the question data of the question.

[0016] Optionally, the method further comprises:

[0017] A multimodal large model is determined, and supervised fine-tuning of the multimodal large model is performed through a thought chain prompt word template for lab report scoring.

[0018] Optionally, the application of the thought chain prompt word template to integrate the answer input by the user and the question data of the question into the input content of the multimodal large model includes:

[0019] The matched question data is used as context reference information and integrated into the thinking chain prompt word template as the input content of the multimodal large model; wherein the thinking chain prompt word template is used to instruct the multimodal large model to extract information, understand information, and infer results in the mode of thinking chain, and output the basis for scoring.

[0020] Optionally, scoring and answering the input content through the multimodal large model to obtain the score of the question and the basis for the score include:

[0021] Inputting the input content into the multimodal macromodel, performing reasoning according to the instructions of the thought chain prompt word template through the multimodal macromodel, and outputting a scored answer to the question;

[0022] Extract the score and scoring basis from the scoring answer, output the score of the question, and display the scoring basis.

[0023] In a second aspect, an embodiment of the present invention provides a physical, chemical and biological experiment report scoring system based on a multimodal large model, the system comprising:

[0024] at least one processor;

[0025] at least one memory for storing at least one program;

[0026] When the at least one program is executed by the at least one processor, the at least one processor implements any one of the methods described above.

[0027] The beneficial effects of the present invention are as follows: the present invention discloses a method and system for scoring physics, chemistry and biology experiment reports based on a multimodal large model. The present invention realizes automatic scoring of physics, chemistry and biology experiment reports by introducing multimodal large model technology, and significantly improves the efficiency and accuracy of scoring. By utilizing the text and visual understanding capabilities of the multimodal large model, the present method can effectively understand the scoring criteria expressed in natural language, and score various types of questions such as open-ended short-answer fill-in-the-blank questions and drawing questions as required. The present invention not only outputs the scoring results, but also provides detailed scoring basis, making the scoring process more transparent, and both students and teachers can understand the basis for scoring. In the scoring of drawing questions, compared with traditional visual algorithms, the small sample reasoning ability of the multimodal large model reduces the dependence on a large amount of labeled question data, and reduces development and maintenance costs. Through more accurate scoring feedback, the present invention enables students to understand their learning situation more clearly, and teachers can also adjust teaching strategies more effectively, thereby improving the quality of education. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0029] Figure 1 It is a flow chart of a method for scoring a physical, chemical and biological experiment report based on a multimodal large model in an embodiment of the present invention;

[0030] Figure 2 It is a structural schematic diagram of a physical, chemical and biological experiment report scoring system based on a multimodal large model in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0032] With the rapid development of multimodal large-scale artificial intelligence technologies such as language and vision, it has become possible to use these technologies to automatically score graphic and textual laboratory reports, which provides a new solution for improving the efficiency and quality of laboratory examination scoring. Multimodal large-scale models can effectively integrate text and image information, improve the accuracy and reliability of scoring, and thus bring revolutionary changes to educational assessment.

[0033] However, the solutions in the related art have the following problems:

[0034] The problem of semantic matching between the input of open-ended short-answer fill-in-the-blank questions and the reference answers: Traditional natural language processing methods mainly rely on keyword matching and simple grammatical analysis, but due to the diversity and flexibility of students' answers, existing natural language processing methods are difficult to accurately match with the reference answers, resulting in large scoring errors.

[0035] Problems with visual algorithm matching judgment for drawing questions: Drawing questions involve graphics, symbols, and annotations. Existing visual algorithms have difficulty in identifying these contents, especially when dealing with subtle differences between the graphics drawn by students and the reference answers or when the annotation positions are inaccurate. In addition, the visual algorithm development cycle is long and costly, and it only outputs the judgment results and confidence levels, which lacks interpretability.

[0036] The scoring criteria for experimental report topics vary according to different textbooks and teaching requirements. Generally, deep learning methods must undergo supervised training to adapt to different scoring methods. The time and resource costs of algorithm development are huge, and it is impossible to adjust online to adapt to different scoring criteria.

[0037] To solve the problems in the background technology, the embodiments of the present invention provide the following technical solutions.

[0038] See also Figure 1 , Figure 1 The present invention provides a method for scoring physical, chemical and biological experiment reports based on a multimodal large model, the method comprising the following steps:

[0039] S100, obtaining a question and an answer input by a user, and retrieving question data of the question through a scoring standard knowledge base; wherein the question includes a fill-in-the-blank question and a drawing question; the question data includes a question, a reference answer, a scoring standard, and a question score; the reference answer of the drawing question includes a correct figure, a text description of the correct figure, and a text explanation of the component symbol;

[0040] Specifically, scoring standards, reference answers, and question scores are formulated for short-answer fill-in-the-blank questions and drawing questions respectively. The reference answers for drawing questions include, in addition to the correct diagram, a text description of the correct diagram and a text explanation of component symbols, such as the name and connection relationship of the circuit diagram, and the sub-images of each component symbol and their corresponding names.

[0041] S200, applying the thinking chain prompt word template, integrating the answer input by the user and the question data of the question into the input content of the multimodal large model;

[0042] S300, scoring and answering the input content through the multimodal large model to obtain the score of the question and the basis for the scoring.

[0043] In the embodiment provided by the present invention, by introducing the multimodal large model technology, automatic scoring of physics, chemistry and biology experiment reports is realized, which significantly improves the efficiency and accuracy of scoring. By utilizing the text and visual understanding capabilities of the multimodal large model, this method can effectively understand the scoring criteria expressed in natural language, and score various types of questions such as open-ended short-answer fill-in-the-blank questions and drawing questions according to the requirements.

[0044] The present invention not only outputs the scoring results, but also provides detailed scoring basis, making the scoring process more transparent, and both students and teachers can understand the basis of scoring. In the scoring of drawing questions, compared with traditional visual algorithms, the small sample reasoning ability of multimodal large models reduces the dependence on a large amount of labeled question data, reducing development and maintenance costs. Through more accurate scoring feedback, the present invention enables students to understand their learning situation more clearly, and teachers can also adjust teaching strategies more effectively, thereby improving the quality of education.

[0045] In some embodiments, before S100, the method further includes:

[0046] S101, creating a structured scoring document, wherein the scoring document includes a plurality of question data;

[0047] Specifically, a structured scoring document is created, and each question data in the scoring document includes a question described in text, a reference answer in the form of pictures and texts, a scoring standard in the form of pictures and texts, and a question score.

[0048] S102, preprocessing each of the question data, and vectorizing the preprocessed question data using an embedded encoder to obtain an embedded vector corresponding to each of the question data;

[0049] Specifically, the title data of the scoring document is cleaned to remove invalid or erroneous information to ensure the quality of the title data. For text content, word segmentation and stop word removal are performed; for image content, operations such as cropping and scaling are performed to make it meet the input requirements of the multimodal large model, and the path representation of the image in the server is used in the document; then, each title data of the scoring document is vectorized using an embedding encoder to obtain an embedding vector; the embedding encoder can be one of BERT, BCE-Embedding, text2vec, etc.

[0050] S103, storing the embedded vector corresponding to each question data into a vector database to form a scoring standard knowledge base for lab report scoring.

[0051] Specifically, the embedded vector after encoding each question data is stored in the scoring standard knowledge base to form a scoring standard knowledge base for the experimental report. The scoring standard knowledge base can be one of FAISS, Milvus, PGVector, etc. The scoring standard knowledge base can update the scoring method of the question online, and the multimodal large model can automatically score according to the scoring method without repeated training.

[0052] In some embodiments, in S100, the step of obtaining the question and the answer input by the user, and retrieving the question data of the question through the scoring standard knowledge base includes:

[0053] S110, obtaining the answer input by the user when answering the question, and forming a query word with the answer input by the user and the corresponding question;

[0054] Specifically, the examinee enters the answer to a question in the information-based examination platform, and obtains the "question-answer" entered by the examinee to form a query term.

[0055] S120, use an embedding encoder to vectorize the query word to obtain a semantic feature vector, retrieve an embedding vector that is most similar to the semantic feature vector in the scoring standard knowledge base, and use the question data corresponding to the embedding vector as the question data of the question.

[0056] Specifically, the question data entered by the examinee is vectorized using an embedding encoder, and the semantic feature vector is retrieved from the scoring standard knowledge base to find the most similar embedding vector. This embedding vector is the scoring method knowledge for the question. The vector similarity calculation method can be one of cosine similarity, BM25, Tfidf, etc.

[0057] In some embodiments, before S200, the method further includes:

[0058] A multimodal large model is determined, and supervised fine-tuning of the multimodal large model is performed through a thought chain prompt word template for lab report scoring.

[0059] Specifically, a multimodal large model with enhanced retrieval capability and instruction comprehension capability is selected, which may be one of Qwen-VL, MiniCPM-V, GLM-4V, etc., and the multimodal large model is fine-tuned with supervision of special instruction prompt words for experimental report scoring.

[0060] In some embodiments, in S200, the application of the thought chain prompt word template to integrate the answer input by the user and the question data of the question into the input content of the multimodal large model includes:

[0061] The matched question data is used as context reference information and integrated into the thinking chain prompt word template as the input content of the multimodal large model; wherein the thinking chain prompt word template is used to instruct the multimodal large model to extract information, understand information, and infer results in the mode of thinking chain, and output the basis for scoring.

[0062] Specifically, the matched question data is used as contextual reference information, integrated into the instruction prompt word template dedicated to lab report scoring, and integrated into the input of the multimodal large model. The multimodal large model is required to extract information, understand information, and infer results in a chain of thought mode, and at the same time, it is required to output an analysis of the reasons for scoring.

[0063] In some embodiments, in S300, the step of scoring and answering the input content using the multimodal macro model to obtain the score of the question and the basis for the score includes:

[0064] S310, inputting the input content into the multimodal big model, and using the multimodal big model to perform reasoning according to the instructions of the thought chain prompt word template, and outputting a scored answer to the question;

[0065] S320, extracting the score and scoring basis from the scoring answer, outputting the score of the question, and displaying the scoring basis.

[0066] Specifically, the integrated instructions are input into the multimodal big model, and the multimodal big model performs reasoning according to the instruction requirements and outputs a scoring answer. The required information is extracted from the answer output by the multimodal big model, such as the score and scoring basis (including scoring reasons and analysis), to complete the automatic scoring of the experimental report.

[0067] It can be seen from the above embodiments that the present invention has the following advantages compared with the prior art:

[0068] 1. This method uses the multimodal large model's ability to understand text and visual modalities and generate text modalities to automatically score graphic and textual laboratory reports in an effective and feasible manner.

[0069] 2. This method uses the RAG of the multimodal large model to establish a knowledge base of scoring criteria for short-answer fill-in-the-blank questions, retrieves reference answers and scoring criteria through the questions to which the examinee's answers belong, and combines scoring instructions with retrieval results to form prompt words to allow the multimodal large model to deduce and score the examinee's answers; RAG (Retrieval-Augmented Generation) is an artificial intelligence technology that combines information retrieval technology with language generation models. This technology retrieves relevant information from an external knowledge base and inputs it as a prompt to a large language model (LLMs) to enhance the model's ability to handle knowledge-intensive tasks, such as question answering, text summarization, and content generation.

[0070] 3. This method utilizes the visual modality small sample reasoning ability of the multimodal large model, and uses a small number of reference answer images and text descriptions of the scoring criteria for drawing questions to provide natural language and visual prompts to the multimodal large model, so that the multimodal large model has the ability to understand the customized scoring criteria and reference answers and automatically score and interpret the drawing results submitted by the candidates.

[0071] and Figure 1 Corresponding to the method, refer to Figure 2 The embodiment of the present invention provides a physical, chemical and biological experiment report scoring system based on a multimodal large model, including:

[0072] at least one processor;

[0073] at least one memory for storing at least one program;

[0074] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0075] It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0076] It will be appreciated by those skilled in the art that all or some of the methods disclosed above, the system can be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, and the computer-readable medium can include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0077] The above is a specific description of the preferred implementation of the present disclosure, but the present disclosure is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present disclosure. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present disclosure.

Claims

1. A method for scoring physics, chemistry and biology experiment reports based on a multimodal large model, characterized in that: The method comprises the following steps: Obtain the question and the answer input by the user, and retrieve the question data of the question through the scoring standard knowledge base; wherein the question includes fill-in-the-blank questions and drawing questions; the question data includes the question, reference answer, scoring standard and question score; the reference answer of the drawing question includes the correct picture, the text description of the correct picture and the text explanation of the component symbol; Apply the thinking chain prompt word template to integrate the user's input answer and the question data into the input content of the multimodal large model; The input content is scored and answered through a multimodal large model to obtain the score and scoring basis for the question.

2. The method according to claim 1, characterized in that The method further comprises: Creating a structured scoring document, wherein the scoring document includes a plurality of question data; Preprocess each of the question data, and vectorize the preprocessed question data using an embedding encoder to obtain an embedding vector corresponding to each of the question data; The embedded vector corresponding to each question data is stored in the vector database to form a scoring standard knowledge base for lab report scoring.

3. The method according to claim 2, characterized in that The step of obtaining the question and the answer input by the user, and retrieving the question data of the question through the scoring standard knowledge base, includes: Obtain the answer entered by the user when answering the question, and form a query word with the answer entered by the user and the corresponding question; The query word is vectorized using an embedding encoder to obtain a semantic feature vector, an embedding vector that is most similar to the semantic feature vector is retrieved from a scoring standard knowledge base, and the question data corresponding to the embedding vector is used as the question data of the question.

4. The method according to claim 1, characterized in that: The method further comprises: A multimodal large model is determined, and supervised fine-tuning of the multimodal large model is performed through a thought chain prompt word template for lab report scoring.

5. The method according to claim 4, characterized in that The application of the thought chain prompt word template integrates the answer input by the user and the question data of the question into the input content of the multimodal large model, including: The matched question data is used as context reference information and integrated into the thinking chain prompt word template as the input content of the multimodal large model; wherein the thinking chain prompt word template is used to instruct the multimodal large model to extract information, understand information, and infer results in the mode of thinking chain, and output the basis for scoring.

6. The method according to claim 5, characterized in that The input content is scored and answered by the multimodal large model to obtain the score and scoring basis of the question, including: Inputting the input content into the multimodal macromodel, performing reasoning according to the instructions of the thought chain prompt word template through the multimodal macromodel, and outputting a scored answer to the question; Extract the score and scoring basis from the scoring answer, output the score of the question, and display the scoring basis.

7. A physical, chemical and biological experiment report scoring system based on a multimodal large model, characterized in that: The system comprises: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 6.

Images