Subjective question scoring system and method based on large language model combination weighting

Abstract

The invention provides a subjective question scoring system and method based on large language model combination weighting, and the system comprises a key point matching scoring module which is used for recognizing an answer to be scored based on a keyword, and obtaining a key point matching score; the similarity comparison scoring module is used for scoring the similarity between the to-be-scored answer and the standard answer semantics to obtain a semantic similarity comparison score; the language model scoring module is used for obtaining subjective question scores based on a large language model on the basis of a large language model fusion prompt word engineering technology; and the weighted scoring module is used for weighting each scoring result to obtain a final subjective question scoring result. According to the technical scheme, scoring errors caused by limitation of a single method can be effectively avoided, so that the scoring accuracy and credibility are remarkably improved. The integrated design of the method not only enhances the robustness of the scoring system, but also brings a new technical breakthrough for the subjective question scoring field.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a subjective question scoring system and method based on large language model combined weighting. Background Art

[0002] To address the limitations of traditional manual scoring methods, scholars and research teams have introduced information technology and artificial intelligence (AI) methods in recent years, building relevant information systems and software features to address the challenges of subjective scoring. Existing methods can be broadly divided into three categories: First, subjective scoring methods based on key point matching, which scores by comparing the key point matching levels between the subject's answer and the standard answer. However, this method has certain limitations, particularly in its lack of sensitivity in identifying keywords with similar semantics, which can lead to large errors in the scoring results.

[0003] To address this issue, some scholars have proposed a scoring method for subjective questions based on semantic similarity comparison. This method calculates the semantic distance between the answer and the standard answer to score. The higher the semantic similarity between the answer and the standard answer, the closer the semantic distance, resulting in a higher score. This method effectively avoids the insensitivity to keyword recognition found in traditional key point matching scoring methods, as it no longer relies solely on keyword matching, but instead focuses on the overall semantic content of the answer.

[0004] However, the primary problem with this approach is that the criteria for identifying similarity are unclear. Because semantic similarity scoring relies on the algorithm's understanding of the content of the answer, it can result in some answers receiving a certain score even if they simply copy the question or are similar in form to the standard answer. In this case, the scoring system may be unable to accurately distinguish between answers that truly understand the question and mechanical reproduction, affecting the fairness and accuracy of the scoring.

[0005] Furthermore, some scholars have proposed a deep learning-based scoring method for subjective questions. Leveraging the learning capabilities of deep neural networks, this method can effectively identify and handle polysemy. It can also accurately identify verbatim copying, preventing the practice of simply copying the content to score points. However, the flexibility and sensitivity of deep learning models to training parameters also pose challenges. Due to the numerous parameters in deep learning networks, individual scoring results can vary due to slight changes in these parameters, leading to instability in the scoring results. Deep network models must be trained for specific problems, making them inflexible and inapplicable.

[0006] Furthermore, all of the aforementioned methods employ a single technical approach, addressing the scoring of subjective questions from a specific perspective. However, these approaches lack universality and comprehensiveness, failing to address the inconsistencies in scoring principles across different questions. Furthermore, scoring criteria cannot be tailored to the assessment objectives. These characteristics also require customized development of assessment and scoring information systems tailored to specific assessment criteria and requirements, preventing the universal scoring of subjective questions through unified functionality. This limits the versatility and comprehensiveness of the scoring systems. Summary of the Invention

[0007] To address the challenges of the existing technology, the present invention provides a subjective question scoring system and method based on a combined weighted approach using a large language model. This method effectively avoids scoring errors caused by the limitations of a single method, thereby significantly improving scoring accuracy and reliability. This integrated design not only enhances the robustness of the scoring system but also represents a new technological breakthrough in the field of subjective question scoring.

[0008] A subjective question scoring system based on a combined weighted large language model, including:

[0009] A key point matching scoring module is used to extract keywords from the standard answers to subjective questions and identify the answers to be scored based on the keywords to obtain a key point matching score;

[0010] A similarity comparison and scoring module is used to score the semantic similarity between the answer to be scored and the standard answer to obtain a semantic similarity comparison score;

[0011] A language model scoring module is used to score the answers to be scored using the standard answers based on the large language model and to obtain a score for the subjective question based on the large language model;

[0012] The weighted scoring module is used to weight the key point matching score, the semantic similarity comparison score and the subjective question score based on the large language model to obtain the final subjective question scoring result.

[0013] Preferably, the key point matching scoring module includes:

[0014] A keyword extraction unit, configured to extract keywords from the standard answer to obtain a key information sequence;

[0015] A weight setting unit, configured to obtain a weight of the key point matching method based on the importance of the key point matching method;

[0016] A regular matching recognition unit, configured to automatically match and identify whether the answer to be scored contains the keywords in the key information sequence using regular matching technology to obtain a regular matching result;

[0017] The key point matching scoring unit is configured to obtain the key point matching score based on the regular matching result and the key point matching method weight.

[0018] Preferably, the similarity comparison and scoring module includes:

[0019] A fragmentation retrieval unit is used to split the knowledge points of the standard answer and the knowledge points of the answer to be scored respectively and convert them into knowledge point vectors based on a word vector model;

[0020] A similarity calculation unit is used to calculate the semantic similarity between the standard answer knowledge point vector and the knowledge point vector of the answer to be scored;

[0021] The similarity comparison unit is used to compare the semantic similarity with a preset similarity threshold, determine whether to perform scoring processing on the knowledge point vector to be scored, and obtain the semantic similarity comparison score.

[0022] Preferably, the language model scoring module includes:

[0023] A prompt word construction unit is used to construct prompt words based on the question of the question to be graded, the keywords of the standard answer and the answer to be graded;

[0024] The large language model scoring unit is used to score the answers to be scored based on the prompt words using the large language model to obtain a subjective question score based on the large language model.

[0025] The present invention also provides a subjective question scoring method based on combined weighting of a large language model, which uses the system and includes:

[0026] Extract keywords from the standard answers to the subjective questions, and identify the answers to be scored based on the keywords to obtain a key point matching score;

[0027] Scoring the semantic similarity between the answer to be scored and the standard answer to obtain a semantic similarity comparison score;

[0028] Based on the large language model fusion prompt word engineering technology, the standard answer is used to score the answer to be scored, and the subjective question score based on the large language model is obtained;

[0029] The key point matching score, the semantic similarity comparison score, and the subjective question score based on the large language model are weighted to obtain a final subjective question score result.

[0030] Preferably, the method for obtaining the key point matching score includes:

[0031] Extract keywords from the standard answers to obtain key information sequences;

[0032] Based on the importance of the key point matching method, a key point matching method weight is obtained;

[0033] Automatically matching and identifying whether the answer to be scored contains the keywords in the key information sequence using regular matching technology to obtain a regular matching result;

[0034] The key point matching score is obtained based on the regular matching result and the key point matching method weight.

[0035] Preferably, the method for obtaining the semantic similarity comparison score includes:

[0036] Based on the word vector model, the knowledge points of the standard answer and the knowledge points of the answer to be scored are split and converted into knowledge point vectors respectively;

[0037] Calculate the semantic similarity between the knowledge point vector of the standard answer and the knowledge point vector of the answer to be scored;

[0038] The semantic similarity is compared with a preset similarity threshold to determine whether to perform scoring processing on the knowledge point vector to be scored, thereby obtaining the semantic similarity comparison score.

[0039] Preferably, the method for obtaining subjective question scores based on a large language model includes:

[0040] Construct prompt words based on the question to be graded, the keywords of the standard answer, and the answer to be graded;

[0041] Based on the prompt words, the answers to be scored are scored using a large language model to obtain the subjective question scores based on the large language model.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] This invention innovatively applies large language model technology to subjective question scoring, and integrates key point matching, semantic similarity comparison, and prompt word engineering technology, which greatly improves the efficiency and accuracy of subjective question scoring and makes up for the shortcomings of traditional manual subjective question scoring.

[0044] Scoring subjective questions based on key point matching significantly improves scoring efficiency and reduces the time and cost of manual scoring. By pre-setting keywords and scoring points, this key point matching-based scoring method helps standardize scoring, ensuring that all candidates are scored according to the same standards and ensuring fairness and consistency in scoring.

[0045] The scoring of subjective questions based on semantic similarity comparison has added a semantic similarity assessment function, that is, word vectors can convert words into vector representations in a high-dimensional space. These vectors can capture the semantic similarity between words. Therefore, in the scoring process of subjective questions, by calculating the similarity between the word vectors of the scoring object's answer and the standard answer, the semantic closeness of the answer can be more accurately assessed, rather than just a superficial text match. The polysemy processing function has been added. Word vectors can handle the problem of a word having multiple meanings, mapping the different meanings of the same word to different vector representations through contextual information. This is particularly important for scoring subjective questions, because candidates may use different vocabulary from the standard answer when answering the questions, but as long as these words are semantically similar, they can get a reasonable score.

[0046] Subjective questions are scored using prompt word engineering technology. Carefully designed prompt words can guide large language models to more accurately understand the requirements and scoring criteria for subjective questions. These prompt words can include a clear explanation of the question's intent, specific instructions for answer key points, and detailed descriptions of the scoring criteria. This ensures the model captures key information during the scoring process and delivers more accurate and targeted scores. By designing multi-dimensional prompt words, a more comprehensive and in-depth assessment of answers to subjective questions can be achieved.

[0047] The present invention utilizes the deep semantic understanding, reasoning and generation capabilities of the large language model as well as the text extraction and generation capabilities, combined with key point matching, semantic similarity comparison, prompt word engineering technology, etc., to generate a subjective question scoring method based on the combined weighting of the large language model. The method is highly efficient, has a small error in the results, and is easy to unify and standardize for management and use. At the same time, the method liberates manual labor and abandons the subjectivity of manual scoring, thereby realizing the automation of the scoring process, the objectivity of the scoring results, and the smoothness of the scoring process. In general, while realizing the automation of subjective question scoring, the present invention also provides a broader prospect for the application of large language model technology and even artificial intelligence in the field of examination preparation. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 This is a schematic diagram of the structure of a subjective question scoring system based on a large language model combined weighting according to an embodiment of the present invention;

[0050] Figure 2 A logical diagram of semantic similarity comparison and scoring based on an embodiment of the present invention;

[0051] Figure 3 This is a flow chart of a subjective question scoring method based on combined weighting of a large language model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] Example 1

[0055] like Figure 1 As shown, a subjective question scoring system based on large language model combination weighting includes: a key point matching scoring module, a similarity comparison scoring module, a language model scoring module and a weighted scoring module.

[0056] The key point matching scoring module is used to extract keywords from the standard answers to subjective questions, and identify the answers to be scored based on the keywords to obtain the key point matching scores.

[0057] In a further embodiment, the key point matching scoring module includes:

[0058] The keyword extraction unit is used to extract keywords from the standard answers and obtain key information sequences. Specifically, when entering subjective questions, the scorer enters the key information sequence based on the core content of the standard answers:

[0059] A=[A1,A2,…,A m ],

[0060] Among them, A m is the mth keyword or key phrase.

[0061] These keywords or key phrases are usually the core concepts, theoretical points or important information points in the question requirements. They are rigorous, serious and unchangeable, and are key indicators for evaluating the quality of answers.

[0062] The weight setting unit is a key component used to determine the weight according to the degree of importance of the key point matching method. The key point matching method is an evaluation method that determines the degree of matching by comparing the key points in the standard answer and the answer to be scored. This process involves the following steps: First, key point extraction, which extracts key points from the standard answer and the answer to be scored. Second, matching evaluation, which compares the degree of matching between the extracted key points, usually using similarity calculation methods such as Jaccard similarity coefficient or cosine similarity. Finally, weight calculation, which calculates the weight W of the key point matching method based on the degree of matching. W reflects the degree of importance of each key point in the scoring. The weight calculation of the key point matching method can be expressed by the following formula:

[0063]

[0064] Among them, α i is the importance coefficient of each key point, S i is the matching degree of each key point, and n is the total number of key points. The weights obtained in this way can be used in subsequent scoring calculations to ensure that the scoring results can accurately reflect the importance of key point matching.

[0065] The regular matching recognition unit is used to automatically match and identify whether the answer to be scored contains the keywords in the key information sequence using regular matching technology to obtain a regular matching result; specifically, for the answer to be scored, the regular matching technology is used to automatically match and identify whether the answer contains the corresponding keywords or key phrases:

[0066] T=[T1,T2,…,T m ].

[0067] Among them, T i is the matching result of the ith keyword in the answer, which is 1 if the result is matched and 0 if not matched.

[0068] The key point matching scoring unit is used to obtain a key point matching score based on the regular matching result and the key point matching method weight. Specifically, based on the regular matching result and the preset weight, the system calculates Score1, a subjective question score based on the key point matching method.

[0069]

[0070] Where m is the total number of keywords.

[0071] This method automatically completes most keyword scoring tasks by automatically matching pre-set scoring criteria, significantly reducing the time and cost required for manual scoring. It also reduces subjectivity in the scoring process. Scoring results are based on keyword matching and weighting, eliminating the influence of human factors on the scoring results and thus improving the fairness of the scoring. Through precise key point matching and weighted calculations, the system can more accurately assess the accuracy and relevance of answers, ensuring the objectivity, consistency, and interpretability of the scoring results.

[0072] The similarity comparison scoring module is used to score the semantic similarity between the answer to be scored and the standard answer, and obtain a semantic similarity comparison score. The semantic similarity between the answer to the subjective question and the standard answer is scored. The semantic similarity comparison method in the present invention aims to deeply mine the semantic information of the text through word vector technology, and perform accurate semantic scoring based on this. The core of this method is to convert the standard answer and the answer to the scoring object into a vector representation in a multidimensional space through a word embedding model, and then evaluate the similarity between the answer and the correct answer by calculating the similarity between these vectors, so as to score the answer to the subjective question.

[0073] The present invention adopts flexible word embedding technology, which is compatible with a variety of open source models and customized models to meet the needs of different languages. Through pre-training on a large-scale corpus, words can be converted into vector representations that can reflect their semantic relationships. In the vector space, words corresponding to vectors with similar distances are also semantically similar. This feature makes word vectors an ideal tool for measuring semantic similarity. In terms of similarity calculation methods, this scheme uses cosine similarity as the main evaluation method. Cosine similarity evaluates the degree of similarity between two vectors by calculating the cosine value of the angle between them. This method can effectively quantify the semantic differences between vectors. The higher the cosine similarity value of two word vectors, the closer the two words are semantically, thereby providing an objective quantitative indicator for scoring.

[0074] This method first performs a segmented search of the standard answer and the answer of the person being graded to select the most similar text slices. These text slices are then mapped into a high-dimensional vector space using word embedding technology. In this space, the vector representations of semantically similar words or phrases are close to each other, effectively capturing and expressing the semantic relationships and contextual information between words. This process not only relies on direct connections between words but also considers broader contextual factors, enhancing the depth and breadth of semantic understanding. Next, we carefully compare the generated standard answer vector with the vector of the person being graded, and calculate the similarity between the two to obtain the score Score2.

[0075] In a further embodiment, the similarity comparison and scoring module includes:

[0076] The fragmented retrieval unit is a key component for splitting the knowledge points of the standard answer and the answer to be graded and converting them into vector representations. This process relies on word embedding models, particularly the Word2Vec model. Word2Vec maps words to vectors in a high-dimensional space, placing semantically similar words closer together.

[0077] In the specific word segmentation process, the standard answers and the answers to be graded are first segmented to extract key knowledge points. The word segmentation process generally involves the following steps: First, text preprocessing is performed to remove irrelevant information such as punctuation and stop words. Next, word segmentation is performed, using a word segmentation algorithm (such as maximum matching or minimum segmentation) to break the text into individual words or phrases. Finally, word vectorization is performed, using the Word2Vec model to convert each word or phrase into its corresponding vector representation.

[0078] The similarity calculation unit is used to calculate the semantic similarity between the standard answer knowledge point vector and the knowledge point vector of the answer to be scored. In this embodiment, cosine similarity is used as the similarity calculation method. It evaluates the similarity between two vectors by calculating the cosine value of the angle between them in the vector space. When the cosine similarity value is higher, it indicates that the two vectors are closer in semantics. Specifically, the cosine similarity S is defined as:

[0079]

[0080] Where A and B represent the knowledge point vector and the answer knowledge point vector to be graded, respectively. A·B is the dot product of the two vectors. ||A|| and ||B|| are the moduli of the vectors.

[0081] The similarity comparison unit is used to compare the semantic similarity with a preset similarity threshold, determine whether to perform scoring processing on the knowledge point vector to be scored, and obtain a semantic similarity comparison score.

[0082] In this embodiment, the calculated similarity S is compared with a preset threshold H to determine whether to score the knowledge point. The specific threshold comparison process can be expressed as follows:

[0083]

[0084] Among them, the score is S i represents the score for the i-th knowledge point. A score of 0 indicates that the answer fails to cover the knowledge point. By comparing the score with the preset threshold H, the scoring system can more objectively and fairly assess the subject's understanding of the knowledge point, improving the accuracy and efficiency of scoring.

[0085] Specifically, when constructing the scoring mechanism, the present invention comprehensively considers the two major factors of relevance and completeness. The relevance assessment aims to measure the degree of correlation between the generated answer and the question asked, and determines whether the answer accurately responds to the core points of the question by quantitatively analyzing the similarity between the words in the answer and the question or expert review. The completeness assessment focuses on whether the answer comprehensively covers all the key aspects contained in the question. This can be judged by comparing the coverage of the answer with the standard answer or through expert review. The design of the scoring logic framework ensures the scientificity, objectivity and fairness of the scoring process, thereby improving the reliability and validity of the scoring results. According to the above principles, the scoring logic diagram is as follows Figure 2 As shown, the specific steps are as follows:

[0086] 1) Slice the standard answer text and the scoring object answer text, and record the standard answer text as M1, M2, ..., M m , the answers of the scoring objects are recorded as N1, N2, ..., N n .

[0087] 2) During the scoring process, for each standard answer text M i , and compare it with all the scoring object answer text N j Compare them one by one to determine the most matching similar text slice. Specifically, calculate M i With each N j The vector similarity S between ij , and set a similarity threshold H. Only when S ij When it is greater than H, it is considered that M i With N j Similar enough. In all similar N j In the above example, select i The highest similarity N j , its similarity S ij That is M i Score D i If there is no N j If the similarity exceeds the threshold H, then M i The score of 0 indicates that the subject has failed to cover the knowledge point. This method ensures the accuracy and fairness of the scoring, making the scoring results more reliable. i It can be calculated as follows:

[0088]

[0089] Among them, max means taking the maximum value, S ij It's M i and N j The vector similarity between them, and H is the set similarity threshold. If there is no T'j If the similarity of exceeds H, then D i If it is 0, it means that the answer of the scored object is missing this knowledge point.

[0090] To ensure the integrity of the answer evaluation, we use a loop traversal strategy to calculate the similarity of all knowledge points to be evaluated, and evaluate whether the user's answer contains relevant knowledge points. The final score Score2 can be calculated as follows:

[0091]

[0092] The application of semantic similarity comparison methods in scoring subjective questions captures the semantic relationships between words, enabling an objective assessment of the semantic differences between the subject's answer and the standard answer. This overcomes the drawback of Score1, where keywords with similar semantics cannot be identified and scored. This method helps improve the comprehensiveness and fairness of scoring. It also provides new ideas and methods for the application of natural language processing technology in education.

[0093] The language model scoring module integrates prompt word engineering technology based on a large language model to score the candidate answers using standard answers, thereby obtaining a score for subjective questions based on the large language model. Large language models (LLMs) integrated with prompt word engineering technology aim to achieve a deep understanding and scoring of subjective answers. This method uses prompt words, which are arranged into the question, standard answer, and the candidate's answer, as model input. Based on carefully designed prompt word templates and rules, it guides or influences the scoring system's judgment process and calculates the score through the large language model to obtain Score3.

[0094] In a further embodiment, the language model scoring module includes:

[0095] When grading subjective questions, the following issues need to be considered: First, the specific requirements of the grading task must be clearly defined, such as assessing the clarity, logic, and accuracy of the language. Cue engineering can draw on this approach by designing clear, task-oriented cues to help the grading system more accurately understand the scoring objectives. Cue design is crucial. While not directly generating output for the model, cues can be designed to guide the scoring system's attention and judgment. These cues can include keywords, phrases, or short sentences, aiming to cover the main scoring dimensions of subjective questions. By analyzing factors influencing the scoring logic and manually controlling the scoring direction, cues engineering can indirectly influence the scoring system's judgment logic by adjusting and optimizing the cues fed into the scoring system. In subjective scoring, this may manifest itself in the refinement of scoring criteria and adjustments to sensitivity to specific error types. This results in a more intelligent and comprehensive scoring process. Based on the designed cues, the scoring system can intelligently match relevant content within the subject's answer and assign a score based on the pre-set scoring criteria. This process may involve advanced technologies such as natural language processing and machine learning to achieve efficient and accurate scoring.

[0096] Therefore, the present invention constructs a prompt word construction unit for constructing prompt words based on the question to be graded, the keywords of the standard answer and the answer to be graded. In this embodiment, the prompt word syntax based on this logic is described as:

[0097] PA=You are a professional subjective question scoring system, please use +{question}+{key_points}+correct

[0098] +{student_response}

[0099] + Please rate the content. Please ensure that the score given is between 0 and 1. Only the rating will be output.

[0100] In this template, {question}, {key_points}, and {student_response} are placeholders, representing the question, key points of the standard answer, and the answer of the scoring object, respectively. In actual applications, these placeholders will be replaced by the actual question, key points, and student answer. In this way, the prompt word template guides the large language model to focus on the specific requirements of scoring and calculate the score Score3 based on these requirements. Large language models are usually universal, such as Qwen, DeepSeek, GPT, etc. This template helps ensure that the scoring system can evaluate the answers according to the predefined scoring criteria while maintaining the consistency and objectivity of the scoring process. The scoring process based on the above prompt word engineering template is as follows: the question, standard answer, and the answer of the scoring object are used as model inputs, and a prompt word template is given. Here, we can set it as needed, such as directly comparing the standard answer with the answer of the scoring object, or extracting keywords from each of them for matching. Rules can be formulated here. In addition, considering that S2 can score points even if the examinee directly copies the question, the present invention can also stipulate that directly copying the question will not score points. The final model calculates the score Score3 based on the input and prompt word template.

[0101] The large language model scoring unit is used to score the answers to be scored based on the prompt words using the large language model to obtain a subjective question score based on the large language model.

[0102] The weighted scoring module is used to weight the key point matching score, semantic similarity comparison score, and subjective question score based on the large language model to obtain the final subjective question score. To ensure that the score can fully reflect all aspects of the candidate's answer, the three scores are weighted in the weighted scoring module. The total score calculation formula is:

[0103] Score=w1×Score1+w2×Score2+w3×Score3

[0104] Among them, w1, w2 and w3 are the weights of each score and satisfy:

[0105] w1+w2+w3=1

[0106] The Key Point Matching Score (Score 1) ensures accurate capture of key information points in the answer; the Semantic Similarity Comparison Score (Score 2) provides an in-depth assessment of the semantic quality and depth of the answer; and the Cue Word Engineering Score (Score 3), based on a large language model, provides an in-depth analysis of the answer's innovation and logic. This multi-dimensional scoring approach not only improves scoring efficiency but also significantly enhances objectivity and fairness.

[0107] This invention effectively overcomes the problems of traditional manual scoring of subjective questions, such as low efficiency, high subjectivity, and difficulty achieving fair and unified scoring standards. By combining three advanced scoring technologies, we ensure that the scoring results are more comprehensive and accurate when used together, providing a new and efficient solution for scoring subjective questions. This approach not only improves the level of automation in scoring, but also ensures objectivity and fairness, making the results more reliable and bringing innovative technological breakthroughs to the field of educational assessment.

[0108] Example 2

[0109] like Figure 3 As shown, the present invention also provides a subjective question scoring method based on large language model combined weighting, applying the system described in Example 1, including:

[0110] Extract keywords from the standard answers to subjective questions, identify the answers to be scored based on the keywords, and obtain key point matching scores;

[0111] Score the semantic similarity between the answer to be scored and the standard answer to obtain a semantic similarity comparison score;

[0112] Based on the large language model and the fusion of prompt word engineering technology, the standard answers are used to score the answers to be scored, and the subjective question scores based on the large language model are obtained;

[0113] The key point matching score, semantic similarity comparison score, and subjective question score based on the large language model are weighted to obtain the final subjective question score result.

[0114] In a further embodiment, the method for obtaining a key point matching score includes:

[0115] Extract keywords from standard answers to obtain key information sequences;

[0116] Based on the importance of the key point matching method, a key point matching method weight is obtained;

[0117] Use regular matching technology to automatically match and identify whether the answer to be scored contains keywords in the key information sequence to obtain regular matching results;

[0118] Based on the regular matching results and the key point matching method weights, a key point matching score is obtained.

[0119] A further embodiment is that the method for obtaining the semantic similarity comparison score includes:

[0120] Based on the word vector model, the knowledge points of the standard answer and the knowledge points of the answer to be scored are split and converted into knowledge point vectors;

[0121] Calculate the semantic similarity between the knowledge point vector of the standard answer and the knowledge point vector of the answer to be scored;

[0122] The semantic similarity is compared with the preset similarity threshold to determine whether to score the knowledge point vector to be scored, and obtain the semantic similarity comparison score.

[0123] A further embodiment is that the method for obtaining a subjective question score based on a large language model includes:

[0124] Construct prompt words based on the question to be graded, the keywords of the standard answer, and the answer to be graded;

[0125] Based on the prompt words, the answers to be scored are scored using a large language model to obtain the subjective question scores based on the large language model.

[0126] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A subjective question scoring system based on a large language model combined weighting, characterized by: include: A key point matching scoring module is used to extract keywords from the standard answers to subjective questions and identify the answers to be scored based on the keywords to obtain a key point matching score; A similarity comparison and scoring module is used to score the semantic similarity between the answer to be scored and the standard answer to obtain a semantic similarity comparison score; A language model scoring module is used to score the answers to be scored using the standard answers based on the large language model and to obtain a score for the subjective question based on the large language model; The weighted scoring module is used to weight the key point matching score, the semantic similarity comparison score and the subjective question score based on the large language model to obtain the final subjective question scoring result.

2. The system according to claim 1, wherein: The key point matching scoring module includes: A keyword extraction unit, configured to extract keywords from the standard answer to obtain a key information sequence; A weight setting unit, configured to obtain a weight of the key point matching method based on the importance of the key point matching method; A regular matching recognition unit, configured to automatically match and identify whether the answer to be scored contains the keywords in the key information sequence using regular matching technology to obtain a regular matching result; The key point matching scoring unit is configured to obtain the key point matching score based on the regular matching result and the key point matching method weight.

3. The system according to claim 1, wherein: The similarity comparison and scoring module includes: A fragmentation retrieval unit is used to split the knowledge points of the standard answer and the knowledge points of the answer to be scored respectively and convert them into knowledge point vectors based on a word vector model; A similarity calculation unit is used to calculate the semantic similarity between the standard answer knowledge point vector and the knowledge point vector of the answer to be scored; The similarity comparison unit is used to compare the semantic similarity with a preset similarity threshold, determine whether to perform scoring processing on the knowledge point vector to be scored, and obtain the semantic similarity comparison score.

4. The system according to claim 1, wherein: The language model scoring module includes: A prompt word construction unit is used to construct prompt words based on the question of the question to be graded, the keywords of the standard answer and the answer to be graded; The large language model scoring unit is used to score the answers to be scored based on the prompt words using the large language model to obtain a subjective question score based on the large language model.

5. A subjective question scoring method based on combined weighting of a large language model, applying the system according to any one of claims 1 to 4, characterized in that: include: Extract keywords from the standard answers to the subjective questions, and identify the answers to be scored based on the keywords to obtain a key point matching score; Scoring the semantic similarity between the answer to be scored and the standard answer to obtain a semantic similarity comparison score; Based on the large language model fusion prompt word engineering technology, the standard answer is used to score the answer to be scored, and the subjective question score based on the large language model is obtained; The key point matching score, the semantic similarity comparison score, and the subjective question score based on the large language model are weighted to obtain a final subjective question score result.

6. The method according to claim 5, characterized in that The method for obtaining the key point matching score includes: Extract keywords from the standard answers to obtain key information sequences; Based on the importance of the key point matching method, a key point matching method weight is obtained; Automatically matching and identifying whether the answer to be scored contains the keywords in the key information sequence using regular matching technology to obtain a regular matching result; The key point matching score is obtained based on the regular matching result and the key point matching method weight.

7. The method according to claim 5, characterized in that Methods for obtaining semantic similarity comparison scores include: Based on the word vector model, the knowledge points of the standard answer and the knowledge points of the answer to be scored are split and converted into knowledge point vectors respectively; Calculate the semantic similarity between the knowledge point vector of the standard answer and the knowledge point vector of the answer to be scored; The semantic similarity is compared with a preset similarity threshold to determine whether to perform scoring processing on the knowledge point vector to be scored, thereby obtaining the semantic similarity comparison score.

8. The method according to claim 5, characterized in that Methods for obtaining subjective question scores based on large language models include: Construct prompt words based on the question to be graded, the keywords of the standard answer, and the answer to be graded; Based on the prompt words, the answers to be scored are scored using a large language model to obtain the subjective question scores based on the large language model.

Images