Proposition examination question rating method and system based on big data

Through big data technology and knowledge graph model, the problems of low efficiency and strong subjectivity of manual ratings are solved, the automated rating of test questions is realized, the rating accuracy and efficiency are improved, and it is suitable for different professional fields.

CN118885584AInactive Publication Date: 2024-11-01NETTALENT TECHNOLOGY (GUANGZHOU) GROUP CO LTD

Patent Information

Application Number
CN202410893158.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-01
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the prior art, question setting experts have problems of inefficiency and strong subjectivity in manual rating of test questions, and test questions rely on similarity in a certain dimension and are not very accurate.

Method used

Using a big data-based method, a database is constructed by collecting historical test questions and answer data, a test question scoring model is constructed based on the knowledge graph, and a reference answer score, difficulty score and test point depth score are calculated to achieve automated rating of test questions.

Benefits of technology

It improves the accuracy and efficiency of the rating of the test questions, reduces the workload of the question setting experts, improves the quality of the question bank and the accuracy of talent assessment, and can adapt to the test questions in different professional fields.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118885584A_ABST
    Figure CN118885584A_ABST
Patent Text Reader

Abstract

According to the proposition examination question rating method and system based on the big data provided by the embodiment of the invention, the examination question rating is obtained by obtaining the examination question content, utilizing the big data retrieval analysis and constructing the examination question rating model based on the knowledge graph to calculate the examination question scores of three dimensions, so that automatic rating of the proposition examination questions is realized. According to the method, various examination questions can be accurately identified and rated, the rating accuracy is high, the question proposition efficiency is remarkably improved, and the generalization ability is good.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of talent assessment technology, and in particular to a method and system for rating test questions using big data technology. Background Art

[0002] Currently in the field of talent assessment, test papers are still an important means to understand the degree of professional skills mastered by the assessed personnel. In the existing technology, after the question setting experts complete the question setting work of new test questions, the new test questions can only be rated manually. Due to the difference in knowledge mastery between the question setting experts and the actual evaluators, this method has problems such as low efficiency and strong subjectivity. With the development of big data technology, people have begun to try to use data analysis to improve the efficiency and objectivity of test question rating. In the existing technology, the rating of test questions mostly relies on the similarity of a certain dimension for scoring, and the accuracy is not high. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a test question grading method based on big data, which aims to solve the problems of low efficiency and high subjectivity of manual test question grading and improve the accuracy of test question grading.

[0004] A method for grading test questions based on big data includes the following steps: Collect historical test data and answer data as basic big data to form a database; Build a test scoring model based on the knowledge graph; Get the newly uploaded test questions to be graded; Calculate the reference answer score, reference difficulty score, and test point depth score of the test question to be rated according to the test question scoring model and database; Calculate the rating of the test question to be rated according to the reference answer rating, reference difficulty rating, and test point depth rating; Output the rating of the test question to be rated.

[0005] A test question rating system based on big data includes: Database module: used to collect historical test data and answer data as basic big data to form a database; Test question scoring model: used to build a test question scoring model based on the knowledge graph; Data input module: used to obtain the uploaded new test questions to be graded; Test question scoring module; used to calculate the reference answer score, reference difficulty score, and test point depth score of the test question to be rated according to the test question scoring model and database; Test question rating module: used to calculate the rating of the test question to be rated based on the reference answer rating, the difficulty rating, and the test point depth rating; Output module: used to output the rating of the test questions to be rated.

[0006] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned large model-based test paper marking method when executing the computer program.

[0007] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned large-model-based test paper marking method.

[0008] The embodiments of the present invention provide a method, system, computer device and storage medium for rating test questions based on big data. The method obtains the content of the test questions, uses big data retrieval and analysis, and calculates the test scores in three dimensions, thereby obtaining the rating of the new test questions and realizing automatic rating of the test questions.

[0009] The beneficial effects of this application include: High accuracy: Through knowledge graphs and big data, grading is performed based on the three dimensions of test questions. It can accurately identify and score various answer sheet types and question types, and the grading accuracy is high.

[0010] High efficiency: Automating the question rating process significantly improves rating efficiency, reduces the workload of question experts, improves the quality of the question bank, and improves the accuracy of talent assessment; the question rating system can make the test questions recommended to the candidates more adaptable.

[0011] Strong functionality: It can adapt to test questions in different professional fields and has good generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 It is a flow chart of an implementation of a method for grading test questions based on big data provided by an embodiment of the present invention; Figure 2 It is an entity-relationship ER diagram of a test question rating database provided by an embodiment of the present invention; Figure 3 It is a relationship diagram of a knowledge graph for test question rating provided by an embodiment of the present invention; Figure 4It is a framework diagram of a big data-based test question grading system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0015] See also Figure 1 , Figure 1 A flowchart of an implementation method of a test question rating method based on big data provided by an embodiment of the present invention is shown, and the method is described in detail as follows: S1: Collect historical test data and answer data as basic big data to form a database.

[0016] Specifically, when building a database, it is necessary to first design its concept, draw an Entity-Relationship (ER) diagram, then design the logical structure based on the ER diagram, and finally build the database. The ER diagram can help us understand the data structure and the relationship between them, and provide an intuitive reference for creating a database model. The main entities of the database include evaluators, question experts, history test questions, administrators, professional fields, such as Figure 2 The figure shows an entity-relationship ER diagram of the test question rating database. The database uses the mainstream MySQL database to facilitate access to various systems of the test center.

[0017] Specifically, the history test question data includes: professional field, test text, and knowledge points; the answer data includes: answer time, answer score, and standard answer.

[0018] S2: Build a test scoring model based on the knowledge graph.

[0019] Specifically, the knowledge graph is composed of many triples S={h, r, t}, where h and t represent the head node and tail node respectively, and r represents the relationship between the head node and the tail node. By building this relationship, it is easy to manage and utilize massive information. Through the knowledge graph, the potential connections between different entities can be discovered, providing additional information for test question rating. This application uses the existing knowledge graph for entity embedding. The knowledge graph contains related entities such as professional fields, positions, knowledge points, and historical test questions, such as Figure 3 The figure shows a relationship diagram of the knowledge graph used for exam question rating.

[0020] By embedding the test questions to be rated into the knowledge graph, the correlation and potential connection between the test questions to be rated and the historical test questions can be fully explored. According to the entities of the test questions to be rated in the knowledge graph, the entity-relationship projection matrix of the test questions to be rated will be constructed. Each entity-relationship is represented by two vectors, one vector represents its own embedding, and the other vector is used to construct the projection matrix. Each "entity-relationship" uses a different projection matrix to project the head and tail entities separately to avoid ghosting. The projection matrix is ​​represented as follows: ; ; in and are the projection matrices obtained by projecting the head and tail entities into different relational spaces, and are the projection vectors obtained by projecting into the relational space, is the projection vector of the relationship between each entity, n is the dimension of the head and tail entities, m is the dimension of the relationship, is the identity matrix used to initialize the projection matrix.

[0021] Use the projection matrix to perform the projection operation and define the projection vector as follows: ; ; Its scoring function is: ; The scoring function is used to evaluate the credibility of triples (entity-relationship-entity) in the knowledge graph, that is, to predict whether a triple is true. The purpose of the scoring function is to provide feedback to the training model so that the model can learn the appropriate representation of entities and relations in the triples.

[0022] After embedding all the exercises into the low-dimensional space, we get the vectors of all the historical test questions and calculate the cosine similarity with the test questions to be rated: ; In the mid-term, d is the dimension of the test vector, is the vector of test questions to be graded, is a history test question vector. The greater the similarity, the smaller the distance between the test question to be rated and the history test question. Sort by similarity and take the first ck (ck is an integer greater than or equal to 2) history test questions as the reference history test question set. Preferably, ck=5.

[0023] S3: Obtain the newly uploaded test questions to be graded.

[0024] Specifically, the question setting experts upload the content of the new questions to be graded by adding or importing them; You can add or import by directly entering the test questions to be graded, or by uploading PDF, word documents, photos and other files that meet the format requirements.

[0025] S4: Calculate the reference answer score, reference difficulty score, and test point depth score of the test questions to be rated based on the test question scoring model and database.

[0026] Specifically, the reference answer score F1 = average answer time of the reference history test set / average score, which reflects the time dimension score of the evaluator to obtain the unit score through the test questions to be rated; Reference difficulty score F2 = average score / average score of the reference history test set, which reflects the score given by the assessor on the difficulty dimension of the test questions to be rated; The test point depth score F3 = the number of knowledge points in the test question to be rated * the average number of knowledge points in the reference history test question set / the average score, which reflects the knowledge point coverage score of the test question to be rated.

[0027] S5: Calculate the rating of the test question to be rated based on the reference answer score, reference difficulty score, and test point depth score.

[0028] Specifically, the rating of the test question to be rated = a*F1+ b*F2+ c*F3, Among them, a, b, and c are the reference answer rating coefficient, reference difficulty rating coefficient, and test point depth rating coefficient respectively.

[0029] Specifically, an upper threshold and a lower threshold are set for F1, F2, and F3 respectively. When it is detected that a score is lower than a lower threshold or higher than an upper threshold, a reminder message is issued to remind the question experts that the test question to be rated is too difficult, too simple, or does not meet the requirements of the current professional field knowledge points, and other situations require checking the test questions.

[0030] Specifically, when it is necessary to adjust the test question rating according to the assessment focus, test pass rate and other indicators of the current stage, it can be done by adjusting the reference answer rating coefficient, the reference difficulty rating coefficient or the test point depth rating coefficient.

[0031] S6: Output the rating of the test question to be rated.

[0032] Specifically, the ratings of the test questions are output to the question setting experts, who verify the ratings of the test questions to be rated.

[0033] See also Figure 4 , Figure 4 A system for grading test questions based on big data provided by an embodiment of the present invention is shown, and the steps for implementing the above-mentioned method for grading test questions based on big data include: Database module: used to collect historical test data and answer data as basic big data to form a database; Test question scoring model module: used to build a test question scoring model based on the knowledge graph; Input module: used to obtain the uploaded new test questions to be graded; Scoring calculation module: used to calculate the reference answer score, reference difficulty score, and test point depth score of the test questions to be rated according to the test question scoring model and database; Rating module: used to calculate the rating of the test questions to be rated according to the reference answer rating, reference difficulty rating, and test point depth rating; Output module: used to output the rating of the test questions to be rated.

[0034] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for grading test questions based on big data when executing the computer program.

[0035] In one embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for grading test questions based on big data are implemented.

[0036] It can be understood that the computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal and telecommunication signal, etc.

[0037] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for grading test questions based on big data, characterized in that: include: Collect historical test data and answer data as basic big data to form a database; Build a test scoring model based on the knowledge graph; Get the newly uploaded test questions to be graded; Calculate the reference answer score, reference difficulty score, and test point depth score of the test question to be rated according to the test question scoring model and database; Calculate the rating of the test question to be rated according to the reference answer rating, reference difficulty rating, and test point depth rating; Output the rating of the test question to be rated.

2. The method according to claim 1, characterized in that: The collection of historical test data and answer data as basic big data to form a database specifically includes the following steps: Draw an entity-relationship ER diagram, design the logical structure based on the ER diagram, and then establish a database; the database uses a Mysql database; the historical test question data includes the professional field, test question text, and knowledge points; the answer data includes the answer time, answer score, and standard answer.

3. The method according to claim 1, characterized in that The test question scoring model is constructed based on the knowledge graph, wherein: Specifically, the knowledge graph consists of many triples S={h, r, t}, where h and t represent the head node entity and the tail node entity respectively, and r represents the relationship between the head node entity and the tail node entity; The entities include professional fields, positions, test questions and knowledge points; The relationships described include subordination and emphasis; Embed the test questions to be rated into the knowledge graph and construct an entity-relationship projection matrix of the test questions to be rated; The test questions to be rated are converted into a low-dimensional space by a projection matrix to obtain a vector of the test questions to be rated; The reference historical test question set is obtained by calculating the cosine similarity of the vectors of the test questions to be rated.

4. The method according to claim 3, characterized in that The step of calculating the reference answer score, reference difficulty score, and test point depth score of the test questions to be rated based on the test question scoring model and the database specifically includes the following steps: The reference answer score = average answer time / average score of the reference history test set; The reference difficulty score = the average score of the reference history test set / the average score; The test point depth score = the total knowledge points of the test questions to be graded * the average knowledge points of the reference history test question set / average score.

5. The method according to claim 4, characterized in that The step of calculating the rating of the test questions to be rated based on the reference answer rating, reference difficulty rating, and test point depth rating specifically includes the following steps: The rating of the test questions to be graded = a*reference answer score + b*reference difficulty score + c*test point depth score, Among them, a, b, and c are the reference answer rating coefficient, reference difficulty rating coefficient, and test point depth rating coefficient respectively.

6. A test question rating system based on big data, characterized in that: The steps for implementing the method for grading test questions based on big data as described in any one of claims 1 to 5 include: Database module: used to collect historical test data and answer data as basic big data to form a database; Test question scoring model module: used to build a test question scoring model based on the knowledge graph; Input module: used to obtain the uploaded new test questions to be graded; Scoring calculation module: used to calculate the reference answer score, reference difficulty score, and test point depth score of the test questions to be rated according to the test question scoring model and database; Rating module: used to calculate the rating of the test questions to be rated according to the reference answer rating, reference difficulty rating, and test point depth rating; Output module: used to output the rating of the test questions to be rated.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for grading test questions based on big data as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for grading test questions based on big data as described in any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Knowledge point recommendation method and device, terminal and computer readable storage medium

    CN113590956A

  • Examination interface generation method and device and readable storage medium

    CN116431800A

  • Construction method, system and equipment for digital skill authentication question bank

    CN116680249A

  • Personalized intelligent question answering method and device based on knowledge tracking and knowledge graph

    CN118193701A

  • Method and system for a document search system using search criteria comprised of ratings prepared by experts

    US20030014428A1

Cited By

  • Proposition examination question rating method and system based on big data deep mining

    CN120632083A