A test question difficulty labeling method and device based on a test paper structure and a storage medium

Abstract

The application discloses a test question difficulty marking method and device based on a test paper structure and a storage medium, the test question difficulty marking method comprises the following steps: obtaining a test paper set with the same test paper attribute information in a test paper library; performing test paper structure analysis and test question information analysis on the test papers in the test paper set; constructing a test question difficulty transmission structure diagram of all test questions in the test paper set according to a test question difficulty relationship corresponding to each type of test paper structure preset by the system; and marking the difficulty of each test question based on the test question difficulty transmission diagram. The test question difficulty transmission structure diagram of all test questions in the test paper set is constructed, the system automatically assigns difficulty values to each test question according to the test question difficulty transmission structure diagram, the automatic and batch marking of the test question difficulty values is realized, the mode of manual test question marking is simpler and faster, the test question difficulty values are obtained through a relative relationship, and the marking basis of the test question difficulty values is more reasonable, accurate and reliable.

Description

TECHNICAL FIELD

[0001] The present application relates to the online education technical field, and in particular to a test question difficulty labeling method and device based on test paper structure and a storage medium. BACKGROUND

[0002] In order to provide online services such as photographing searching for questions, intelligent practice, and homework correction, an online education platform generally needs to be based on a powerful test question database. For test questions in the test question database, in order to provide users with a more comprehensive use experience, test question attribute labels such as subject, grade, region, and difficulty value of the test question are often labeled. Among them, the test question attribute labels such as subject, grade, and region can be automatically obtained from the test paper resources from which the test questions are obtained, while the difficulty value label of the test question generally needs to be labeled by a teacher one by one.

[0003] The above-mentioned labeling method by the teacher for the difficulty value label of the test question has the following problems:

[0004] On the one hand, for an exam question database with a capacity of hundreds of millions or even larger, the number and frequency of test question updates are large, which requires a large amount of manpower, financial resources, and time to label the difficulty value of the test question one by one, and the cost of manpower, time, and money is large.

[0005] On the other hand, since the difficulty value of the test question has subjective factors, different teachers have different evaluation standards for the same test question, resulting in deviation in the labeling of the difficulty value of the same test question. This characteristic makes it difficult to evaluate whether the difficulty value of a test question is reasonable.

[0006] In summary, the existing test question difficulty value labeling method not only needs to spend a large amount of manpower, time, and money, but also lacks labeling and review of subjective evaluation of the difficulty value of the test question, and it is impossible to determine whether the difficulty value of the test question is reasonable.

[0007] Therefore, the present application is proposed. SUMMARY

[0008] In order to solve the above technical problems, the present application provides a test question difficulty labeling method and device based on test paper structure and a storage medium, and the specific technical solutions are as follows:

[0009] The present application provides a test question difficulty labeling method based on test paper structure, which comprises:

[0010] Obtaining a test paper set with the same test paper attribute information in a test paper library;

[0011] Performing test paper structure analysis and test question information analysis on the test papers in the test paper set;

[0012] constructing a question difficulty transmission structure diagram of all questions in the question bank according to the question difficulty relations corresponding to the question bank structure of each type of question bank preset by the system;

[0013] annotating the question difficulty of each question based on the question difficulty transmission diagram.

[0014] As an optional embodiment of the present application, the step of constructing a question difficulty transmission structure diagram of all questions in the question bank according to the question difficulty relations corresponding to the question bank structure of each type of question bank preset by the system comprises:

[0015] In the same question bank, according to the question difficulty relations corresponding to the question bank structure of the question bank preset by the system, a question bank structure is formed using a directed acyclic graph according to difficulty from small to large;

[0016] The question bank structures of different question banks are connected through the same questions, and a question difficulty transmission structure diagram of all questions in the question bank is constructed.

[0017] As an optional embodiment of the present application, the step of annotating the question difficulty of each question based on the question difficulty transmission diagram comprises:

[0018] finding the longest sequence in the question difficulty transmission structure diagram;

[0019] setting the question difficulty value of the question at the head of the sequence as an initial value K0, setting the question difficulty value of the question at the tail of the sequence as Kt, and forming a uniform gradient between K0 and Kt according to the length of the longest sequence to assign the question difficulty values of the questions between the head and the tail of the longest sequence.

[0020] As an optional embodiment of the present application, the step of annotating the question difficulty of each question based on the question difficulty transmission diagram comprises:

[0021] For the questions in the sub-sequences of the non-longest sequences in the question difficulty transmission structure diagram, starting from the question connected to the longest sequence, the question difficulty values of the sub-sequences pointed to by the questions are sequentially increased according to the gradient, and the question difficulty values of the sub-sequences pointed by the questions are sequentially decreased.

[0022] As an optional embodiment of the present application, the step of annotating the question difficulty of each question based on the question difficulty transmission diagram comprises:

[0023] for the sub-diagrams containing strongly connected components existing in the question difficulty transmission structure diagram;

[0024] the strongly connected components in the sub-diagram are merged into one structure point, and the question difficulty values of the questions in the strongly connected components are equal.

[0025] As an optional embodiment of the present application, the question difficulty relations corresponding to the question bank structure of each type of question bank preset by the system are:

[0026] The difficulty value of each test question in the same type of test question in the same set of test papers increases with the increase of the serial number;

[0027] Or through the neural network model, the test papers of various types of test paper structures are trained and learned, and the corresponding test question difficulty relationship of each type of test paper structure is obtained.

[0028] As an optional embodiment of the present application, the test paper set with the same test paper attribute information in the test paper library comprises:

[0029] The test paper attribute information comprises a test paper name, a subject label, a grade label and a region label;

[0030] The test papers with similar test paper names, the same subject labels, grade labels and region labels are screened out in the test paper library by taking the test paper name, the subject label, the grade label and the region label as the screening conditions, and are stored as the test paper set;

[0031] Optionally, the similar test paper name refers to the successful matching of one or more keywords / words set in the test paper name.

[0032] As an optional embodiment of the present application, the test paper structure analysis on the test papers in the test paper set comprises:

[0033] The test question type and the number of test questions contained in each test question type in each set of test papers in the test paper set are acquired respectively;

[0034] The corresponding test paper structure is matched out in the system preset test paper structure library according to the test question type of the test paper and the number of test questions contained in each test question type;

[0035] The test question information analysis on the test papers in the test paper set comprises:

[0036] The test question ID, the subject information and the grade information of each test question in each set of test papers in the test paper set are acquired respectively.

[0037] The present application simultaneously provides a test question difficulty marking device based on test paper structure, comprising:

[0038] An acquisition module acquires a test paper set with the same test paper attribute information in a test paper library;

[0039] An analysis module analyzes the test paper structure and the test question information of the test papers in the test paper set;

[0040] A structure diagram construction module constructs a test question difficulty transmission structure diagram of all test questions in the test paper set according to the test question difficulty relationship corresponding to each type of test paper structure preset by the system;

[0041] And the test question difficulty marking module marks the difficulty of each test question based on the test question difficulty transmission graph.

[0042] The application also provides a storage medium storing a computer executable program, which, when executed, implements the test question difficulty marking method based on the test paper structure.

[0043] Compared with the prior art, the application has the following beneficial effects:

[0044] The test question difficulty marking method based on the test paper structure provided by the application realizes the automatic and batch marking of the test question difficulty value, is simpler and faster than the manual marking method, and is more reasonable and accurate in marking the test question difficulty value. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 The processing flowchart of the test question difficulty marking method based on the test paper structure of the application

[0046] Figure 2 The test question difficulty transmission graph in the test question difficulty marking method based on the test paper structure of the application Figure 1 ;

[0047] Figure 3 The test question difficulty transmission graph in the test question difficulty marking method based on the test paper structure of the application Figure 2 ;

[0048] Figure 4 The flowchart of the test question difficulty marking method based on the test paper structure of the application Figure 1 ;

[0049] Figure 5 The flowchart of the test question difficulty marking method based on the test paper structure of the application Figure 2 ;

[0050] Figure 6 The flowchart of the test question difficulty marking method based on the test paper structure of the application Figure 3 ;

[0051] Figure 7 The flowchart of the test question difficulty marking method based on the test paper structure of the application Figure 4 ;

[0052] Figure 8 The processing flowchart of the test question difficulty value confidence evaluation method of the application. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application.

[0054] Therefore, the following detailed description of the embodiments of the present application is not intended to limit the scope of the claimed application, but merely represents some embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0055] It should be noted that the embodiments in the present application and the features and technical solutions in the embodiments can be combined with each other without conflict.

[0056] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0057] In the description of the present application, it should be noted that the terms "upper", "lower", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the present application is usually placed, or the orientation or positional relationship commonly understood by those skilled in the art, and such terms are only for the convenience of describing the present application and simplifying the description, and are not intended to indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", and the like are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0058] Example 1

[0059] Referring to Figure 1 and Figure 4 The embodiment provides a test question difficulty marking method based on a test paper structure, and the method comprises the following steps:

[0060] In step S1, a test paper set with same test paper attribute information in a test paper library is acquired;

[0061] In step S2, test paper structure analysis and test question information analysis are performed on the test papers in the test paper set;

[0062] In step S3, a test question difficulty transmission structure diagram of all test questions in the test paper set is constructed according to a test question difficulty relationship corresponding to each type of test paper structure preset by the system;

[0063] Step S4, based on the test question difficulty transfer graph, each test question is marked with test question difficulty.

[0064] The test question difficulty marking method based on the test paper structure provided in this embodiment realizes the automatic and batch marking of test question difficulty values, which is simpler and faster than the manual test question marking method. Since the test question difficulty transfer structure graph is used, the test question difficulty values are obtained through the relative relationship, and the marking basis of the test question difficulty values is more reasonable, and the test question difficulty values are more accurate and reliable.

[0065] Further, referring to Figure 5 The test question difficulty transfer graph of all test questions in the test paper set is constructed according to the test question difficulty relationship corresponding to each type of test paper structure preset by the system, which includes:

[0066] Step S301, in the same test paper, according to the test question difficulty relationship corresponding to the test paper structure of the test paper preset by the system, a one-way acyclic graph is used to form a test paper graph structure according to the difficulty from small to large;

[0067] Step S302, the test paper graph structures of different test papers are connected through the same test questions, and the test question difficulty transfer structure graph of all test questions in the test paper set is constructed.

[0068] Referring to Figure 2 The test questions 1, 2, 3, 4, and 5 in the figure belong to the same type of test questions in the same test paper, and form a test paper graph structure of 1→2→3→4→5 according to the difficulty relationship; while the test questions 6, 7, 3, 8, 9, 10, and 11 belong to the same type of test questions in another test paper, and form a test paper graph structure of 6→7→3→8→9→10→11 according to the difficulty relationship, and the two test papers contain the same test question 3, which realizes the connection of the two test paper graph structures. In this way, all test questions in the entire test paper set can be marked into a test question difficulty transfer structure graph as shown in Figure 2 The test questions 1, 2, 3, 4, and 5 in the figure belong to the same type of test questions in the same test paper, and form a test paper graph structure of 1→2→3→4→5 according to the difficulty relationship; while the test questions 6, 7, 3, 8, 9, 10, and 11 belong to the same type of test questions in another test paper, and form a test paper graph structure of 6→7→3→8→9→10→11 according to the difficulty relationship, and the two test papers contain the same test question 3, which realizes the connection of the two test paper graph structures. In this way, all test questions in the entire test paper set can be marked into a test question difficulty transfer structure graph as shown in

[0069] As an optional implementation manner of this embodiment, referring to Figure 5 The test question difficulty marking based on the test question difficulty transfer graph includes:

[0070] Step S410, finding the longest sequence in the test question difficulty transfer structure diagram;

[0071] Step S411: Set the difficulty value of the test questions at the head of the sequence to the initial value K0, and the difficulty value of the test questions at the tail of the sequence to Kt. According to the length of the longest sequence, form a uniform gradient between K0 and Kt to assign the difficulty values ​​of the test questions between the head and tail of the longest sequence.

[0072] In the above-mentioned test question difficulty value algorithm of this embodiment, the longest sequence in the test question difficulty transfer structure diagram is first used as the basis, the minimum test question difficulty value K0 is assigned to the test question at the head of the sequence, and the maximum test question difficulty value Kt is assigned to the test question at the end of the sequence. The test questions between the head and tail of the sequence are evenly assigned, and then any test question in the middle is used to assign values ​​to other test questions in the sequence connected to it.

[0073] Optionally, the difficulty value of the test questions in this embodiment is between 0 and 1.0. Therefore, the initial value K0 of this embodiment is 0, Kt=1.0, and the difficulty values ​​of the test questions between the head and the tail of the sequence are all between 0 and 1.0.

[0074] For completely independent sequences in the question difficulty transmission structure diagram, independent calculations are performed using the method of assigning the minimum question difficulty value to the head question, the maximum question difficulty value to the tail question, and uniform distribution of difficulty values ​​in the middle. Alternatively, the head and tail questions in the independent sequence are sent to manual annotation. The manual annotation is performed on the head and tail questions in the independent sequence with reference to the question difficulty values ​​automatically annotated by the system, and then the difficulty values ​​are returned to the system. The system automatically calculates and copies the question difficulty values ​​of other questions in the independent sequence based on the manually annotated question difficulty values ​​of the head and tail questions.

[0075] Further, see Figure 5 As shown, in this embodiment, marking the difficulty of each question based on the question difficulty transfer graph includes:

[0076] Step S420, testing questions in a subsequence that is not the longest sequence in the test question difficulty transfer structure diagram;

[0077] Step S421, starting from the test question connected to the longest sequence, the difficulty value of the test question in the subsequence pointed to by the test question increases in sequence according to the gradient, and the difficulty value of the test question in the subsequence pointed to by the test question decreases in sequence.

[0078] The test question difficulty marking method based on the test paper structure of this embodiment can automatically mark the difficulty values ​​of all test questions with mutual correlation in batches through the test question difficulty transfer structure diagram, which is simpler and more intelligent.

[0079] In addition, for the special cases existing in the process of constructing the test question difficulty transmission structure diagram, the test question difficulty labeling of each test question based on the test question difficulty transmission diagram described in this embodiment further includes:

[0080] For the subgraph containing a strongly connected component in the test question difficulty transmission structure diagram;

[0081] The strongly connected component in the subgraph is merged into one structure point, and the test question difficulty values of each test question in the strongly connected component are equal.

[0082] Specifically, in a directed graph G, if there is a directed path from u to v between two vertices u and v, and there is also a directed path from v to u, then the two vertices are strongly connected. If every two vertices of a directed graph G are strongly connected, G is called a strongly connected graph. The maximal strongly connected subgraph of a directed acyclic graph is called a strongly connected component. As shown in the test question difficulty transmission diagram, Figure 3 8→9→12→8 forms a strongly connected component S1, and the formation of the strongly connected component will cause the difficulty transmission to fall into an infinite loop. We believe that the test question difficulty values in the strongly connected component are equal, and can be reduced to one point.

[0083] The test question difficulty relationship corresponding to each type of test paper structure preset in the system in the test question difficulty labeling method based on test paper structure of this embodiment is:

[0084] The test question difficulty values of each test question in the same type of test question in the same set of test papers increase with the increase of the serial number;

[0085] Or train and learn the test papers of various types of test paper structures through a neural network model to obtain the test question difficulty relationship corresponding to each type of test paper structure.

[0086] The test question difficulty relationship corresponding to each type of test paper structure in this embodiment is preset in the system after being determined through the above two ways. Taking the past years' middle school examination papers and high school examination papers as examples, each set of test papers includes several types of questions, each type of question includes a certain number of test questions, and the difficulty relationship of test questions in each type of question is fixed, that is, the structure of the test paper is generally fixed. According to this feature, the structure type of the test paper and the test question difficulty relationship corresponding to each type of test paper structure can be summarized.

[0087] As an optional implementation manner of this embodiment, the test question difficulty labeling method based on test paper structure of this embodiment, the test paper set with the same test paper attribute information in the test paper library includes:

[0088] The test paper attribute information includes test paper name, subject label, grade label and region label;

[0089] The test paper name, the subject label, the grade label, and the region label are taken as the screening conditions to screen the test papers with similar test paper names and the same subject label, grade label, and region label from the test paper library, and the test papers are stored as a test paper set.

[0090] Optionally, the similar test paper name refers to that one or more keywords / words set in the test paper name are matched successfully.

[0091] Referring to FIG. 1, the test paper structure-based test question difficulty labeling method of the embodiment includes the following steps. Figure 6 Referring to FIG. 1, the test paper structure-based test question difficulty labeling method of the embodiment includes the following steps.

[0092] In step S211, the test question type and the number of test questions included in each test question type are obtained for each set of test papers in the test paper set.

[0093] In step S212, the corresponding test paper structure is matched from the system-pre-set test paper structure library according to the test question type of the test paper and the number of test questions included in each test question type.

[0094] Referring to FIG. 1, the test question information analysis for the test papers in the test paper set includes the following steps. Figure 7

[0095] In step S221, the test question ID, subject information, and grade information of each test question are obtained for each set of test papers in the test paper set.

[0096] Taking a mathematics test question as an example, a mathematics test paper has a special test paper structure and a test question difficulty value relationship, the types of the test questions in the mathematics test paper generally include a selection question, a fill-in-the-blank question, and a solution question, and the difficulty value of the test questions in each type increases with the increase of the test question serial number. Based on the above test paper structure of the mathematics test paper and the test question difficulty value relationship corresponding to the test paper structure, the difficulty value of the mathematics test question can be automatically batch-labeled.

[0097] The test question difficulty labeling method based on the mathematics test paper structure of the embodiment is as follows.

[0098] In step S1', the test paper structure information of the mathematics test paper and the relationship between the test question difficulty value and the test paper structure are pre-stored in the test paper library.

[0099] In step S2', the screening condition of the test paper is set in the test paper library, for example, a Beijing region-2018 annual-high school second grade-mathematics-spring semester midterm test paper is set, and all the mathematics test papers of the midterm test of the Beijing region 2018 high school second grade spring semester in the test paper library will be screened out to form a mathematics test paper set.

[0100] ​Step S3', in the same set of mathematical papers, according to the system preset difficulty relationship of the paper structure corresponding to the paper, the paper structure is formed by using the one-way acyclic graph according to the difficulty from small to large; the paper structure of different papers is connected through the same question, and the question difficulty transmission structure diagram of all questions in the paper set is constructed.

[0101] Step S4', in the process of constructing the question difficulty transmission structure diagram, if there is a subgraph containing a strongly connected component; the strongly connected component in the subgraph is merged into a structure point, and the question difficulty values of each question in the strongly connected component are equal. The formation of the strongly connected component will make the difficulty transmission fall into an infinite loop. We believe that the question difficulty values in the strongly connected component are equal, which can be reduced to a point to avoid falling into an infinite loop in the process of constructing the question difficulty transmission structure diagram.

[0102] Step S5', find the longest sequence in the question difficulty transmission structure diagram; the question difficulty value of the head question of the sequence is set to 0, and the question difficulty value of the tail question of the sequence is set to 1.0; according to the length of the longest sequence, a uniform gradient is formed between 0-1.0, and the question difficulty values of the questions between the head and tail of the longest sequence are assigned.

[0103] Step S6', for the questions on the subsequence of the non-longest sequence in the question difficulty transmission structure diagram, starting from the question connected with the longest sequence, the question difficulty values of the subsequence pointed by the question are sequentially increased according to the gradient, and the question difficulty values of the subsequence pointed by the question are sequentially decreased, thereby obtaining the question difficulty values of the questions on each subsequence.

[0104] The embodiment also provides a question difficulty labeling device based on a paper structure, which comprises:

[0105] An acquisition module acquires a paper set with the same paper attribute information in a paper library;

[0106] An analysis module analyzes the paper structure and question information of the papers in the paper set;

[0107] A structure diagram construction module constructs a question difficulty transmission structure diagram of all questions in the paper set according to the system preset question difficulty relationship corresponding to each type of paper structure;

[0108] A question difficulty labeling module labels the question difficulty of each question based on the question difficulty transmission diagram.

[0109] Further, the structure diagram construction module constructs a question difficulty transmission diagram of all questions in the paper set according to the system preset question difficulty relationship corresponding to each type of paper structure.

[0110] In the same test paper, according to the difficulty relationship of the test paper structure corresponding to the system preset for the test paper, a one-way acyclic graph is used to form a test paper graph structure according to difficulty from small to large;

[0111] The test paper graph structures of different test papers are connected through the same test questions, and a test question difficulty transmission structure graph of all test questions in the test paper set is constructed.

[0112] Referring to Figure 2 As shown in the figure, test questions 1, 2, 3, 4, and 5 belong to the same type of test questions in the same test paper, and form a test paper graph structure of 1→2→3→4→5 according to the difficulty relationship; while test questions 6, 7, 3, 8, 9, 10, and 11 belong to the same type of test questions in another test paper, and form a test paper structure graph of 6→7→3→8→9→10→11 according to the difficulty relationship, and the two test papers contain the same test question 3, realizing the connection of the test paper graph structures of the two test papers. In this way, all test questions in the entire test paper set can be marked as a test question difficulty transmission structure graph as shown in Figure 2 As shown in the figure, the test question difficulty transmission structure graph may exist in the form of a mutual interlaced connection of all test questions in the entire test paper set; or may exist in the form of multiple test question difficulty transmission structure graphs, which are independent of each other; or may exist in the form of a test paper alone as a test question difficulty transmission structure graph, and has no correlation with others. No matter what kind of test question difficulty transmission structure graph is finally constructed in the test paper set, as long as there is a difficulty relative relationship between the test questions, the test question difficulty value of each test question can be calculated.

[0113] As an optional implementation of the embodiment, the test question difficulty marking module marks the test questions based on the test question difficulty transmission graph, including:

[0114] Finding the longest sequence in the test question difficulty transmission structure graph;

[0115] Setting the test question difficulty value of the head test question of the sequence as an initial value K0, and setting the test question difficulty value of the tail test question of the sequence as Kt, forming a uniform gradient between K0 and Kt according to the length of the longest sequence, and assigning the test question difficulty values of the test questions between the head and the tail of the longest sequence.

[0116] In the above test question difficulty value algorithm of the embodiment, the minimum difficulty value K0 of the test question is assigned to the head test question of the sequence based on the longest sequence in the test question difficulty transmission structure graph, the maximum difficulty value Kt of the test question is assigned to the tail test question of the sequence, and the test questions between the head and the tail of the sequence are uniformly assigned, and then any one of the intermediate test questions is taken as a starting point to assign values to other test questions in the sequence connected thereto.

[0117] Optionally, the test question difficulty value of the embodiment is between 0 and 1.0, so the initial value K0 of the embodiment is 0, Kt=1.0, and the test question difficulty value of the test questions between the head and the tail of the sequence is between 0 and 1.0.

[0118] For the completely independent sequence in the test question difficulty transmission structure diagram, the minimum test question difficulty value of the head test question, the maximum difficulty value of the tail test question, and the uniform assignment method in the middle are independently operated. Alternatively, the head test question and the tail test question in the independent sequence are pushed to manual marking, and the test question difficulty value marked by the system is returned to the system after the test question difficulty value of the head test question and the tail test question in the independent sequence is manually marked. The system automatically calculates and copies the test question difficulty value of the other test questions in the independent sequence according to the independently marked test question difficulty value of the head test question and the tail test question.

[0119] Further, the test question difficulty marking module based on the test question difficulty transmission diagram for marking the test question difficulty of each test question further comprises:

[0120] For the test questions on the subsequence of the non-longest sequence in the test question difficulty transmission structure diagram, starting from the test question connected to the longest sequence, the test question difficulty value of the subsequence pointed by the test question is sequentially increased according to the gradient, and the test question difficulty value of the subsequence pointed by the test question is sequentially decreased.

[0121] The test question difficulty marking device based on the test paper structure of the embodiment can automatically and batch mark the test question difficulty values of all test questions having a mutual correlation relationship through the test question difficulty transmission structure diagram, which is more simple and intelligent.

[0122] In addition, for the special cases existing in the process of constructing the test question difficulty transmission structure diagram, the test question difficulty marking module based on the test question difficulty transmission diagram for marking the test question difficulty of each test question further comprises:

[0123] For the subgraph containing a strongly connected component existing in the test question difficulty transmission structure diagram;

[0124] The strongly connected component in the subgraph is merged into one structure point, and the test question difficulty values of the test questions in the strongly connected component are equal.

[0125] Specifically, in a directed graph G, if there is a directed path from u to v between two vertices u and v, and there is also a directed path from v to u, then the two vertices are strongly connected. If every two vertices of a directed graph G are strongly connected, G is called a strongly connected graph. The maximum strongly connected subgraph of a directed non-strongly connected graph is called a strongly connected component. For example, Figure 3In the shown test question difficulty transfer graph, 8→9→12→8 constitutes a strongly connected component S1, and the formation of the strongly connected component causes the difficulty transfer to fall into an infinite loop. We believe that the question difficulty values in the strongly connected component are equal, and can be reduced to a point.

[0126] In the test question difficulty marking device based on the test paper structure of this embodiment, the test question difficulty relationship corresponding to each type of test paper structure preset by the system is:

[0127] The test question difficulty values of each test question in the same type of test question in the same set of test papers increase with the increase of the serial number;

[0128] Or the neural network model is trained and learned for test papers of various types of test paper structures, and the test question difficulty relationship corresponding to each type of test paper structure is obtained.

[0129] The test question difficulty relationship corresponding to each type of test paper structure in this embodiment is preset in the system after being determined by the above two ways. Taking the past years' middle school examination papers and high school examination papers as examples, each set of test papers includes several types of questions, each type of question includes a certain number of test questions, and the difficulty relationship of the test questions in each type of question is fixed, that is, the structure of the test paper is generally fixed. According to this feature, the structure type of the test paper and the test question difficulty relationship corresponding to each type of test paper structure can be summarized.

[0130] As an optional implementation manner of this embodiment, the test question difficulty marking device based on the test paper structure of this embodiment, the obtaining module obtains a test paper set with the same test paper attribute information in the test paper library includes:

[0131] The test paper attribute information includes test paper name, subject label, grade label and region label;

[0132] The test papers with similar test paper names, the same subject labels, grade labels and region labels are screened out in the test paper library as the screening conditions, and stored as the test paper set.

[0133] Optionally, the similar test paper name means that one or more keywords / words set in the test paper name are successfully matched.

[0134] The test paper structure analysis of the test paper set by the analysis module in the test question difficulty marking device based on the test paper structure of this embodiment includes:

[0135] The test question types and the number of test questions included in each test question type of each set of test papers in the test paper set are obtained;

[0136] The corresponding test paper structure is matched in the test paper structure library preset by the system according to the test question types of the test papers and the number of test questions included in each test question type.

[0137] The test question information analysis for the test papers in the test paper set comprises:

[0138] The test question ID, subject information and grade information of each test question are obtained for each test paper in the test paper set.

[0139] The embodiment further provides a storage medium which stores a computer executable program, and the computer executable program is executed to realize the test question difficulty labeling method based on a test paper structure.

[0140] The storage medium in the embodiment can include a data signal which is propagated in a baseband or as a part of a carrier wave, and the data signal bears readable program codes. The propagated data signal can adopt various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit a program for use by or in combination with an instruction execution system, device or apparatus. The program codes included in the readable storage medium can be transmitted by using any suitable medium, including but not limited to wireless, wired, optical cable, RF and the like, or any suitable combination of the above.

[0141] The embodiment further provides an electronic device which includes a processor and a memory, and the memory is used to store a computer executable program, and when the computer program is executed by the processor, the processor executes the test question difficulty labeling method based on a test paper structure.

[0142] The electronic device is in the form of a general computing device. The processor can be one or multiple and work cooperatively. The present application also does not exclude distributed processing, that is, the processor can be dispersed in different entity devices. The electronic device of the present application is not limited to a single entity, and can also be a sum of multiple entity devices.

[0143] The memory stores a computer executable program, which is usually machine readable code. The computer readable program can be executed by the processor to enable the electronic device to execute the method of the present application or at least part of the steps in the method.

[0144] The memory includes a volatile memory, such as a random access memory (RAM) and / or a cache memory unit, and can also be a non-volatile memory, such as a read only memory (ROM).

[0145] It should be understood that the electronic device of the present application can also include elements or components not shown in the above examples. For example, some electronic devices also include display units such as display screens, and some electronic devices also include human-computer interaction elements such as buttons and keyboards. As long as the electronic device can execute the computer-readable program in the memory to implement the method or at least part of the steps of the method of the present application, it can be considered as an electronic device covered by the present application.

[0146] From the above description of the embodiments, those skilled in the art will readily understand that the present application can be implemented by hardware capable of executing a specific computer program, such as the system of the present application, and the electronic processing unit, server, client, mobile phone, control unit, processor, etc. contained in the system. The present application can also be implemented by computer software that executes the method of the present application, such as control software executed by microprocessors, electronic control units, clients, server sides, etc. It should be noted that the computer software that executes the method of the present application is not limited to being executed by one or a specific hardware entity, but can also be implemented in a distributed manner by non-specific hardware. For computer software, the software product can be stored in a computer-readable storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), or can be distributed on a network, as long as it can make the electronic device execute the method according to the present application.

[0147] Example 2

[0148] Referring to Figure 8 The present embodiment provides a confidence evaluation method for test question difficulty value, which is used to evaluate the reliability of the test question difficulty value obtained by the test question difficulty labeling method of embodiment one, and includes the following steps:

[0149] Extract a certain number of test questions from the test question library, and form test question pairs two by two;

[0150] Obtain the test question difficulty values of the test questions in the test question pair, compare the test question difficulty values, and aggregate to obtain system evaluation result data;

[0151] Artificially evaluate the difficulty size relationship of the test questions in the test question pair, and obtain artificial evaluation result data;

[0152] Take the artificial evaluation result data as the evaluation standard, compare the results with the system evaluation result data, statistically calculate the proportion of test question pairs that are the same, and obtain the confidence for evaluating the test question difficulty value information in the test question library.

[0153] Since the evaluation criteria of each teacher for the difficulty value of the test questions are different and have strong subjectivity, simply evaluating the difficulty value of the test questions one by one is not only time-consuming and laborious, but also has little significance, because more attention should be paid to whether the rules for evaluating the difficulty value of the test questions are reliable or whether the difficulty values of most test questions in the test question bank are reliable.

[0154] Based on the above technical problems, in order to more objectively and accurately evaluate the reliability of the difficulty values of the test questions in the test question bank, the embodiment provides a confidence evaluation method for the difficulty values of the test questions, which groups the test questions into test question pairs, and artificially evaluates the difficulty size relationship of the test question pairs. The artificial evaluation of the difficulty values of the test questions has subjectivity in the size of the scores, but the comparison of the difficulty relationship of the two test questions in the test question pair is relatively objective. In this way, the subjective evaluation of the difficulty value of the single test question is changed into the relatively objective evaluation of the comparison of the difficulty size relationship of the test question pair, and the artificial evaluation result data is obtained. At the same time, the system automatically compares the difficulty size relationship of the test questions in the test question pair, and the system evaluation result data is obtained. The artificial evaluation result data is used as the evaluation standard, and the system evaluation result data is compared, the proportion of the test question pairs that are the same is calculated and compared, and the confidence for evaluating the difficulty value information of the test questions in the test question bank is obtained.

[0155] The confidence evaluation method for the difficulty values of the test questions of the embodiment no longer pays too much attention to the difficulty value of a specific test question, and the proportion of the artificial evaluation result data and the system evaluation result data that are consistent is calculated and compared through the comparison result of the difficulty value size relationship of the test question pair. If the proportion exceeds a certain value, it indicates that the confidence of the difficulty value of the test questions in the current test question bank is high, so that the confidence evaluation of the difficulty value of the test questions is realized. Therefore, the confidence evaluation method for the difficulty values of the test questions of the embodiment realizes the indexation of the confidence evaluation of the difficulty values of the test questions, provides an operable reference basis for judging the confidence of the difficulty values of the test questions in the test question bank, and is more simple, objective and efficient.

[0156] As an optional implementation manner of the embodiment, the confidence evaluation method for the difficulty values of the test questions of the embodiment comprises the following steps:

[0157] The test questions in the test question pair are stored in the format {test question A, test question B};

[0158] A preset test question difficulty value difference n is provided, and n is a positive number;

[0159] The size relationship between the difference between the difficulty value of test question A and the difficulty value of test question B and the preset test question difficulty value difference is compared;

[0160] If | difficulty value of test question A - difficulty value of test question B | < n;

[0161] Then, it is determined that the difficulty value of test question A is equivalent to the difficulty value of test question B, and the system evaluation result data is recorded as i0;

[0162] If difficulty value of test question A - difficulty value of test question B ≥ n;

[0163] Then, it is determined that the difficulty value of test question A is greater than the difficulty value of test question B, and the system evaluation result data is recorded as i1;

[0164] If difficulty value of test question A - difficulty value of test question B ≤ -n;

[0165] Then, it is determined that the difficulty value of test question A is less than the difficulty value of test question B, and the system evaluation result data is recorded as i2;

[0166] The system evaluation result data i0, i1 and i2 are summarized and counted.

[0167] The system evaluation result data of the embodiment is obtained by directly acquiring the difficulty values of test question A and test question B by the system and comparing the difference values. Considering that the difficulty value is a subjective evaluation index, in order to reduce the size deviation of the difficulty value caused by subjective difference, the embodiment introduces the difficulty value difference n. If | difficulty value of test question A - difficulty value of test question B | < n, it is determined that the difficulty value of test question A is equivalent to the difficulty value of test question B. The limitation condition of the difficulty value of the test question in the test question pair in the system evaluation result data is relaxed, which can better correspond to and compare with the artificial evaluation result data.

[0168] The system evaluation result data i0, i1 and i2 in the embodiment are mainly used to record the difficulty value system comparison results of the three cases that the difficulty value of test question A is greater than the difficulty value of test question B, the difficulty value of test question A is less than the difficulty value of test question B, and the difficulty value of test question A is equivalent to the difficulty value of test question B.

[0169] As an optional implementation manner of the embodiment, a confidence evaluation method of the difficulty value of a test question of the embodiment includes:

[0170] The test questions in the test question pair are stored as a test question pair data packet in the format {test question A, test question B};

[0171] The test question pair data packet is sent to at least one artificial evaluation account;

[0172] If the artificial evaluation difficulty value of test question A is greater than the difficulty value of test question B, the artificial evaluation result data j1 is obtained.

[0173] If the artificial evaluation difficulty value of the test question A is less than the artificial evaluation difficulty value of the test question B, the artificial evaluation result data j2 is obtained.

[0174] If the artificial evaluation difficulty value of the test question A is equal to the artificial evaluation difficulty value of the test question B, the artificial evaluation result data j0 is obtained.

[0175] The artificial evaluation result data j0, j1 and j2 are summarized and counted.

[0176] The confidence evaluation method of the test question difficulty value in the embodiment sets the artificial evaluation result of the test question A and the test question B as three evaluation results, i.e., the test question difficulty value of the test question A is greater than the test question difficulty value of the test question B, the test question difficulty value of the test question A is less than the test question difficulty value of the test question B, and the test question difficulty value of the test question A is equal to the test question difficulty value of the test question B according to the possible situation in the artificial evaluation. The artificial evaluation submits the test question difficulty value comparison result of the test question pair by the selection mode, and the artificial evaluation result data j0, j1 and j2 are summarized and counted.

[0177] In this way, the confidence evaluation method of the test question difficulty value in the embodiment provides a more simple and convenient mode for the artificial evaluation of the difficulty value size relationship of the test question pair, and is consistent with the type of the system evaluation result data, so that the system evaluation result data and the artificial evaluation result data can be compared one by one, and the proportion of the consistent comparison result is obtained.

[0178] Further, in the confidence evaluation method of the test question difficulty value in the embodiment, if the same test question pair data packet is sent to multiple artificial evaluation accounts, the artificial evaluation result data of the multiple artificial evaluation accounts is obtained respectively; the artificial evaluation result data of each test question pair in the test question pair data packet is counted and voted, and the artificial evaluation result data with a high proportion is selected as the artificial evaluation result of the test question pair; if there is a test question pair with the same artificial evaluation result data proportion, the artificial evaluation result of the test question pair is determined by comprehensively considering the artificial evaluation result data of other test question pairs in the test question pair data packet, or the system extracts the test question pair for re-evaluation and delivery.

[0179] In the confidence evaluation method of the test question difficulty value in the embodiment, the artificial evaluation of the difficulty value size relationship is performed, the evaluation is performed by multiple artificial evaluation accounts respectively, and the evaluation results are summarized, so that the artificial evaluation result data is more objective and accurate.

[0180] As an optional implementation manner of the embodiment, in the confidence evaluation method of the test question difficulty value in the embodiment, the artificial evaluation result data is used as the evaluation standard, the system evaluation result data is compared with the result, the proportion of the test question pairs with the same comparison result is counted and calculated, and the confidence of the test question difficulty value information in the test question library is evaluated, which includes:

[0181] For each test question pair, the system evaluation result data and the manual evaluation result data are compared;

[0182] whether the i0 of the system evaluation result data corresponds to the j0 of the manual evaluation result data;

[0183] whether the i1 of the system evaluation result data corresponds to the j1 of the manual evaluation result data;

[0184] whether the i2 of the system evaluation result data corresponds to the j2 of the manual evaluation result data;

[0185] If the result of the above step is yes, the system evaluation result data and the manual evaluation result data are the same, and are marked as R;

[0186] If the result of the above step is no, the system evaluation result data and the manual evaluation result data are different, and are marked as F;

[0187] The number of marks R is counted, the proportion of marks R is calculated and output, and the confidence of the test question difficulty value information in the test question bank is obtained.

[0188] Optionally, the i0 of the system evaluation result data and the j0 of the manual evaluation result data are both marked as 0, the i1 of the system evaluation result data and the j1 of the manual evaluation result data are both marked as 1, the i2 of the system evaluation result data and the j2 of the manual evaluation result data are both marked as 2, the comparison result mark R is marked as 1, and the comparison result mark F is marked as 0.

[0189] As an optional implementation of the embodiment, the confidence evaluation method of the test question difficulty value of the embodiment includes:

[0190] a certain number of test questions are randomly extracted from the test question bank;

[0191] test questions with the same test question attribute are selected according to the test question attribute filtering condition;

[0192] The selected test questions are paired and numbered;

[0193] Alternatively, a preset test question attribute filtering condition is used to select test questions with the same test question attribute in the test question bank;

[0194] A certain number of selected test questions are randomly extracted, paired, and numbered.

[0195] The test questions in the test question database extracted by the embodiment should have the same test question attributes or be grouped and numbered according to the test question attributes, because the test questions are only comparable in the same subject and grade, and the test question pairs need to be sent to the corresponding teachers for manual evaluation according to the corresponding subject, grade and other test question attribute information.

[0196] Optionally, the test question attributes include grade, subject and type, and further, the test question attributes can also include region and other (such as Olympiad test questions).

[0197] The embodiment extracts a certain number of test question pairs from the test question database. The number of test question pairs should not be too large, otherwise the workload of evaluation will be too large, and should not be too small, otherwise the reliability of evaluation will not be enough. Therefore, the number of test question pairs extracted by the embodiment should not be less than 50% of the number of test questions in the test question database under the corresponding test question attribute conditions.

[0198] As an optional implementation of the embodiment, the test question difficulty value of the test question pairs extracted from the test question database is manually labeled by the same account or the same test question difficulty value labeling rule.

[0199] According to the obtained confidence evaluation, the accuracy of the test question difficulty labeled by the account or the test question difficulty value labeling rule is evaluated.

[0200] As an optional implementation of the embodiment, the confidence evaluation method of the test question difficulty value of the embodiment includes:

[0201] The system presets a first confidence Z1 and a second confidence Z2, and the first confidence Z1 is less than the second confidence Z2.

[0202] The size relationship between the statistical confidence and the first confidence Z1 and the second confidence Z2 is calculated.

[0203] If the statistical confidence is greater than the second confidence Z2;

[0204] The test question difficulty value of the test question extracted from the test question database is accurate, and the current test question difficulty value label is retained.

[0205] If the statistical confidence is less than the first confidence Z1;

[0206] The test question difficulty value of the test question extracted from the test question database is inaccurate, and the current test question difficulty value label is discarded.

[0207] The test question is labeled again for test question difficulty value and put into the test question database.

[0208] If the confidence calculated by the statistical calculation is greater than or equal to the first confidence Z1 and less than or equal to the second confidence Z2;

[0209] The test question difficulty value to be evaluated is extracted from the test question library, and the test question is evaluated again for the confidence of the test question difficulty value.

[0210] The above embodiments are only used to illustrate the technical solutions described in the present application and not to limit the present application. Although the present application has been described in detail with reference to the above embodiments, the present application is not limited to the above specific embodiments. Therefore, any modification or equivalent replacement of the present application; and all technical solutions and improvements without departing from the spirit and scope of the application are all included in the scope of the claims of the present application.

Claims

1. A test question difficulty labeling method based on a test paper structure, characterized in that, The system comprises: obtain a test paper set with the same test paper attribute information in the test paper library; perform test paper structure analysis and test question information analysis on the test papers in the test paper set; construct a test question difficulty transfer structure diagram of all test questions in the test paper set according to the test question difficulty relationship corresponding to each type of test paper structure preset by the system; perform test question difficulty labeling on each test question based on the test question difficulty transfer structure diagram. The test question difficulty transfer structure diagram of all test questions in the test paper set is constructed according to the test question difficulty relationship corresponding to each type of test paper structure preset by the system, which comprises: In the same test paper, according to the test question difficulty relationship corresponding to the test paper structure of the test paper preset by the system, a test paper graph structure is formed according to the difficulty from small to large using a unidirectional acyclic graph; The test paper graph structures of different test papers are connected through the same test questions to construct a test question difficulty transfer structure diagram of all test questions in the test paper set. The test question difficulty labeling on each test question based on the test question difficulty transfer structure diagram comprises: Find the longest sequence in the test question difficulty transfer structure diagram; Set the test question difficulty value of the head test question of the sequence as the initial value K0, and the test question difficulty value of the tail test question as Kt. According to the length of the longest sequence, a uniform gradient is formed between K0 and Kt, and the test question difficulty values of the test questions between the head and tail of the longest sequence are assigned. The test question difficulty labeling on each test question based on the test question difficulty transfer structure diagram comprises: For the test questions in the subsequence of the non-longest sequence in the test question difficulty transfer structure diagram, starting from the test question connected to the longest sequence, the test question difficulty values of the subsequence pointed by the test question are sequentially increased according to the gradient, and the test question difficulty values of the subsequence pointed by the test question are sequentially decreased. The test question difficulty labeling on each test question based on the test question difficulty transfer structure diagram comprises: For the subgraph containing a strongly connected component existing in the test question difficulty transfer structure diagram; Merge the strongly connected component in the subgraph into a structure point, and the test question difficulty values of each test question in the strongly connected component are equal.

2. The test question difficulty labeling method based on a test paper structure according to claim 1, characterized in that, The test question difficulty relationship corresponding to each type of test paper structure preset by the system is: In the same set of test papers, the test question difficulty values of each test question in the same type of test question type increase with the increase of the serial number; Or train and learn the test papers of various types of test paper structure through a neural network model to obtain the test question difficulty relationship corresponding to each type of test paper structure.

3. The method of claim 1, wherein the method further comprises: The test paper set with the same test paper attribute information in the test paper library comprises: The test paper attribute information comprises test paper name, subject label, grade label and region label; The test paper name, subject label, grade label and region label are used as the screening conditions to screen the test papers with similar test paper names, the same subject labels, grade labels and region labels in the test paper library, and store them as a test paper set.

4. The test question difficulty labeling method based on the test paper structure according to claim 3, characterized in that, The test paper name similar means that one or more keywords / words in the test paper name are matched successfully.

5. The test question difficulty labeling method based on the test paper structure according to claim 1 or 3, characterized in that, The test paper structure analysis on the test papers in the test paper set comprises: Obtain the test question type and the number of test questions contained in each test question type for each set of test papers in the test paper set; Match the corresponding test paper structure in the system preset test paper structure library according to the test question type of the test paper and the number of test questions contained in each test question type. The question information analysis for the test papers in the test paper set comprises: For each test paper in the test paper set, the question ID, subject information and grade information of each question are acquired.

6. A test question difficulty labeling device based on a test paper structure, characterized by, Comprise: An acquisition module acquires a test paper set with the same test paper attribute information in a test paper library; An analysis module analyzes the test paper structure and question information for the test papers in the test paper set; A structure diagram construction module constructs a question difficulty transfer structure diagram of all questions in the test paper set according to the question difficulty relationship corresponding to each type of test paper structure preset by the system; And a question difficulty labeling module labels the question difficulty of each question based on the question difficulty transfer structure diagram; The question difficulty transfer structure diagram of all questions in the test paper set is constructed according to the question difficulty relationship corresponding to each type of test paper structure preset by the system, which comprises: In the same test paper, according to the question difficulty relationship corresponding to the test paper structure of the test paper preset by the system, a test paper structure diagram is formed using a unidirectional acyclic graph according to difficulty from small to large; The test paper structure diagrams of different test papers are connected through the same questions to construct a question difficulty transfer structure diagram of all questions in the test paper set; The question difficulty labeling of each question based on the question difficulty transfer structure diagram comprises: Find the longest sequence in the question difficulty transfer structure diagram; The question difficulty value of the head question of the sequence is set as an initial value K0, and the question difficulty value of the tail question is set as Kt, and a uniform gradient is formed between K0 and Kt according to the length of the longest sequence, and the question difficulty values of the questions between the head and tail of the longest sequence are assigned; The question difficulty labeling of each question based on the question difficulty transfer structure diagram comprises: For the questions in the subsequence of the non-longest sequence in the question difficulty transfer structure diagram, starting from the question connected to the longest sequence, the question difficulty values of the subsequence pointed to by the question increase in turn according to the gradient, and the question difficulty values of the subsequence pointed to by the question decrease in turn; The question difficulty labeling of each question based on the question difficulty transfer structure diagram comprises: For the subgraph containing a strongly connected component existing in the question difficulty transfer structure diagram; The strongly connected components in the subgraph are merged into one structure point, and the question difficulty values of the questions in the strongly connected components are equal.

7. A storage medium, characterized by A computer executable program is stored, and when the computer executable program is executed, a question difficulty labeling method based on test paper structure according to any one of claims 1-5 is realized.

Images