Test question classification model training method, device, equipment, medium and program product

Abstract

The application provides a test question classification model training method and device, equipment, medium and product. The model includes a first classification layer, a second classification layer and a mapping layer. The method includes: obtaining a first test question sample carrying a first label and a second test question sample carrying a second label with the same test question text; performing classification prediction based on the first test question sample through the first classification layer to obtain a first predicted node to which the test question text belongs, and performing classification prediction based on the second test question sample through the second classification layer to obtain a second predicted node to which the test question text belongs; mapping the second predicted node based on the correlation between the nodes in the teaching material system through the mapping layer to obtain a mapping node corresponding to the second predicted node; and updating the model parameters of the test question classification model in combination with the first label, the second label, the first predicted node, the second predicted node and the mapping node. In this way, the training efficiency and classification accuracy of the test question classification model can be improved.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to a training method, device, electronic device, computer-readable storage medium and computer program product for a test question classification model. Background Art

[0002] The teaching materials system of each subject in teaching often includes chapters (units), sections (lessons), knowledge points, etc., which are nodes at different levels in the teaching materials system. In related technologies, determining the node corresponding to a test question, such as determining the knowledge point to which a test question belongs, can be achieved by training a corresponding classification model. However, the model training method in related technologies often constructs training samples carrying corresponding node labels, and trains the corresponding classification model based on the constructed training samples. However, if the trained classification model is to achieve a certain accuracy, this training method requires constructing a large number of training samples, which makes the model training efficiency low. Summary of the Invention

[0003] The embodiments of the present application provide a training method, device, electronic device, computer-readable storage medium and computer program product for a test question classification model, which can improve the training efficiency and classification accuracy of the test question classification model.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The embodiment of the present application provides a training method for a test question classification model, wherein the test question classification model includes: a first classification layer, a second classification layer, and a mapping layer, including:

[0006] Obtain a first test question sample and a second test question sample having the same test question text, wherein the first test question sample carries a first label, and the second test question sample carries a second label;

[0007] The first tag is used to indicate, in a teaching material system including a plurality of content nodes, the first node to which the test question text belongs, and the second tag is used to indicate the second node to which the test question text belongs, and the first node and the second node are at different node levels in the teaching material system;

[0008] Performing classification prediction based on the first test question sample through the first classification layer to obtain a first prediction node to which the test question text belongs, and performing classification prediction based on the second test question sample through the second classification layer to obtain a second prediction node to which the test question text belongs;

[0009] Mapping the second prediction node based on the association relationship between nodes in the teaching material system through the mapping layer to obtain a mapping node corresponding to the second prediction node, wherein the mapping node and the first prediction node are at the same node level;

[0010] In combination with the first label, the second label, the first prediction node, the second prediction node and the mapping node, the model parameters of the test question classification model are updated.

[0011] The embodiment of the present application provides a training device for a test question classification model, wherein the test question classification model includes: a first classification layer, a second classification layer, and a mapping layer, including:

[0012] an acquisition module, configured to acquire a first test question sample and a second test question sample having the same test question text, wherein the first test question sample carries a first label, and the second test question sample carries a second label; wherein the first label is used to indicate, in a teaching material system including a plurality of content nodes, a first node to which the test question text belongs, and the second label is used to indicate a second node to which the test question text belongs, and the first node and the second node are at different node levels in the teaching material system;

[0013] a classification module configured to perform classification prediction based on the first test question sample through the first classification layer to obtain a first prediction node to which the test question text belongs, and to perform classification prediction based on the second test question sample through the second classification layer to obtain a second prediction node to which the test question text belongs;

[0014] a mapping module, configured to map the second prediction node based on the association relationship between nodes in the teaching material system to obtain a mapping node corresponding to the second prediction node, wherein the mapping node and the first prediction node are at the same node level;

[0015] An updating module is used to update the model parameters of the test question classification model by combining the first label, the second label, the first prediction node, the second prediction node and the mapping node.

[0016] In the above scheme, the first classification layer includes a first encoding layer and a first prediction layer, and the second classification layer includes a second encoding layer and a second prediction layer; wherein, the first encoding layer and the second encoding layer share model parameters; the classification module is also used to perform vector encoding on the first test question sample through the first encoding layer to obtain a first encoding vector, and perform classification prediction based on the first encoding vector through the first prediction layer to obtain a first prediction node to which the test question text belongs; perform vector encoding on the second test question sample through the second encoding layer to obtain a second encoding vector, and perform classification prediction based on the second encoding vector through the second prediction layer to obtain a second prediction node to which the test question text belongs.

[0017] In the above scheme, the classification module is also used to perform word segmentation processing on the first test question sample through the first encoding layer to obtain multiple sample words; encode each of the sample words separately to obtain the word vector corresponding to each of the sample words; and perform vector averaging on the word vectors corresponding to each of the sample words to obtain the first encoding vector.

[0018] In the above scheme, the classification module is also used to extract keywords from the first test question sample through the first coding layer to obtain multiple keywords; encode each of the keywords separately to obtain the keyword vector corresponding to each of the keywords; obtain the weight corresponding to each of the keywords, and perform weighted summation of the keyword vectors corresponding to each of the keywords based on the weight to obtain the first coding vector.

[0019] In the above scheme, the acquisition module is also used to obtain the stem, answer and analytical content corresponding to the answer of the target test question when the test question text corresponds to the target test question; splice the stem, answer and analytical content corresponding to the answer of the target test question to obtain the test question text; and label based on the test question text to obtain the first test question sample carrying the first label and the second test question sample carrying the second label.

[0020] In the above scheme, the update module is also used to determine the value of the first loss function corresponding to the first classification layer based on the first label and the first prediction node, determine the value of the second loss function corresponding to the second classification layer based on the second label and the second prediction node, and determine the value of the third loss function corresponding to the mapping layer based on the first label and the mapping node; obtain the loss function of the test question classification model constructed by the first loss function, the second loss function and the third loss function; determine the value of the loss function of the test question classification model based on the value of the first loss function, the value of the second loss function, and the value of the third loss function; and update the model parameters of the test question classification model based on the value of the loss function of the test question classification model.

[0021] In the above scheme, the device also includes an application module, which is used to obtain the test text of the test question to be classified; through the first classification layer, the test text of the test question to be classified is classified and predicted to obtain the target node to which the test text of the test question to be classified belongs; or, through the second classification layer, the test text of the test question to be classified is classified and predicted to obtain the node to be mapped to which the test text of the test question to be classified belongs, and through the mapping layer, based on the association relationship between the nodes in the teaching material system, the node to be mapped is mapped to obtain the target node to which the test text of the test question to be classified belongs.

[0022] In the above scheme, the application module is also used to mark the test questions to be classified based on the target node to obtain marked test questions with annotations; obtain the test question outline, and match the annotations of the marked test questions with the content in the test question outline to obtain a matching result; when the matching result indicates that the annotations do not match the content in the test question outline, determine that the marked test questions are out-of-scope test questions.

[0023] In the above solution, the device further includes an association module, which is used to obtain a node association relationship table corresponding to the teaching material system; and update the association relationship between nodes in the teaching material system based on the node association relationship table.

[0024] In the above scheme, the teaching material system includes multiple unit nodes, each of the unit nodes includes at least two unit sub-nodes, and each of the unit sub-nodes includes at least one knowledge point; when the first node is the unit sub-node and the second node is the knowledge point, the first prediction node is the predicted unit sub-node and the second prediction node is the predicted knowledge point; the mapping module is also used to map the predicted knowledge point through the mapping layer based on the association relationship between the unit sub-nodes and the knowledge points in the teaching material system, and obtain the unit sub-node corresponding to the predicted knowledge point as the mapping node.

[0025] An embodiment of the present application provides an electronic device, including:

[0026] a memory for storing executable instructions;

[0027] The processor is used to implement the training method of the test question classification model provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0028] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the training method of the test question classification model provided in the embodiment of the present application.

[0029] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the training method for the test question classification model provided in the present invention.

[0030] The embodiments of the present application have the following beneficial effects:

[0031] By performing classification prediction on a first test question sample and a second test question sample with the same test question text but different labels, a first prediction node and a second prediction node are obtained. The second prediction node is then mapped based on the association between nodes in the teaching material system to obtain a mapping node at the same node level as the first prediction node. Finally, the test question classification model is updated based on the first prediction node, the second prediction node, the mapping node, and the corresponding labels. In this way, by updating the test question classification model during the training process using the mapping nodes obtained based on the association between nodes in the teaching material system, not only can the training efficiency of the test question classification model be improved, but also the accuracy of the test question classification model in classifying test questions can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 1 is a schematic diagram of the architecture of a training system 100 for a test question classification model provided in an embodiment of the present application;

[0033] Figure 2 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0034] Figure 3 Schematic diagram of the flow of the training method of the test question classification model provided in the embodiment of the present application;

[0035] Figure 4 It is a structural diagram of the test question classification model provided in the embodiment of the present application;

[0036] Figure 5 It is a structural diagram of the teaching material system provided in the embodiment of this application;

[0037] Figure 6 It is a structural diagram of the test question classification model provided in the embodiment of the present application;

[0038] Figure 7 1 is a flow chart of a process for determining a first prediction node and a second prediction node according to an embodiment of the present application;

[0039] Figure 8 is a schematic diagram of a process for determining a first encoding vector provided in an embodiment of the present application;

[0040] Figure 9 is a schematic diagram of a process for determining a first encoding vector provided in an embodiment of the present application;

[0041] Figure 10 It is a structural diagram of the test question classification model provided in the embodiment of the present application;

[0042] Figure 11 It is a structural diagram of the test question classification model provided in the embodiment of the present application;

[0043] Figure 12 Schematic diagram of the flow of the training method of the test question classification model provided in the embodiment of the present application;

[0044] Figure 13 This is a schematic diagram of the classification interface of the test questions to be classified provided in the embodiment of the present application;

[0045] Figure 14 This is a schematic diagram of the classification interface of the test questions to be classified provided in the embodiment of the present application;

[0046] Figure 15 This is a schematic diagram comparing the effects provided by the embodiments of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0048] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0049] In the following description, the terms "first, second, and third" are used merely to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that the specific order or sequence of "first, second, and third" can be interchanged where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In the following description, the term "plurality" refers to at least two.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0051] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0052] 1) BERT (Bidirectional Encoder Representations from Transformer) is a pre-training technology for natural language processing. It is used to train on large-scale unlabeled corpora to obtain semantic representations of text containing rich semantic information. The semantic representations of text are then fine-tuned for specific natural language processing tasks and ultimately applied to the tasks.

[0053] 2) In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed can be real-time or have a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations executed are executed.

[0054] 3) Client: an application running in the terminal to provide various services, such as video client, instant messaging client, browser client, education client, live broadcast client, map client, etc.

[0055] 4) LSTM (Long short-term memory) model, a long short-term memory model, can be used for sequence modeling.

[0056] 5) Convolutional Neural Network (CNN): A feedforward neural network, generally composed of one or more convolutional layers (network layers that use convolutional mathematical operations) and a fully connected layer at the end. The neurons within the network can respond to partial areas of the input image and generally have excellent performance in the field of visual image processing.

[0057] 6) Adam (Adaptive momentum) algorithm: A stochastic optimization method with adaptive momentum, often used as an optimizer algorithm in deep learning.

[0058] 7) batch_size: indicates the number of parameters passed to the program for training at a single time.

[0059] In order to comply with the Ministry of Education's policy that student assignments must not exceed the syllabus, it is necessary to check whether the assignments assigned by teachers exceed the syllabus. In this scenario, correctly identifying the test points of the test questions is particularly important. However, the related art only identifies the test points of the test questions. However, a test question may correspond to multiple test points at different levels in the textbook system, such as chapters and knowledge points. Therefore, the related art cannot accurately identify the test points of the test questions, and thus cannot accurately detect whether the test questions exceed the syllabus. At the same time, because the related art only identifies the test points of the test questions, the number of annotations corresponding to test points other than knowledge points is relatively small. For example, the chapter annotations of the test questions are far less than the knowledge point annotations of the test questions. In this way, manual annotation of the test chapters is required. However, relying solely on manual annotation of the test chapters will be very time-consuming and labor-intensive.

[0060] Based on this, the embodiments of the present application provide a training method, device, electronic device, computer-readable storage medium and computer program product for a test question classification model, which can improve the training efficiency and classification accuracy of the test question classification model, thereby accurately identifying the test points corresponding to the test questions at different levels in the textbook system, which not only improves the efficiency of marking other test points other than knowledge points, but also greatly improves the accuracy of detecting test questions that are out of syllabus.

[0061] See also Figure 1 , Figure 1 FIG3 is a schematic diagram of the architecture of a training system 100 for a test question classification model provided in an embodiment of the present application. A terminal (terminal 400 is shown as an example) is connected to a server 200 via a network 300. Network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of the two. Terminal 400 and server 200 are interconnected via a wired or wireless network.

[0062] The terminal 400 is used to send a first test question sample with a first label and a second test question sample with a second label, both of which have the same test question text, to the server 200;

[0063] The server 200 is used to obtain a first test question sample and a second test question sample having the same test question text, wherein the first test question sample carries a first label and the second test question sample carries a second label; wherein the first label is used to indicate, in a teaching material system including a plurality of content nodes, a first node to which the test question text belongs, and the second label is used to indicate a second node to which the test question text belongs, and the first node and the second node are at different node levels in the teaching material system; through the first classification layer of the test question classification model, classification prediction is performed based on the first test question sample to obtain a first prediction node to which the test question text belongs, and through the second classification layer of the test question classification model, classification prediction is performed based on the second test question sample to obtain a second prediction node to which the test question text belongs; through the mapping layer of the test question classification model, based on the association relationship between nodes in the teaching material system, the second prediction node is mapped to obtain a mapping node corresponding to the second prediction node, and the mapping node and the first prediction node are at the same node level; and the model parameters of the test question classification model are updated by combining the first label, the second label, the first prediction node, the second prediction node and the mapping node.

[0064] The server 200 is also used to perform classification prediction on the test questions to be classified based on the updated test question classification model, determine the target node to which the test question text of the test question to be classified belongs; and mark the corresponding test questions based on the target node, and send the corresponding marks to the terminal 400.

[0065] The terminal 400 is used to present the annotations obtained from the server 200 in the display interface 401 - 1 .

[0066] In some embodiments, the server 200 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Deliver Network), and big data and artificial intelligence platforms. The terminal 400 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, and a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device, a smart speaker, and a smart watch), etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0067] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. In practical applications, the electronic device can be Figure 1The server 200 or terminal 400 shown, see Figure 2 , Figure 2 The electronic device shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 2 Various buses are labeled as bus system 440 .

[0068] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0069] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0070] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0071] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0072] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0073] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0074] A network communication module 452 for reaching other computing devices via one or more (wired or wireless) network interfaces 420 , exemplary network interfaces 420 including Bluetooth, WiFi, and USB;

[0075] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripheral devices and displaying content and information);

[0076] The input processing module 454 is configured to detect one or more user inputs or interactions from one of the one or more input devices 432 and to translate the detected inputs or interactions.

[0077] In some embodiments, the event processing device based on the event processing model provided in the embodiments of the present application can be implemented in software. Figure 2 A training device 455 of a test question classification model stored in a memory 450 is shown, which can be software in the form of a program and plug-in, including the following software modules: an acquisition module 4551, a classification module 4552, a mapping module 4553 and an update module 4554. These modules are logical and can therefore be arbitrarily combined or further split according to the functions implemented.

[0078] In other embodiments, the training device of the test question classification model provided in the embodiments of the present application can be implemented in hardware. As an example, the training device of the basic test question classification model provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the training method of the test question classification model provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs) or other electronic components.

[0079] In some embodiments, the terminal or server can implement the training method of the test question classification model provided in the embodiment of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a local (Native) application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as an instant messaging APP, a web browser APP; it can also be a small program, that is, a program that can be run only by downloading it into a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be an application, module or plug-in in any form.

[0080] Based on the above description of the training system and electronic device for the test question classification model provided by the embodiment of the present application, the following describes the training method for the test question classification model provided by the embodiment of the present application. In actual implementation, the training method for the test question classification model provided by the embodiment of the present application can be implemented by the terminal or the server alone, or by the terminal and the server in collaboration, so that Figure 1 The server 200 in the embodiment of the present application alone performs the training method of the test question classification model as an example for explanation. Figure 3 , Figure 3 This is a flow chart of the training method of the test question classification model provided in the embodiment of the present application. It should be noted that the test question classification model includes: a first classification layer, a second classification layer and a mapping layer. Figure 4 , Figure 4 This is a schematic diagram of the structure of the test question classification model provided in the embodiment of the present application. Figure 3 and Figure 4 The steps shown will be described.

[0081] In step 101 , the server obtains a first test question sample and a second test question sample having the same test question text. The first test question sample carries a first label, and the second test question sample carries a second label.

[0082] The first label is used to indicate, within a textbook system comprising multiple content nodes, the first node to which the test question text belongs, and the second label is used to indicate the second node to which the test question text belongs, with the first node and the second node being at different node levels within the textbook system. It should be noted that the textbook system comprises multiple node levels and multiple content nodes, with the multiple content nodes being at the same or different node levels, and the first node and the second node being at different node levels within the textbook system.

[0083] For example, see Figure 5 , Figure 5 This is a schematic diagram of the structure of the teaching material system provided in the embodiment of this application, based on Figure 5The teaching material system includes three node levels and multiple content nodes. The multiple content nodes are at the same or different node levels. When the first node is at the first node level of the teaching material system, the second node can be at the second node level or the third node level of the teaching material system; when the first node is at the second node level of the teaching material system, the second node can also be at the third node level; this embodiment of the application does not impose any restrictions on this.

[0084] In actual implementation, the textbook system can be textbooks for different subjects or different versions of textbooks for the same subject; the content nodes included in the textbook system can be the test points of the textbook, that is, the chapters (or units), sections (or sub-units) and knowledge points of the textbook, and the multiple node levels included in the textbook system can be determined according to the chapters, sections and knowledge points of the textbook. For example, see Figure 5 The first node level corresponds to the chapters of the textbook, the second node level corresponds to the sections of the textbook, and the third node level corresponds to the knowledge points of the textbook. Since the chapters, sections and knowledge points in each textbook are related, the textbook system corresponding to each textbook is determined based on the chapters, sections and knowledge points.

[0085] In actual implementation, the first choice is to obtain a first test question sample and a second test question sample with the same test question text. Specifically, when the test question text corresponds to a target test question, the stem, answer and analytical content corresponding to the answer of the target test question are obtained; the stem, answer and analytical content corresponding to the answer of the target test question are spliced ​​to obtain the test question text; and labeling is performed based on the test question text to obtain a first test question sample carrying a first label and a second test question sample carrying a second label.

[0086] It should be noted that the same test question text means that the question stem, answer and the corresponding analysis content of the answer are all the same, and the label refers to the node to which the test question belongs, such as the chapter and knowledge point corresponding to the test question. Therefore, the first label here can be the chapter corresponding to the test question, and the second label can be the knowledge point corresponding to the test question.

[0087] Step 102: Perform classification prediction based on the first test question sample through the first classification layer to obtain a first prediction node to which the test question text belongs, and perform classification prediction based on the second test question sample through the second classification layer to obtain a second prediction node to which the test question text belongs.

[0088] In actual implementation, see Figure 6 , Figure 6 This is a schematic diagram of the structure of the test question classification model provided in the embodiment of the present application, based on Figure 6The first classification layer includes a first coding layer and a first prediction layer, and the second classification layer includes a second coding layer and a second prediction layer; wherein the first coding layer and the second coding layer share model parameters. Here, after obtaining the first test question sample and the second test question sample, the process of obtaining the first prediction node and the second prediction node based on the first coding layer and the first prediction layer, and the second coding layer and the second prediction layer, see Figure 7 , Figure 7 This is a flow chart of the process of determining the first prediction node and the second prediction node provided by the embodiment of the present application, based on Figure 3 , step 102 can be implemented as follows:

[0089] Step 1021: vector encode the first test question sample through the first encoding layer to obtain a first encoding vector, and perform classification prediction based on the first encoding vector through the first prediction layer to obtain a first prediction node to which the test question text belongs.

[0090] In actual implementation, there are two ways to obtain the first coding vector by performing vector coding on the first test question sample through the first coding layer. Next, the two ways to obtain the first coding vector by performing vector coding on the first test question sample are explained.

[0091] In some embodiments, see Figure 8 , Figure 8 This is a schematic diagram of the process of determining the first encoding vector provided by the embodiment of the present application, based on Figure 8 First, the first test question sample is segmented through the first encoding layer to obtain multiple sample words; then each sample word is encoded separately to obtain the word vector corresponding to each sample word; the word vector corresponding to each sample word is averaged to obtain the first encoding vector.

[0092] As an example, firstly, the first test question sample is segmented to obtain multiple sample words; then each sample word is encoded separately to obtain the word vector h corresponding to each sample word. cls ,h1,h2,……,h n , denoted as H = {h cls ,h1,h2,……,h n}, then according to formula (1) H={h cls ,h1,h2,……,h n}, and the vectors of each word in the word vector are averaged to obtain the first encoding vector, that is,

[0093]

[0094] Among them, n is the number of sample words, r is the first encoding vector, and "cls" is the distinguishing mark corresponding to each test question sample, which is obtained before encoding each test question sample. Here, the distinguishing mark distinguishes different test question samples, so that each test question sample is encoded based on the distinguishing mark.

[0095] In other embodiments, see Figure 9 , Figure 9 This is a schematic diagram of the process of determining the first encoding vector provided by the embodiment of the present application, based on Figure 9 First, through the first coding layer, keywords are extracted from the first test question sample to obtain multiple keywords; each keyword is encoded separately to obtain the keyword vector corresponding to each keyword; the weight corresponding to each keyword is obtained, and the keyword vectors corresponding to each keyword are weighted summed based on the weight to obtain the first coding vector.

[0096] As an example, firstly, the first coding layer is used to extract keywords from the first test question sample to obtain multiple keywords; then, each keyword is encoded separately to obtain the keyword vector h corresponding to each keyword. cls ,h1,h2,……,h n , denoted as H = {h cls ,h1,h2,……,h n}; Then get the weight p corresponding to each keyword cls ,p1,p2,……,p n Finally, based on formula (2), H={h cls ,h1,h2,……,h n} perform weighted summation on the keyword vectors corresponding to each keyword to obtain the first encoding vector, i.e.

[0097] r=p cls *h cls +p1*h1+……+p n *h n ...Formula (2);

[0098] Among them, n is the number of sample words, r is the first encoding vector, and "cls" is the distinguishing mark corresponding to each test question sample, which is obtained before encoding each test question sample. Here, the distinguishing mark distinguishes different test question samples, so that each test question sample is encoded based on the distinguishing mark.

[0099] It should be noted that the weights corresponding to each keyword may be the same. When the weights corresponding to each keyword are the same, the keyword vectors corresponding to each keyword are weighted and summed to obtain the first coding vector, which is also the process of performing vector averaging on the keyword vectors corresponding to each keyword to obtain the first coding vector.

[0100] In actual implementation, after obtaining the first coding vector, classification prediction is performed based on the first coding vector through the first prediction layer to obtain the first prediction node to which the test question text belongs.

[0101] Continuing with the above example, after obtaining the first coding vector r, the first prediction layer is used to perform classification prediction based on the first coding vector to obtain the first prediction node to which the test question text belongs, that is,

[0102] y k =W k *r……Formula (3);

[0103] Where y is the prediction node, W is the model parameter to be updated, k∈{1, 2, 3}, when k is 1, y1 is the first prediction node, W1 is the model parameter to be updated in the first classification layer; when k is 2, y2 is the second prediction node, W2 is the model parameter to be updated in the second classification layer.

[0104] Step 1022: Perform vector encoding on the second test question sample through the second encoding layer to obtain a second encoding vector, and perform classification prediction based on the second encoding vector through the second prediction layer to obtain a second prediction node to which the test question text belongs.

[0105] In actual implementation, since the first coding layer and the second coding layer share model parameters, the process of performing vector encoding on the second test question sample through the second coding layer to obtain a second coding vector, and performing classification prediction based on the second coding vector through the second prediction layer to obtain the second prediction node to which the test question text belongs is the same as the process of performing vector encoding on the first test question sample through the first coding layer to obtain a first coding vector, and performing classification prediction based on the first coding vector through the first prediction layer to obtain the first prediction node to which the test question text belongs. Therefore, the process of determining the second prediction node is not described in detail. At the same time, since the first coding layer and the second coding layer share model parameters, the amount of calculation when updating the model parameters is reduced during the training process of the test question classification model, thereby reducing the consumption of computing resources and accelerating the model convergence speed.

[0106] Step 103 : Mapping the second prediction node through the mapping layer based on the association relationship between nodes in the teaching material system to obtain a mapping node corresponding to the second prediction node.

[0107] Here, the mapping node and the first prediction node are at the same node level.

[0108] In actual implementation, the associations between nodes in the textbook system are first obtained. Then, based on the associations between nodes in the textbook system, the second predicted node is mapped through the mapping layer to obtain a mapped node corresponding to the second predicted node. It should be noted that the associations between nodes in the textbook system are pre-set for each textbook.

[0109] Continuing with the above example, after determining the second prediction node, the second prediction node is mapped based on the association relationship between nodes in the teaching material system through the mapping layer to obtain the mapping node corresponding to the second prediction node, that is,

[0110] y g =G*y2...Formula (4);

[0111] Among them, y g is the mapping node, y2 is the second prediction node, and G is the model parameter to be updated in the mapping layer.

[0112] Step 104 , combining the first label, the second label, the first prediction node, the second prediction node and the mapping node to update the model parameters of the test question classification model.

[0113] In actual implementation, after determining the first prediction node, the second prediction node and the mapping node, the process of updating the model parameters of the test question classification model specifically includes: determining the value of the first loss function corresponding to the first classification layer based on the first label and the first prediction node, determining the value of the second loss function corresponding to the second classification layer based on the second label and the second prediction node, and determining the value of the third loss function corresponding to the mapping layer based on the first label and the mapping node; obtaining the loss function of the test question classification model constructed by the first loss function, the second loss function and the third loss function; determining the value of the loss function of the test question classification model based on the value of the first loss function, the value of the second loss function, and the value of the third loss function; and updating the model parameters of the test question classification model based on the value of the loss function of the test question classification model.

[0114] As an example, the first loss function, the second loss function, the third loss function and the loss function of the test question classification model can all be sigmoid functions. After determining the first prediction node, the second prediction node and the mapping node, the value loss1 of the first loss function corresponding to the first classification layer is determined based on the first label and the first prediction node, the value loss2 of the second loss function corresponding to the second classification layer is determined based on the second label and the second prediction node, and the value loss3 of the third loss function corresponding to the mapping layer is determined based on the first label and the mapping node. Then, based on the value of the first loss function, the value of the second loss function, and the value of the third loss function, the value of the loss function of the test question classification model is determined. Finally, the value of the loss function based on the test question classification model Update the model parameters of the test question classification model.

[0115] In actual implementation, after obtaining the trained test question classification model, the test questions to be classified can be classified based on the trained test question classification model. Specifically, the test question text of the test question to be classified is obtained; through the first classification layer, the test question text of the test question to be classified is classified and predicted, and the target node to which the test question text of the test question to be classified belongs is obtained; or, through the second classification layer, the test question text of the test question to be classified is classified and predicted, and the node to be mapped to which the test text of the test question to be classified belongs is obtained; and through the mapping layer, based on the association relationship between nodes in the teaching material system, the node to be mapped is mapped to obtain the target node to which the test question text of the test question to be classified belongs.

[0116] It should be noted that after determining the target node to which the test question text of the test question to be classified belongs, the test question text is annotated based on the target node to which the test question text belongs, and annotated test questions with annotations are obtained, so that corresponding applications are performed based on the annotated test questions.

[0117] As an example, when detecting out-of-syllabus questions based on marked questions, specifically, first obtain the question outline, and match the annotations of the marked questions with the question outline to obtain a matching result; when the matching result indicates that the annotations do not match the question outline, it is determined that the marked questions are out-of-syllabus questions.

[0118] As an example, when recommending test questions based on annotated test questions, specifically, first obtain user information, and determine the target annotation that is suitable for the user based on the user information, and based on the target annotation, filter out the target test questions from multiple annotated test questions, and recommend the target test questions to the user. It should be noted that in the embodiments of the present application, when user information and other related data are involved, when the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0119] In actual implementation, since the node association table is pre-set, when it is updated, the node association table corresponding to the updated textbook system is obtained; then, based on the node association table, the associations between nodes in the textbook system are updated. This ensures timely updates of the associations between nodes, allowing the updated node associations to be incorporated into the training of the test question classification model, improving the accuracy of the test question classification model.

[0120] In some embodiments, see Figure 5 The teaching material system includes multiple unit nodes of the first node level, each unit node includes at least two unit sub-nodes of the second node level, and each unit sub-node includes at least one knowledge point of the third node level.

[0121] As an example, when the first node is a unit subnode and the second node is a knowledge point, the first predicted node is a predicted unit subnode, and the second predicted node is a predicted knowledge point. The mapping layer maps the second predicted node based on the association between nodes in the textbook system to obtain a mapping node corresponding to the second predicted node. Specifically, the mapping layer maps the predicted knowledge point based on the association between the unit subnode and the knowledge point in the textbook system to obtain a unit subnode corresponding to the predicted knowledge point as the mapping node. The test question classification model is then updated by combining the predicted unit subnode, the predicted knowledge point, the mapped unit subnode, the first label, and the second label.

[0122] In actual implementation, when the first node is a unit sub-node and the second node is a knowledge point, the trained test question classification model can be used to determine the sub-unit to which the corresponding test question belongs. Specifically, the test text of the test question to be classified is obtained; through the first classification layer, the test text of the test question to be classified is classified and predicted, and the sub-unit to which the test text of the test question to be classified belongs is obtained; or, through the second classification layer, the test text of the test question to be classified is classified and predicted, and the knowledge point to which the test text of the test question to be classified belongs is obtained; and through the mapping layer, based on the association relationship between nodes in the teaching material system, the knowledge points are mapped to obtain the sub-unit to which the test text of the test question to be classified belongs.

[0123] It should be noted that the question classification model can also be used to determine the knowledge point to which the corresponding question belongs, see Figure 10 , Figure 10 This is a schematic diagram of the structure of the test question classification model provided in the embodiment of the present application, based on Figure 10 In addition to mapping the second prediction node, the mapping layer in the test question classification model can also map the first prediction node. Specifically, after determining the first prediction node and the second prediction node to which the test question text belongs, the first prediction node is mapped through the mapping layer based on the association relationship between nodes in the teaching material system to obtain a mapping node corresponding to the first prediction node, wherein the mapping node and the second prediction node are at the same node level.

[0124] In actual implementation, after determining the first prediction node, the second prediction node and the mapping node, the process of updating the model parameters of the test question classification model specifically includes: determining the value of the first loss function corresponding to the first classification layer based on the first label and the first prediction node, determining the value of the second loss function corresponding to the second classification layer based on the second label and the second prediction node, and determining the value of the third loss function corresponding to the mapping layer based on the second label and the mapping node; obtaining the loss function of the test question classification model constructed by the first loss function, the second loss function and the third loss function; determining the value of the loss function of the test question classification model based on the value of the first loss function, the value of the second loss function, and the value of the third loss function; and updating the model parameters of the test question classification model based on the value of the loss function of the test question classification model.

[0125] Continue to see Figure 5 The teaching material system includes multiple unit nodes of the first node level, each unit node includes at least two unit sub-nodes of the second node level, and each unit sub-node includes at least one knowledge point of the third node level.

[0126] As an example, when the first node is a unit subnode and the second node is a knowledge point, the first predicted node is a predicted unit subnode, and the second predicted node is a predicted knowledge point. The process of mapping the first predicted node based on the associations between nodes in the textbook system to obtain a mapped node corresponding to the first predicted node specifically includes mapping the predicted unit subnode based on the associations between unit subnodes and knowledge points in the textbook system to obtain the knowledge point corresponding to the predicted unit subnode as the mapped node. The test question classification model is then updated by combining the predicted unit subnode, the predicted knowledge point, the mapped knowledge point, the first label, and the second label.

[0127] In actual implementation, when the first node is a unit sub-node and the second node is a knowledge point, the trained test question classification model can be used to determine the knowledge point to which the corresponding test question belongs. Specifically, the test text of the test question to be classified is obtained; through the second classification layer, the test text of the test question to be classified is classified and predicted to obtain the knowledge point to which the test text of the test question to be classified belongs; or, through the first classification layer, the test text of the test question to be classified is classified and predicted to obtain the sub-unit to which the test text of the test question to be classified belongs, and through the mapping layer, based on the association relationship between nodes in the teaching material system, the sub-unit is mapped to obtain the knowledge point to which the test text of the test question to be classified belongs.

[0128] In some embodiments, the question classification model can also be used to simultaneously determine the subunit and knowledge point to which the corresponding question belongs. Figure 11 , Figure 11 This is a schematic diagram of the structure of the test question classification model provided in the embodiment of the present application, based on Figure 11 The mapping layer in the test question classification model can also simultaneously map the first prediction node and the second prediction node. Specifically, after determining the first prediction node and the second prediction node to which the test question text belongs, the first prediction node is mapped through the first mapping layer based on the association relationship between nodes in the teaching material system to obtain a first mapping node corresponding to the first prediction node, and the second prediction node is mapped through the second mapping layer to obtain a second mapping node corresponding to the second prediction node, wherein the first mapping node and the first prediction node are at the same node level, and the second mapping node and the second prediction node are at the same node level.

[0129] In actual implementation, after determining the first prediction node, the second prediction node, the first mapping node and the second mapping node, the process of updating the model parameters of the test question classification model specifically includes: determining the value of the first loss function corresponding to the first classification layer based on the first label and the first prediction node, determining the value of the second loss function corresponding to the second classification layer based on the second label and the second prediction node, determining the value of the third loss function corresponding to the first mapping layer based on the first label and the first mapping node, and determining the value of the fourth loss function corresponding to the second mapping layer based on the second label and the second mapping node; obtaining the loss function of the test question classification model constructed by the first loss function, the second loss function, the third loss function and the fourth loss function; determining the value of the loss function of the test question classification model based on the value of the first loss function, the value of the second loss function, the value of the third loss function, and the value of the fourth loss function; and updating the model parameters of the test question classification model based on the value of the loss function of the test question classification model.

[0130] Continue to see Figure 5 The teaching material system includes multiple unit nodes of the first node level, each unit node includes at least two unit sub-nodes of the second node level, and each unit sub-node includes at least one knowledge point of the third node level.

[0131] As an example, when the first node is a unit subnode and the second node is a knowledge point, the first prediction node is a prediction unit subnode, and the second prediction node is a prediction knowledge point, the first prediction node is mapped through the mapping layer based on the association relationship between nodes in the teaching material system to obtain a first mapping node corresponding to the first prediction node, and the second prediction node is mapped to obtain a second mapping node corresponding to the second prediction node. The process specifically includes mapping the prediction unit subnode through the mapping layer based on the association relationship between the unit subnode and the knowledge point in the teaching material system to obtain a mapped knowledge point corresponding to the prediction unit subnode, and mapping the predicted knowledge point to obtain a mapped unit subnode corresponding to the predicted knowledge point. Then, the test question classification model is updated by combining the predicted unit subnode, the predicted knowledge point, the mapped unit subnode, the mapped knowledge point, the first label, and the second label.

[0132] In actual implementation, when the first node is a unit subnode and the second node is a knowledge point, the trained test question classification model can be used to simultaneously determine the subunit and knowledge point to which the corresponding test question belongs. Specifically, the test text of the test question to be classified is obtained; through the first classification layer, the test text of the test question to be classified is classified and predicted to obtain the subunit to which the test text of the test question to be classified belongs, or, through the second classification layer, the test text of the test question to be classified is classified and predicted to obtain the knowledge point to which the test text of the test question to be classified belongs, and through the mapping layer, based on the association relationship between nodes in the teaching material system, the knowledge points are mapped to obtain the subunit to which the test text of the test question to be classified belongs; through the second classification layer, the test text of the test question to be classified is classified and predicted to obtain the knowledge point to which the test text of the test question to be classified belongs, or, through the first classification layer, the test text of the test question to be classified is classified and predicted to obtain the subunit to which the test text of the test question to be classified belongs, and through the mapping layer, based on the association relationship between nodes in the teaching material system, the subunits are mapped to obtain the knowledge point to which the test text of the test question to be classified belongs.

[0133] Next, we will continue to introduce the training method of the test question classification model provided in the embodiment of the present application. Figure 12 This is a flow chart of the training method of the test question classification model provided in the embodiment of the present application, see Figure 12 The training method of the test question classification model provided in the embodiment of the present application is implemented collaboratively by the client and the server.

[0134] In step 201 , the client obtains the first test question sample carrying the first label and the second test question sample carrying the second label in response to an upload operation on the first test question sample and the second test question sample having the same test question text.

[0135] In actual implementation, the client can be a test question classification client set up in the terminal. The first test question sample and the second test question sample can be triggered by the user based on the human-computer interaction interface of the client, so that the upload function item in the human-computer interaction interface is presented to the client on the human-computer interaction interface. The user uploads the first test question sample and the second test question sample from the terminal locally based on the test question sample selection interface, so that the client obtains the uploaded first test question sample and the second test question sample.

[0136] In step 202 , the client sends a first test question sample carrying a first label and a second test question sample carrying a second label to the server.

[0137] In step 203 , the server inputs the received first test question sample and the second test question sample into a test question classification model.

[0138] Step 204: output a first prediction node for the first test question sample, a second prediction node for the second test question sample, and a mapping node corresponding to the second prediction node.

[0139] Step 205 , combining the first label, the second label, the first prediction node, the second prediction node and the mapping node to update the model parameters of the test question classification model.

[0140] In actual implementation, the server iterates the above training process until the loss function converges, completing the training of the test question classification model.

[0141] Step 206: The server generates a prompt message indicating that the training of the test question classification model is completed.

[0142] Step 207: The server sends a prompt message to the client.

[0143] Step 208 : The client obtains the text of the test question to be classified in response to the upload operation on the test question to be classified.

[0144] In actual implementation, the client can present a classification interface of the test questions to be classified, which includes a question stem input box, an answer input box, a solution input box, and a prediction result box. Specifically, the test questions to be classified can be obtained by photographing a camera connected to the terminal. After photographing the test questions to be classified, the camera transmits the test questions to be classified to the terminal and the terminal automatically uploads them to the client, so that the client can identify the test questions to be classified based on the photographed test questions, thereby presenting the question stem, answer, and solution corresponding to the answer in the corresponding input boxes; or the client responds to the trigger operation received for the classification interface including the function item of obtaining the test questions to be classified, and obtains them from the question bank; or the test questions to be classified can be the user inputting the corresponding information based on the input boxes on the classification interface to upload the test questions to be classified to the client. It should be noted that during the uploading process of the test questions to be classified, the classification interface can also present classification range selection items such as elementary school mathematics, elementary school Chinese, junior high school mathematics, etc., so that the server can classify the test questions to be classified based on the corresponding classification range, thereby improving the efficiency of test question classification.

[0145] For example, see Figure 13 , Figure 13 This is a schematic diagram of the classification interface of the test questions to be classified provided in the embodiment of the present application, based on Figure 13 The question text of the test questions to be classified corresponds to the question stem "0.625 tons = ? kilograms, 0.3 square meters = ? square decimeters, 3.16 kilometers = ? kilometers? meters", and the answer is "625, 30, 3, 160", and the corresponding analysis of the answer is "Test analysis: (1) It is the unit conversion of mass, from the higher unit ton to the lower unit kilogram, multiplying by 1000. (2) It is the unit conversion of area, from the higher unit square meter to the lower unit square decimeter, multiplying by 100. (3) It is the unit conversion of length, from a single number to a complex number, "Kilometers are regarded as the sum of kilometers (i.e. 3 kilometers) and kilometers, and kilometers are multiplied by the rate of 1000 to become 160 meters, and then written together with kilometers (i.e. 3 kilometers). Solution: (1) 0.625 tons = 625 kilograms; (2) 0.3 square meters = 30 square decimeters; (3) 3.16 kilometers = 3 kilometers = 160 meters, so the answers are 625, 30, 3, and 160. Comment: To convert units, you must first clarify whether you are converting from a higher-level unit to a lower-level unit or vice versa. Secondly, remember the conversion rate between units; converting from a higher-level unit to a lower-level unit multiplies the conversion rate, and converting from a lower-level unit to a higher-level unit divides the conversion rate. The category is "Elementary School Mathematics," and the function item for obtaining the question to be classified is the "Next Question" function item on this interface.

[0146] It should be noted that before the classification result of the test question to be classified is determined, the prediction result box in the classification interface is blank. After the classification result of the test question to be classified is determined, the prediction result box presents the corresponding prediction result, that is, the classification result of the test question to be classified.

[0147] In step 209 , the client sends the text of the test question to be classified to the server in response to the classification instruction for the test question to be classified.

[0148] In actual implementation, the classification instructions for the test questions to be classified can be automatically generated by the client under certain trigger conditions. For example, after the client obtains the test questions to be classified, it automatically generates classification instructions for the test questions to be classified. It can also be sent to the client by other devices connected to the terminal for communication. It can also be generated by the user based on the client's human-computer interaction interface after triggering the corresponding submission function item.

[0149] In step 210 , the server inputs the received test question text of the test question to be classified into the test question classification model, so that the test question classification model classifies the test question and obtains the target node to which the test question text of the test question to be classified belongs.

[0150] Step 211 : Based on the target node, the test questions to be classified are labeled to obtain labeled test questions.

[0151] Step 212: Send the corresponding annotation to the client.

[0152] Step 213: The client outputs the received annotation.

[0153] In actual implementation, the client can present the classification results for the test questions to be classified in the human-computer interaction interface of the client, save the classification results locally to the terminal, and send the classification results to other devices connected to the terminal.

[0154] Continuing with the above example, see Figure 14 , Figure 14 This is a schematic diagram of the classification interface of the test questions to be classified provided in the embodiment of the present application, based on Figure 14 The classification interface presents the classification results of the corresponding test questions to be classified based on the prediction result box, namely "Beijing Normal University Edition, Grade 3, Volume 2 - Four kilograms, grams, tons - How heavy is it" and "Beijing Normal University Edition, Grade 3, Volume 2 - Four kilograms, grams, tons - How heavy is 1 ton".

[0155] Applying the above embodiment of the present application, by classifying and predicting the first test question sample and the second test question sample with the same test question text but with different labels, respectively, a first prediction node and a second prediction node are obtained, and then the second prediction node is mapped through the association relationship between nodes in the teaching material system to obtain a mapping node at the same node level as the first prediction node. Finally, the test question classification model is updated based on the first prediction node, the second prediction node and the mapping node, as well as the corresponding labels. In this way, by combining the mapping nodes obtained based on the association relationship between nodes in the teaching material system during the training process to update the test question classification model, not only the training efficiency of the test question classification model can be improved, but also the classification accuracy of the test question classification model can be improved.

[0156] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0157] In response to the Ministry of Education's policy that student assignments must not exceed the syllabus, it is necessary to check whether the chapters corresponding to the assignments assigned by teachers are beyond the syllabus. Furthermore, chapter prediction is widely used in educational products such as question resource search, question recommendation, question bank construction, and personalized learning. Therefore, correctly identifying the chapters of test questions in this scenario is particularly important. However, there is a situation in the question bank where the annotated data for the test chapters is relatively small. Relying solely on the annotated chapter data from teaching and research would be extremely time-consuming and laborious. At the same time, for test questions, there is a certain relationship between chapters and knowledge points. If this relationship can be effectively utilized, the accuracy and efficiency of chapter annotation can be improved.

[0158] Based on this, this application proposes a test question chapter marking method that integrates knowledge point chapter relationship information, that is, using the knowledge point chapter correspondence compiled by teaching and research teachers, a test question classification model that integrates the relationship between chapters and knowledge points is constructed.

[0159] In actual implementation, first obtain the training test questions and the chapter data and knowledge point data corresponding to the training test questions, then splice the question stems, answers and the corresponding analyses of the answers of the training test questions to obtain the chapter test question text (first test question sample) carrying the test question chapter data (first label), and the knowledge point test question text (second test question sample) carrying the test question knowledge point data (second label), and input the chapter test question text into the chapter BERT model (first encoding layer), and input the knowledge point test question text into the knowledge point BERT model (second encoding layer), so as to obtain the text identifier H = {h cls ,h1,h2,……,h n}, and the text identifier H={h cls ,h1,h2,……,h n}, where H represents the final output of each word, and the BERT model parts of the test questions and knowledge points share parameters; then, based on formula (1), the word vectors in H are added and averaged to obtain the text representation of the chapter test question text (first encoding vector) and the text representation of the knowledge point test question text (second encoding vector); then, the text representation of the chapter test question text is input into a chapter task layer (first prediction layer), and the predicted chapter (first prediction node) is obtained based on formula (3); the text representation of the knowledge point test question text is input into a knowledge point task layer (second prediction layer), and the predicted knowledge point (second prediction node) is obtained based on formula (3); then the predicted knowledge point is input into a knowledge point chapter association layer (mapping layer), and based on the association relationship between the chapter and the knowledge point, the mapped chapter (mapping node) is obtained based on formula (4). Finally, after determining the predicted chapter, predicted knowledge point and mapped chapter, based on the first label and the predicted chapter, determine the value loss1 of the first loss function corresponding to the chapter task layer, based on the second label and the predicted knowledge point, determine the value loss2 of the second loss function corresponding to the knowledge point task layer, and based on the first label and the mapping node, determine the value loss3 of the third loss function corresponding to the knowledge point chapter association layer, and then based on loss1, loss2, and loss3, determine the value of the loss function of the test question classification model Finally, the value of the loss function based on the test question classification model Update the model parameters of the test question classification model. For example, the Adam algorithm can be used to optimize the parameters of each layer of the model, with the learning rate set to 0.0000125 and the batch_size set to 6. Once the test question classification model is trained, save the model. This model can be used to predict the chapters to be tested.

[0160] It should be noted that the BERT model is selected as the common layer in the above process. In addition, the traditional CNN model and LSTM model can also be selected as the common layer.

[0161] In actual implementation, see Figure 15 , Figure 15 This is a schematic diagram of the effect comparison provided by the embodiment of the present application, based on Figure 15 The first row of the table shows the test chapter marking based only on the BERT model, and the second row shows the test chapter marking based on the test question classification model of this application. Here, by comparing the first five columns of data, we can clearly see that the test question classification model of this application greatly improves the accuracy of test question chapter marking.

[0162] Applying the above embodiment of the present application, by classifying and predicting the first test question sample and the second test question sample with the same test question text but with different labels, respectively, a first prediction node and a second prediction node are obtained, and then the second prediction node is mapped through the association relationship between nodes in the teaching material system to obtain a mapping node at the same node level as the first prediction node. Finally, the test question classification model is updated based on the first prediction node, the second prediction node and the mapping node, as well as the corresponding labels. In this way, by combining the mapping nodes obtained based on the association relationship between nodes in the teaching material system during the training process to update the test question classification model, not only the training efficiency of the test question classification model can be improved, but also the classification accuracy of the test question classification model can be improved.

[0163] The following continues to describe the exemplary structure of the training device 455 of the test question classification model provided in the embodiment of the present application as a software module, wherein the test question classification model includes: a first classification layer, a second classification layer and a mapping layer. In some embodiments, such as Figure 2 As shown, the software modules stored in the training device 455 of the test question classification model in the memory 440 may include:

[0164] Acquisition module 4551 is configured to acquire a first test question sample and a second test question sample having the same test question text, wherein the first test question sample carries a first tag, and the second test question sample carries a second tag; wherein the first tag is used to indicate, in a textbook system including multiple content nodes, a first node to which the test question text belongs, and the second tag is used to indicate a second node to which the test question text belongs, and the first node and the second node are at different node levels in the textbook system;

[0165] The classification module 4552 is configured to perform classification prediction based on the first test question sample using the first classification layer to obtain a first prediction node to which the test question text belongs, and to perform classification prediction based on the second test question sample using the second classification layer to obtain a second prediction node to which the test question text belongs;

[0166] A mapping module 4553 is configured to map the second prediction node based on the association relationship between nodes in the teaching material system to obtain a mapping node corresponding to the second prediction node, wherein the mapping node and the first prediction node are at the same node level;

[0167] The updating module 4554 is used to update the model parameters of the test question classification model by combining the first label, the second label, the first prediction node, the second prediction node and the mapping node.

[0168] In some embodiments, the first classification layer includes a first encoding layer and a first prediction layer, and the second classification layer includes a second encoding layer and a second prediction layer; wherein, the first encoding layer and the second encoding layer share model parameters; the classification module 4552 is also used to perform vector encoding on the first test question sample through the first encoding layer to obtain a first encoding vector, and perform classification prediction based on the first encoding vector through the first prediction layer to obtain a first prediction node to which the test question text belongs; perform vector encoding on the second test question sample through the second encoding layer to obtain a second encoding vector, and perform classification prediction based on the second encoding vector through the second prediction layer to obtain a second prediction node to which the test question text belongs.

[0169] In some embodiments, the classification module 4552 is further used to perform word segmentation processing on the first test question sample through the first encoding layer to obtain multiple sample words; encode each of the sample words separately to obtain the word vector corresponding to each of the sample words; and perform vector averaging on the word vectors corresponding to each of the sample words to obtain the first encoding vector.

[0170] In some embodiments, the classification module 4552 is also used to extract keywords from the first test question sample through the first coding layer to obtain multiple keywords; encode each of the keywords separately to obtain a keyword vector corresponding to each of the keywords; obtain the weight corresponding to each of the keywords, and perform weighted summation on the keyword vectors corresponding to each of the keywords based on the weight to obtain the first coding vector.

[0171] In some embodiments, the acquisition module 4551 is also used to obtain the stem, answer and analytical content corresponding to the answer of the target test question when the test question text corresponds to the target test question; splice the stem, answer and analytical content corresponding to the answer of the target test question to obtain the test question text; and perform labeling based on the test question text to obtain the first test question sample carrying the first label and the second test question sample carrying the second label.

[0172] In some embodiments, the update module 4554 is also used to determine the value of the first loss function corresponding to the first classification layer based on the first label and the first prediction node, determine the value of the second loss function corresponding to the second classification layer based on the second label and the second prediction node, and determine the value of the third loss function corresponding to the mapping layer based on the first label and the mapping node; obtain the loss function of the test question classification model constructed by the first loss function, the second loss function and the third loss function; determine the value of the loss function of the test question classification model based on the value of the first loss function, the value of the second loss function, and the value of the third loss function; and update the model parameters of the test question classification model based on the value of the loss function of the test question classification model.

[0173] In some embodiments, the device also includes an application module, which is used to obtain the test text of the test question to be classified; through the first classification layer, the test text of the test question to be classified is classified and predicted to obtain the target node to which the test text of the test question to be classified belongs; or, through the second classification layer, the test text of the test question to be classified is classified and predicted to obtain the node to be mapped to which the test text of the test question to be classified belongs, and through the mapping layer, based on the association relationship between the nodes in the teaching material system, the node to be mapped is mapped to obtain the target node to which the test text of the test question to be classified belongs.

[0174] In some embodiments, the application module is also used to label the test questions to be classified based on the target node to obtain labeled test questions; obtain the test question outline, and match the labels of the labeled test questions with the content in the test question outline to obtain a matching result; when the matching result indicates that the labels do not match the content in the test question outline, determine that the labeled test questions are out-of-scope test questions.

[0175] In some embodiments, the device further includes an association module, which is configured to obtain a node association relationship table corresponding to the teaching material system; and update the association relationships between nodes in the teaching material system based on the node association relationship table.

[0176] In some embodiments, the teaching material system includes multiple unit nodes, each of the unit nodes includes at least two unit sub-nodes, and each of the unit sub-nodes includes at least one knowledge point; when the first node is the unit sub-node and the second node is the knowledge point, the first prediction node is the predicted unit sub-node and the second prediction node is the predicted knowledge point; the mapping module 4553 is also used to map the predicted knowledge point through the mapping layer based on the association relationship between the unit sub-nodes and the knowledge points in the teaching material system, and obtain the unit sub-node corresponding to the predicted knowledge point as the mapping node.

[0177] The present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the training method for the test question classification model described in the present application.

[0178] The embodiment of the present application provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the training method of the test question classification model provided in the embodiment of the present application, for example, Figure 3 The training method of the test question classification model is shown.

[0179] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or various devices including one or any combination of the above memories.

[0180] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0181] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0182] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0183] In summary, the following technical effects can be achieved through the embodiments of the present application:

[0184] (1) By combining the mapping nodes obtained based on the association relationship between nodes in the textbook system during the training process to update the test question classification model, not only the training efficiency of the test question classification model can be improved, but also the classification accuracy of the test question classification model can be improved.

[0185] (2) By sharing model parameters between the first coding layer and the second coding layer, the amount of calculation required to update the model parameters is reduced during the training of the test question classification model, thereby reducing the consumption of computing resources and accelerating the model convergence speed.

[0186] (3) By ensuring that the association relationship between nodes is updated in a timely manner, the model is trained in combination with the updated association relationship between nodes during the training process of the test question classification model, thereby improving the accuracy of the test question classification model.

[0187] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A training method for a test question classification model, characterized in that: The test question classification model includes: a first classification layer, a second classification layer and a mapping layer, and the method includes: Obtain a first test question sample and a second test question sample having the same test question text, wherein the first test question sample carries a first label, and the second test question sample carries a second label; The first tag is used to indicate, in a teaching material system including a plurality of content nodes, the first node to which the test question text belongs, and the second tag is used to indicate the second node to which the test question text belongs, and the first node and the second node are at different node levels in the teaching material system; Performing classification prediction based on the first test question sample through the first classification layer to obtain a first prediction node to which the test question text belongs, and performing classification prediction based on the second test question sample through the second classification layer to obtain a second prediction node to which the test question text belongs; Mapping the second prediction node based on the association relationship between nodes in the teaching material system through the mapping layer to obtain a mapping node corresponding to the second prediction node, wherein the mapping node and the first prediction node are at the same node level; In combination with the first label, the second label, the first prediction node, the second prediction node and the mapping node, the model parameters of the test question classification model are updated.

2. The method according to claim 1, wherein The first classification layer includes a first coding layer and a first prediction layer, and the second classification layer includes a second coding layer and a second prediction layer; wherein the first coding layer and the second coding layer share model parameters; Performing classification prediction based on the first test question sample through the first classification layer to obtain a first prediction node to which the test question text belongs includes: Performing vector encoding on the first test question sample through the first encoding layer to obtain a first encoding vector, and performing classification prediction based on the first encoding vector through the first prediction layer to obtain a first prediction node to which the test question text belongs; The step of performing classification prediction based on the second test question sample through the second classification layer to obtain a second prediction node to which the test question text belongs includes: The second test question sample is vector-encoded through the second encoding layer to obtain a second encoding vector, and classification prediction is performed based on the second encoding vector through the second prediction layer to obtain a second prediction node to which the test question text belongs.

3. The method according to claim 2, wherein The step of performing vector encoding on the first test question sample through the first encoding layer to obtain a first encoding vector includes: Performing word segmentation processing on the first test question sample through the first encoding layer to obtain a plurality of sample words; Encode each of the sample words respectively to obtain a word vector corresponding to each of the sample words; Vector averaging is performed on the word vectors corresponding to the sample words to obtain the first encoding vector.

4. The method according to claim 2, wherein The step of performing vector encoding on the first test question sample through the first encoding layer to obtain a first encoding vector includes: Performing keyword extraction on the first test question sample through the first coding layer to obtain a plurality of keywords; Encode each of the keywords respectively to obtain a keyword vector corresponding to each of the keywords; The weight corresponding to each of the keywords is obtained, and the keyword vectors corresponding to each of the keywords are weighted summed based on the weight to obtain the first encoding vector.

5. The method according to claim 1, wherein The obtaining of the first test question sample and the second test question sample having the same test question text includes: When the test question text corresponds to a target test question, obtaining the stem, answer, and analysis content corresponding to the answer of the target test question; The question stem, answer, and analysis content corresponding to the answer of the target test question are spliced ​​together to obtain the test question text; Labeling is performed based on the test question text to obtain the first test question sample carrying the first label and the second test question sample carrying the second label.

6. The method according to claim 1, wherein The updating of the model parameters of the test question classification model by combining the first label, the second label, the first prediction node, the second prediction node, and the mapping node includes: Determine a value of a first loss function corresponding to the first classification layer based on the first label and the first prediction node, determine a value of a second loss function corresponding to the second classification layer based on the second label and the second prediction node, and determine a value of a third loss function corresponding to the mapping layer based on the first label and the mapping node; Obtaining a loss function of the test question classification model constructed by the first loss function, the second loss function, and the third loss function; Determining a value of the loss function of the question classification model based on the value of the first loss function, the value of the second loss function, and the value of the third loss function; Based on the value of the loss function of the test question classification model, the model parameters of the test question classification model are updated.

7. The method according to claim 1, wherein The method further comprises: Get the test question text of the test question to be classified; By using the first classification layer, the test text of the test question to be classified is classified and predicted to obtain the target node to which the test text of the test question to be classified belongs; or Through the second classification layer, the test text of the test question to be classified is classified and predicted to obtain the node to be mapped to which the test text of the test question to be classified belongs, and through the mapping layer, based on the association relationship between the nodes in the teaching material system, the node to be mapped is mapped to obtain the target node to which the test text of the test question to be classified belongs.

8. The method according to claim 7, wherein The method further comprises: Based on the target node, the test questions to be classified are labeled to obtain labeled test questions; Obtaining a test question outline, and matching the annotations of the annotated test question with the content in the test question outline to obtain a matching result; When the matching result indicates that the annotation does not match the content in the test question outline, it is determined that the annotated test question is an out-of-syllabus test question.

9. The method according to claim 1, wherein The method further comprises: Obtaining a node association relationship table corresponding to the teaching material system; The association relationship between nodes in the teaching material system is updated based on the node association relationship table.

10. The method according to claim 1, wherein The teaching material system includes a plurality of unit nodes, each of the unit nodes includes at least two unit sub-nodes, and each of the unit sub-nodes includes at least one knowledge point; When the first node is the unit subnode and the second node is the knowledge point, the first prediction node is a prediction unit subnode and the second prediction node is a prediction knowledge point; The mapping layer maps the second prediction node based on the association relationship between nodes in the teaching material system to obtain a mapping node corresponding to the second prediction node, including: Through the mapping layer, based on the association relationship between the unit sub-nodes and the knowledge points in the teaching material system, the predicted knowledge points are mapped to obtain the unit sub-nodes corresponding to the predicted knowledge points as the mapping nodes.

11. A training device for a test question classification model, characterized in that: The test question classification model includes: a first classification layer, a second classification layer and a mapping layer, and the device includes: an acquisition module, configured to acquire a first test question sample and a second test question sample having the same test question text, wherein the first test question sample carries a first label, and the second test question sample carries a second label; wherein the first label is used to indicate, in a teaching material system including a plurality of content nodes, a first node to which the test question text belongs, and the second label is used to indicate a second node to which the test question text belongs, and the first node and the second node are at different node levels in the teaching material system; a classification module configured to perform classification prediction based on the first test question sample through the first classification layer to obtain a first prediction node to which the test question text belongs, and to perform classification prediction based on the second test question sample through the second classification layer to obtain a second prediction node to which the test question text belongs; a mapping module, configured to map the second prediction node based on the association relationship between nodes in the teaching material system to obtain a mapping node corresponding to the second prediction node, wherein the mapping node and the first prediction node are at the same node level; An updating module is used to update the model parameters of the test question classification model by combining the first label, the second label, the first prediction node, the second prediction node and the mapping node.

12. An electronic device, characterized in that: include: a memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 10 when executing the executable instructions stored in the memory.

13. A computer-readable storage medium, characterized in that Executable instructions are stored, which are used to cause a processor to execute and implement the method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images