Method and device for modifying test questions, equipment and storage medium

Abstract

The invention provides a test question modifying method and device, equipment and a storage medium, and relates to the technical field of computers. According to the method and the device, the change of the knowledge content can be automatically detected, the associated test questions can be automatically determined according to the knowledge content, and then the test questions are modified based on the large language model. The whole process is automatically processed by a computer, the test questions do not need to be manually modified, the modification efficiency can be improved, the situation of test question modification errors or missed modification caused by carelessness is avoided, and then the accuracy of test question modification is improved.

Description

[0001] This disclosure claims priority to Chinese patent application number 202410178059.3, filed on February 8, 2024, entitled “A test question updating method, system, device and storage medium,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for modifying test questions. Background Art

[0003] With the popularization and development of computer technology, the education industry has achieved informatization, and cloud education platforms have emerged. Cloud education platforms integrate various educational resources into the cloud, allowing users to access and use educational resources anytime and anywhere through the Internet. Educational resources include a question bank, from which users can obtain various questions based on their needs. In the process of maintaining the question bank, when certain knowledge content changes, the questions associated with this knowledge content in the question bank need to be modified. For example, a certain piece of knowledge is that Pluto is listed as one of the nine planets in the solar system. However, with the development of astronomical exploration, scientists downgraded Pluto to a dwarf planet in the solar system, reducing the number of planets in the solar system from nine to eight. At this time, the questions associated with this knowledge content in the question bank need to be modified.

[0004] Generally, when a certain knowledge content changes, the test questions related to the knowledge content are first screened out from the test question bank, and then these test questions are manually modified one by one.

[0005] Modifying test questions in this way is very inefficient, and manual modification of test questions may result in modification errors or omission of modification of a test question. Summary of the Invention

[0006] The present disclosure provides a method, apparatus, device and storage medium for modifying test questions, which improve the efficiency of test question modification and avoid test question modification errors or omissions due to carelessness, thereby improving the accuracy of test question modification.

[0007] In a first aspect, a method for modifying test questions is provided, which is applied to a cloud education platform. The method includes: first detecting that a first knowledge content is changed to a second knowledge content, and obtaining test questions associated with the first knowledge content in a test question bank, and then adding the first knowledge content, the second knowledge content and the test questions to a text template to obtain text description information, and finally processing the text description information through a large language model to obtain a modified test question, and replacing the test questions in the test question bank with the modified test question.

[0008] The text description information is used to indicate modification of the test question so that the test question is changed from being adapted to the first knowledge content to being adapted to the second knowledge content.

[0009] This allows automatic detection of changes in knowledge content, automatic identification of related test questions based on the knowledge content, and then modification of the test questions based on the large language model. This entire process is handled automatically by a computer, eliminating the need for manual modification. This improves modification efficiency and avoids careless modification errors or omissions, thereby increasing the accuracy of test modification.

[0010] In one possible implementation, the first knowledge content is crawled from the historical web page data of the target URL, and the second knowledge content with the same title as the first knowledge content is crawled from the current web page data of the target URL. Then, the first knowledge content and the second knowledge content are compared. If the two are different, it is determined that the first knowledge content is changed to the second knowledge content.

[0011] The comparison can be done using either plain text matching or semantic matching. These two methods can also be combined: first using plain text matching. If the text is identical, the match is successful. If the text is different, semantic matching is used. If the semantics are identical, the match is successful, but if they are different, the match fails. Semantic matching can be implemented using machine learning models.

[0012] In this way, changes in knowledge content can be automatically detected, so that test questions with changed knowledge points can be modified in a timely manner.

[0013] In a possible implementation, a knowledge change notification is received, where the knowledge change notification is used to indicate that first knowledge content is changed to second knowledge content.

[0014] Knowledge change notifications can be manually input by relevant technical personnel.

[0015] This way, the way to detect knowledge changes is more flexible.

[0016] In one possible implementation, a knowledge tag corresponding to each test question is stored in the test question bank. First, among the sub-contents included in the first knowledge content, key sub-contents whose similarity with the knowledge information elements in the knowledge graph is greater than a first threshold are determined. Then, in the test question bank, test questions whose corresponding knowledge tags have a similarity with the key sub-content greater than a second threshold are determined as test questions associated with the first knowledge content. The knowledge graph includes multiple nodes and edges between different nodes, and each node and edge corresponds to a knowledge information element.

[0017] First, the target key sub-content is determined in the key sub-content. The target key sub-content is the knowledge information element whose similarity with it is greater than the threshold value. The knowledge information element corresponding to the edge is the knowledge information element. If the knowledge label corresponding to a test question only has a similarity with the target key sub-content greater than the threshold value, then the test question is removed from the test questions associated with the first knowledge content, that is, the test question is considered not to be associated with the first knowledge content.

[0018] In this way, the test questions associated with the first knowledge content are determined to be more accurate.

[0019] In a possible implementation, in the question database, questions whose similarity to the first knowledge content is greater than a third threshold are determined as questions associated with the first knowledge content.

[0020] In this way, in some extreme cases, when the key sub-content cannot be determined from the first knowledge content through the knowledge graph, the test questions associated with the first knowledge content can also be determined, making the method more applicable.

[0021] In a possible implementation, a text template is obtained based on the type of the test question.

[0022] In this way, different test question types correspond to different text templates, which can more accurately guide the large language model to modify the test questions and improve the accuracy of test question modification.

[0023] In one possible implementation, the first knowledge content, the second knowledge content, the test questions and the test question modification examples are added to a text template to obtain text description information, wherein the test question modification examples include the third knowledge content, the fourth knowledge content, the example test questions and the modified example test questions, the example test questions are adapted to the third knowledge content, and the modified example test questions are adapted to the fourth knowledge content.

[0024] In this way, the accuracy of the results output by the large language model can be improved, and the examples can provide a reference for the output form of the large language model.

[0025] In a possible implementation, a test question modification example is obtained based on the type of the test question.

[0026] In this way, different test question types correspond to different test question modification examples, which can more accurately guide the large language model to modify the test questions and improve the accuracy of test question modification.

[0027] In one possible implementation, the first knowledge content, the second knowledge content, the test questions and the thought chain are added to a text template to obtain text description information, wherein the thought chain is used to indicate the steps of processing the text description information to the large language model.

[0028] This can help the large language model decompose a problem into several sub-problems, and then the large language model solves the sub-problems one by one, strengthening the detail processing ability of the large language model, so that the output results of the large language model are obtained by step-by-step reasoning, thereby improving the accuracy of the results output by the large language model.

[0029] In a possible implementation, a thought chain is obtained based on the type of the test question.

[0030] In this way, different test question types correspond to different thinking chains, which can more accurately guide the large language model to modify the test questions and improve the accuracy of test question modification.

[0031] In a second aspect, a device for modifying test questions is provided, which includes at least one module, and the at least one module is used to implement the method provided by the above-mentioned first aspect and its possible implementation methods.

[0032] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the method provided by the above-mentioned first aspect and its possible implementation methods.

[0033] In a fourth aspect, a computer device is provided, comprising a memory and a processor, wherein the memory is configured to store computer instructions, and the processor executes the computer instructions stored in the memory to cause the computer device to perform the method provided in the first aspect and possible implementations thereof.

[0034] In a fifth aspect, a computer-readable storage medium is provided, which includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method provided by the above-mentioned first aspect and its possible implementation methods.

[0035] In a sixth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the method provided by the first aspect and its possible implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a schematic diagram of a knowledge graph provided by an embodiment of the present disclosure;

[0037] Figure 2 This is a flow chart of a method for modifying test questions provided by an embodiment of the present disclosure;

[0038] Figure 3 This is a flow chart of a method for detecting changes in knowledge content provided by an embodiment of the present disclosure;

[0039] Figure 4 is a schematic diagram of detecting changes in knowledge content provided by an embodiment of the present disclosure;

[0040] Figure 5 This is a flow chart of a method for determining test questions provided by an embodiment of the present disclosure;

[0041] Figure 6 is a schematic diagram of determining a test question provided by an embodiment of the present disclosure;

[0042] Figure 7 is a schematic diagram of determining a test question provided by an embodiment of the present disclosure;

[0043] Figure 8 is a schematic diagram of a template library provided by an embodiment of the present disclosure;

[0044] Figure 9 is a schematic diagram of a text template provided by an embodiment of the present disclosure;

[0045] Figure 10 is a schematic diagram of an example library provided by an embodiment of the present disclosure;

[0046] Figure 11 is a schematic diagram of a text template provided by an embodiment of the present disclosure;

[0047] Figure 12 is a schematic diagram of a thought chain library provided by an embodiment of the present disclosure;

[0048] Figure 13 is a schematic diagram of a test question modification provided by an embodiment of the present disclosure;

[0049] Figure 14 is a schematic diagram of a web page provided by an embodiment of the present disclosure;

[0050] Figure 15 is a schematic diagram of a change detection method provided by an embodiment of the present disclosure;

[0051] Figure 16 1 is a schematic structural diagram of a device for modifying test questions provided by an embodiment of the present disclosure;

[0052] Figure 17 is a schematic diagram of a computing device provided by an embodiment of the present disclosure;

[0053] Figure 18 is a schematic diagram of a computing device cluster provided by an embodiment of the present disclosure;

[0054] Figure 19 This is a schematic diagram of a computing device cluster provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0056] The concepts involved in this disclosure are explained below:

[0057] Large language model (LLM)

[0058] A large language model is a machine learning model designed to understand and generate human language. It is trained using large amounts of text data to learn the complex structures and grammar of human language. When using a large language model, human language is input, and the model interprets the semantics based on the human language and performs corresponding operations. This feature makes large language models applicable in a wide range of fields, such as machine translation, question-answering, and text generation.

[0059] Text Template

[0060] A text template is a paragraph or multiple paragraphs of text with some missing content. By adding the missing content to the text template, a complete text description is generated. This text description is fed into the large language model to guide it in generating specific, clear text information. For example, a text description might read, "Please write a simple cooking recipe based on the following ingredients: chicken breast, vegetables (such as carrots, broccoli, or onions), salt, pepper, and olive oil."

[0061] In the disclosed embodiment, the text description information is used to instruct the large language model to modify the test questions based on the change of knowledge content. The blank content in the text template may include the knowledge content before and after the change and the test questions to be modified.

[0062] Example

[0063] Examples are used to provide reference information for the large language model to guide its operation. Examples can be placed in text templates. An example can be simply understood as an example of input and output when using the large language model. For example, if a large language model is used for Chinese-English translation, an example would be the Chinese-English translation. Entering examples into the large language model can improve the accuracy of its output and provide a reference for the output format of the large language model.

[0064] Thought Chain

[0065] Thought chains are used to guide the large language model's operations step by step. These chains can also be placed in text templates. These chains can include multiple steps, spelled out in text. These steps represent the internal flow of information processing within the large language model, specifically the steps involved in converting input data into output data. Incorporating thought chains into the large language model helps it decompose a problem into several sub-problems, which it then solves one by one. This strengthens the model's ability to process details, ensuring that its output is derived step by step through reasoning, thereby improving the accuracy of its output.

[0066] Knowledge Graph

[0067] A knowledge graph is a type of graph data used to visually record and store various knowledge contents. A knowledge graph consists of a large number of nodes and edges. Nodes represent entities, which can be various things. Due to the wide variety of knowledge contents, entities can also be words, sentences, and so on. Edges can be directed, representing connections between entities, such as friendships between people or supply relationships between products and suppliers. Entities and relationships are collectively referred to as knowledge information elements. In a knowledge graph, "entity-relationship-entity" can form a triple, which is used to represent a relatively complete piece of knowledge content.

[0068] The following is an example of knowledge graph. Figure 1 The following is a knowledge graph with a relatively simple structure. The recorded knowledge content is that Zhang San was born in City A, his occupation is a basketball player, his nationality is Country B, his wife is Li Si, and Li Si's birthplace is also City A. Among them, "Zhang San", "basketball player", "Country B", "City A" and "Li Si" are entities, and "occupation", "wife", "nationality" and "birthplace" are relationships.

[0069] Knowledge Tags

[0070] Knowledge tags are used to mark the knowledge content contained in the test questions. Knowledge tags and test questions are recorded in the test question bank in correspondence, and test questions associated with certain knowledge content can be quickly locked based on knowledge tags. A test question can correspond to one knowledge tag or multiple knowledge tags. There are many ways to add knowledge tags to test questions, and the embodiments of the present disclosure do not limit this. For example, knowledge tags can be added based on the knowledge graph. The specific method is to compare the words or sentences in the test question with the various knowledge information elements in the knowledge graph. If they can be matched, the knowledge information element is added as the knowledge tag of the test question. For example, knowledge tags can be added manually, and so on.

[0071] The method for modifying test questions provided in the embodiments of the present disclosure can be implemented by a computer device, which can be a server of a cloud education platform. The cloud education platform is designed for the education industry and uses cloud services to integrate various educational resources into the cloud, allowing users to access and use educational resources anytime and anywhere through the Internet.

[0072] A question bank is run and maintained in the cloud education platform. When a certain knowledge content changes, the questions in the question bank that include the knowledge content need to be modified. The disclosed embodiment provides a method for modifying questions, which can automatically detect the changed knowledge content and then automatically modify the corresponding questions using a large language model, thereby improving the efficiency of question modification and avoiding the situation where the question modification is wrong or omitted due to carelessness, thereby improving the accuracy of question modification. The processing flow of this method is as follows: Figure 2 As shown, the following steps are included:

[0073] Step 201: Detect that the first knowledge content is changed to the second knowledge content.

[0074] The first and second knowledge contents describe the same knowledge point, and the second knowledge content is obtained by modifying the first knowledge content. For example, the solar system originally had nine planets, namely Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, Neptune, and Pluto. With the advancement of science and technology, scientists have deepened their exploration of celestial bodies and downgraded Pluto to a dwarf planet, removing it from the ranks of the nine planets in the solar system. The number of planets in the solar system has decreased from nine to eight. In this modification, the first knowledge content can be "There are nine planets in the solar system, namely Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, Neptune, and Pluto." The second knowledge content can be "There are eight planets in the solar system, namely Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, and Neptune, and Pluto has been downgraded to a dwarf planet."

[0075] There are many ways to detect changes in knowledge content. For example, you can periodically obtain knowledge content from a web page, and then compare the knowledge content obtained from the current web page with the knowledge content obtained from historical web pages to determine the changed knowledge content. The specific processing flow is as follows: Figure 3 As shown, it will be described in detail later. Optionally, the relevant technical personnel can also enter the knowledge change notification, such as Figure 4 As shown, the knowledge change notification is used to indicate that the first knowledge content is changed to the second knowledge content.

[0076] Step 202: Obtain test questions associated with the first knowledge content from the test question bank.

[0077] The question bank stores a large number of test questions, each with a corresponding knowledge tag. Technicians can pre-set the question storage format. For example, each question in the question bank can be stored in the format of "Question Type: ..., Knowledge Tag: ..., Question Stem: ..., Options: ..., Answer: ...."

[0078] There are many possible implementations for this step, such as obtaining test questions associated with the first knowledge content based on the knowledge graph and knowledge tags. Another example is matching the first knowledge content with the test questions to obtain test questions associated with the first knowledge content. Each of these is explained below:

[0079] Method 1: Based on the knowledge graph and the knowledge labels of the test questions, the test questions associated with the first knowledge content can be determined. The processing flow is as follows: Figure 5 As shown, the following steps are included:

[0080] Step 2021: Determine, among the sub-contents included in the first knowledge content, key sub-contents whose similarity with the knowledge information elements in the knowledge graph is greater than a first threshold.

[0081] First, multiple sub-contents can be extracted from the first knowledge content. The extraction method can be: first divide the first knowledge content into multiple basic units, which can be words or characters, and then use the basic units to combine to obtain sub-contents. A single basic unit can be used as a sub-content, and any number of consecutive basic units can be combined into sub-contents. There can be overlap between different sub-contents. For example, a certain knowledge content "cat is an animal" is divided into basic units including "cat", "is" and "animal", and the determined sub-contents may include "cat", "is", "animal", "cat is", "is animal", and "cat is animal".

[0082] Feature vectors are extracted for each sub-content and knowledge information element respectively. For each sub-content, the similarity between the feature vectors of the sub-content and each knowledge information element is calculated, and the maximum similarity obtained is used as the similarity corresponding to the sub-content. Then, among all the sub-contents, the sub-contents whose corresponding similarity is less than the first threshold are filtered out. The first threshold can be set by a technician based on experience. Optionally, for each sub-content remaining after the screening process, if multiple sub-contents overlap, the sub-content with the highest corresponding similarity can be retained among these multiple sub-contents, and the remaining sub-contents can be filtered out. The sub-contents that remain are determined to be the key sub-contents of the first knowledge content.

[0083] The following describes a method for calculating the similarity of feature vectors. For example, the vector cosine of two feature vectors can be determined as the similarity between the two. The calculation formula is:

[0084]

[0085] Among them, X and Y represent the feature vectors of two texts, θ represents the angle between the vectors, and cos(θ) represents the cosine value of the angle between vector X and vector Y. The value range is [-1, 1]. The larger the value, the higher the similarity of the vectors.

[0086] For another example, the Euclidean distance between two feature vectors can be determined as their similarity, and the calculation formula is:

[0087]

[0088] Where X and Y represent the feature vectors of two texts, and d(X,Y) represents the Euclidean distance between vector X and vector Y. The smaller the value, the higher the similarity of the vectors.

[0089] Optionally, there are many other methods for determining the similarity between vectors, such as Mahalanobis distance, Langmuir distance, correlation coefficient, etc., which will not be described in detail here.

[0090] Step 2022: In the question database, determine the questions whose corresponding knowledge tags have similarities with the key sub-content greater than a second threshold value, and use them as questions associated with the first knowledge content.

[0091] In the question bank, for each question, if the similarity between a knowledge tag and a key sub-content among the knowledge tags corresponding to the question is greater than a second threshold, the question is determined to be associated with the first knowledge content. The second threshold can be set by technical personnel based on experience and can be equal to or different from the first threshold.

[0092] The following is an example to illustrate the above steps 2021-2022. Figure 6As shown, the first knowledge content includes sub-content "A," "B," "C," and "D," etc. Each sub-content is compared with each knowledge information element in the knowledge graph. It is found that the similarity between "A" and "A'" is greater than a threshold, the similarity between "B" and "B'" is greater than a threshold, the similarity between "C" and "C'" is greater than a threshold, and the similarity between "D" and "D'" is greater than a threshold. Therefore, "A," "B," "C," and "D" are determined to be key sub-content. The key sub-content is then compared with the knowledge labels of each question in the question bank. The similarity between the key sub-contents "A", "B", "C" and "D" and the knowledge labels "A", "B", "C" and "D" respectively is greater than the threshold. The knowledge label corresponding to Question 1 is "C", the knowledge label corresponding to Question 2 includes "A", "B", the knowledge label corresponding to Question 4 includes "D", the knowledge label corresponding to Question 5 is "A", "B", "C" and "D", and the knowledge label corresponding to Question 6 includes "A", so Questions 1, 2, 4, 5 and 6 are determined to be questions associated with the first knowledge content. There is no knowledge label corresponding to Questions 3 and 7 whose similarity with the key sub-content is greater than the threshold, so Questions 3 and 7 are not questions associated with the first knowledge content.

[0093] The key sub-content may be a word that includes a descriptive relationship. For example, if a key sub-content is "application scenario", as long as a knowledge tag with a similarity to "application scenario" is greater than a threshold value in a test question, the test question will be determined as a test question associated with the first knowledge content. This may result in a large number of test questions in the determined test questions being irrelevant to the first knowledge content. Therefore, the following method can be used to screen test questions associated with the first knowledge content. First, determine the target key sub-content in the key sub-content. The target key sub-content is the knowledge information element corresponding to the "edge" with a similarity greater than a threshold value. If the knowledge tag corresponding to a test question only has a similarity greater than the threshold value with the target key sub-content, but has a similarity less than the threshold value with other key sub-contents, then the test question will be removed from the test questions associated with the first knowledge content, that is, the test question is considered not to be associated with the first knowledge content.

[0094] Method 2: The first knowledge content can be compared with each test question in the test question bank. When the similarity between the first knowledge content and a test question is greater than a threshold, the test question is determined to be a test question associated with the first knowledge content. Figure 7 As shown, the first knowledge content is compared with each test question, and it is found that the similarity between the first knowledge content and test question 1, test question 2, test question 4, test question 5 and test question 6 is greater than the threshold, and test question 1, test question 2, test question 4, test question 5 and test question 6 are determined to be test questions associated with the first knowledge content.

[0095] Optionally, the applicable scenario of this method may be: in some extreme cases, it is impossible to determine the key sub-content from the first knowledge content through the knowledge graph.

[0096] Step 203: Add the first knowledge content, the second knowledge content and the test questions to the text template to obtain text description information.

[0097] The text description information is used to indicate the modification of the test question so that the test question is changed from being adapted to the first knowledge content to being adapted to the second knowledge content.

[0098] The following is an example of a text template. The text template can be as follows:

[0099]

[0100] Add the first knowledge content, the second knowledge content, and the test questions associated with the first knowledge content to the text template. Add the first knowledge content after "Old knowledge point:", the second knowledge content after "New knowledge point:", and the test questions associated with the first knowledge content in curly brackets after "Test questions to be modified:". After adding the above information to the text template, you will get the text description information. The text description information can be in the following format:

[0101]

[0102] Optionally, different text templates can be set based on different question types, and the text templates and question types can be stored in the template library accordingly, such as Figure 8 As shown, the template library contains text templates corresponding to various question types, including single-choice questions, multiple-choice questions, true-or-false questions, fill-in-the-blank questions, short-answer questions, etc. Enter the type of question in the template library to obtain the corresponding text template.

[0103] In addition to the above-mentioned content added to the text template, you can also add test question modification examples to the text template, such as Figure 9 As shown, the test question modification example includes the third knowledge content (i.e., the old knowledge point in the example), the fourth knowledge content (i.e., the new knowledge point in the example), the example test question and the modified example test question. The example test question is adapted to the third knowledge content, and the modified example test question is adapted to the fourth knowledge content. The fourth knowledge content is obtained by changing the third knowledge content. The corresponding text description information can be as follows:

[0104]

[0105]

[0106] Optionally, different question modification examples can be set based on different question types, and the question modification examples and question types can be stored in the example library accordingly, such as Figure 10 As shown, the sample library contains test question modification examples for various question types, including single-choice questions, multiple-choice questions, true / false questions, fill-in-the-blank questions, short-answer questions, etc. Enter the test question type in the sample library to obtain the corresponding test question modification examples.

[0107] In addition to adding the first knowledge content, the second knowledge content and the test questions to be modified in the template, you can also add thinking chains in the text template, such as Figure 11 As shown, the thought chain may include multiple steps, namely: the first step is to instruct the large language model to compare the first knowledge content and the second knowledge content to determine the changed content; the second step is to instruct the large language model to modify the test question to be modified based on the changed content; the third step is to instruct the large language model to output the modified test question. The corresponding text description information may be as follows:

[0108]

[0109] In the above example, the content following "Please follow the steps below:" constitutes a thought chain. This example not only indicates the output of the modified question, but also limits the format of the output question, so that the output modified question is the same as the question to be modified, arranged according to question type, knowledge label, question stem, options, and answer. This relatively organized information allows for easier modification of the question. Optionally, the third step of the thought chain can also include "listing the knowledge changes" to allow users to intuitively understand the knowledge changes.

[0110] Optionally, different thinking chains can be set based on different question types, and the thinking chains and question types can be stored in the thinking chain library accordingly, such as Figure 12 As shown, the Thinking Chain Library contains thinking chains corresponding to various question types, including single-choice questions, multiple-choice questions, true-or-false questions, fill-in-the-blank questions, short-answer questions, etc. Enter the question type in the example library to obtain the corresponding question modification examples.

[0111] Optionally, you can add the above-mentioned test question modification examples and thought chains to the template. The corresponding text description information is as follows:

[0112]

[0113]

[0114] Step 204: Process the text description information based on the large language model to obtain a modified test question.

[0115] The large language model processes the text template with various information added to it, modifies the test question to be modified, and outputs the modified test question. The following example illustrates the output of the large language model. The output format can be as follows:

[0116]

[0117] If the text template includes a thought chain, and the thought chain includes "listing the content of knowledge changes", the output of the large language model can be as follows:

[0118]

[0119] Step 205: Replace the test questions in the test question bank with the modified test questions.

[0120] In the question bank, replace the question before modification with the question after modification.

[0121] For ease of understanding, Figure 13 This is a schematic diagram of steps 201 to 205.

[0122] like Figure 3 The following steps are shown for detecting changes in knowledge content from a target web page:

[0123] Step 2011: crawl the first knowledge content from the historical web page data of the target website.

[0124] Step 2012: crawl the current webpage data of the target URL to obtain the second knowledge content having the same title as the first knowledge content.

[0125] The target URL is the URL corresponding to the web page that records the knowledge content. The target URL selected by the technician can be a URL with a certain authority to ensure the reliability of the knowledge content crawled from the web page corresponding to the URL.

[0126] A time period can be pre-set to periodically crawl the webpage data of a target URL. The time period can be set based on the update cycle of the webpage data corresponding to the target URL. For example, if a URL updates its data once a month, the time period can be set to one month. In addition to the above method, technicians can also set the time for crawling webpages independently without relying on a time period, and the embodiments of the present disclosure are not limited to this.

[0127] During the crawling process, we first start by acquiring web page data from the first-level page of the target URL. In addition to web page data, the first-level page may also contain links. In this case, we can further access the page corresponding to the link (also called the second-level page) to obtain the web page data of that page. If there are also links on that page, we will further access the pages corresponding to the links until there are no lower-level pages. At this point, all the web page data corresponding to the target URL has been obtained.

[0128] After obtaining all the web page data of the target website, the web page data can be preliminarily processed to remove other content that is not related to the knowledge content. Then the knowledge content can be listed according to the title. The following example illustrates: Figure 14 As shown in the figure, the knowledge content recorded on a webpage of a certain URL is an introduction to various numbers in elementary school mathematics. The text in the dotted box is the title. The knowledge content recorded on the webpage can be structured as follows:

[0129]

[0130] It can be seen that there are multiple levels of data in the structured web page data. "Introduction to Numbers in Elementary School Mathematics" is the first level, "(I) Integers", "(II) Decimals" and "(III) Fractions" are the second level, and so on. Except for the lowest level, the data at other levels can be considered as titles of the corresponding levels. For example, "Introduction to Numbers in Elementary School Mathematics" is the title of the first level, and "(I) Integers" is the title of the second level. The data at the lowest level includes the title of that level and the corresponding knowledge content, for example, "1. The meaning of integers": "Natural numbers and 0 are both integers.". The structured web page data also reflects the relationship between the data at each level. For example, "(I) Integers" includes "1. The meaning of integers", "2. How to read integers", etc.

[0131] In the process of structuring, if there are links in the above web pages, the text content on the links can be used as the parent title of the web page data in the links.

[0132] Based on structured web page data, we can determine the correspondence between titles and knowledge content. Titles can be a combination of multiple levels, in the form of "first-level title - second-level title - ... - lowest-level title." For example, the title is "Introduction to Numbers in Elementary Mathematics - (I) Integers - 1. The Meaning of Integers," and the corresponding knowledge content is "Natural numbers and 0 are both integers."

[0133] When the time period is reached, the above-mentioned processing method is used to crawl the webpage data in the target URL, obtain the knowledge content corresponding to each title, and record it. At the same time, the knowledge content corresponding to each title recorded in the previous time period can be obtained. For each title, the corresponding knowledge content recorded in the previous time period (i.e., the first knowledge content) and the knowledge content recorded in the current time period (i.e., the second knowledge content) can be obtained for subsequent processing steps.

[0134] Step 2013 : comparing the first knowledge content and the second knowledge content. If the two are different, determining that the first knowledge content is changed to the second knowledge content.

[0135] Compare the first and second knowledge contents of all titles. If they are different, determine that the first knowledge content has been changed to the second knowledge content. Using the knowledge content of step 2011 as an example, compare the first and second knowledge contents corresponding to the titles "1. Meaning of Integers," "2. Pronunciation of Integers," "3. Writing of Integers," "4. Comparison of Integer Sizes," and so on.

[0136] The comparison can be done using either plain text matching or semantic matching. These two methods can also be combined: first using plain text matching. If the text is identical, the match is successful. If the text is different, semantic matching is used. If the semantics are identical, the match is successful, but if they are different, the match fails. Semantic matching can be implemented using machine learning models.

[0137] The following is an example to illustrate the above steps 2011-2013. Figure 15 As shown, the historical webpage data includes the first knowledge content corresponding to a certain title: "Application scenarios of content distribution networks include "website acceleration," "file download acceleration," and "on-demand acceleration." The current webpage data also includes the second knowledge content corresponding to the same title: "Application scenarios of content distribution networks include "website acceleration," "file download acceleration," "on-demand acceleration," and "full site acceleration." The first and second knowledge contents are compared to determine that the first knowledge content has been changed to the second knowledge content.

[0138] Based on the same technical concept, the embodiment of the present disclosure provides a device for modifying test questions, which can be applied to the above-mentioned cloud education platform. Figure 16 As shown, the device includes:

[0139] The detection module 1610 is used to detect that the first knowledge content has been changed to the second knowledge content, and can specifically implement the processing function of the above step 201 and other implicit steps.

[0140] The acquisition module 1620 is used to acquire the test questions associated with the first knowledge content from the test question bank, and can specifically implement the processing function of the above step 202 and other implicit steps.

[0141] Adding module 1630 is configured to add the first knowledge content, the second knowledge content, and the test question to the text template to obtain text description information. The text description information is used to indicate the modification of the test question so that the test question is adapted from the first knowledge content to the second knowledge content. Specifically, this module implements the processing functions of step 203 above, as well as other implicit steps.

[0142] The processing module 1640 is used to process the text description information based on the large language model to obtain a modified test question. Specifically, it can implement the processing function of the above step 204 and other implicit steps.

[0143] The replacement module 1650 is used to replace the test questions in the test question bank with the modified test questions, and can specifically implement the processing function of the above step 205 and other implicit steps.

[0144] In one possible implementation, detection module 1610 is configured to crawl first knowledge content from the target URL's historical webpage data, crawl second knowledge content with the same title as the first knowledge content from the target URL's current webpage data, and then compare the first and second knowledge content. If the two are different, determining that the first knowledge content has been changed to the second knowledge content is a good start. Specifically, this can implement the processing functions of steps 2011-2013 above, as well as other implicit steps.

[0145] In a possible implementation, the detection module 1610 is configured to: receive a knowledge change notification, where the knowledge change notification is used to indicate that the first knowledge content is changed to the second knowledge content.

[0146] In one possible implementation, the knowledge label corresponding to each test question is stored in the test question bank. The acquisition module 1620 is used to: determine the key sub-contents included in the first knowledge content, whose similarity with the knowledge information elements in the knowledge graph is greater than a first threshold, and then determine the test questions in the test question bank, whose corresponding knowledge labels have a similarity with the key sub-content greater than a second threshold, as test questions associated with the first knowledge content. The knowledge graph includes multiple nodes and edges between different nodes, and each node and edge corresponds to a knowledge information element. Specifically, the processing functions of the above steps 2021-2022 and other implicit steps can be implemented.

[0147] In a possible implementation, the acquisition module 1620 is configured to: determine, in the question database, questions whose similarity to the first knowledge content is greater than a third threshold, as questions associated with the first knowledge content.

[0148] In a possible implementation, the acquisition module 1620 is further configured to acquire a text template based on the type of the test question.

[0149] In one possible implementation, adding module 1630 is used to: add the first knowledge content, the second knowledge content, the test questions and the test question modification examples to the text template to obtain text description information, wherein the test question modification examples include the third knowledge content, the fourth knowledge content, the example test questions and the modified example test questions, the example test questions are adapted to the third knowledge content, and the modified example test questions are adapted to the fourth knowledge content.

[0150] In a possible implementation, the acquisition module 1620 is further configured to acquire test question modification examples based on the type of the test question.

[0151] In one possible implementation, adding module 1630 is used to add the first knowledge content, the second knowledge content, the test questions and the thinking chain to the text template to obtain text description information, wherein the thinking chain is used to indicate the steps of processing the text description information to the large language model.

[0152] In a possible implementation, the acquisition module 1620 is further configured to acquire a thought chain based on the type of the test question.

[0153] In the disclosed embodiments, changes to knowledge content can be automatically detected, and related test questions can be automatically determined based on the knowledge content. These questions can then be modified based on the large language model. This entire process is handled automatically by a computer, eliminating the need for manual modification. This improves modification efficiency and avoids careless modification errors or omissions, thereby increasing the accuracy of test modification.

[0154] Detection module 1610, acquisition module 1620, addition module 1630, processing module 1640, and replacement module 1650 can all be implemented via software or hardware. For example, the following describes the implementation of detection module 1610, taking detection module 1610 as an example. Similarly, the implementation of acquisition module 1620, addition module 1630, processing module 1640, and replacement module 1650 can refer to the implementation of detection module 1610.

[0155] As an example of a software functional unit, the detection module 1610 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the detection module 1610 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0156] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0157] As an example of a hardware functional unit, detection module 1610 may include at least one computing device, such as a server. Alternatively, detection module 1610 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0158] The multiple computing devices included in detection module 1610 can be distributed in the same region or in different regions. They can also be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in detection module 1610 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0159] It should be noted that, in other embodiments, the detection module 1610, the acquisition module 1620, the adding module 1630, the processing module 1640 and the replacement module 1650 can be used to modify any step in the method of modifying test questions. The steps that the detection module 1610, the acquisition module 1620, the adding module 1630, the processing module 1640 and the replacement module 1650 are responsible for implementing can be specified as needed. The full functions of the device for modifying test questions can be realized by respectively implementing different steps in the method of modifying test questions through the detection module 1610, the acquisition module 1620, the adding module 1630, the processing module 1640 and the replacement module 1650.

[0160] The present disclosure also provides a computing device 100. Figure 17 As shown, computing device 100 includes a bus 102, a processor 104, a memory 106, and a communication interface 108. Processor 104, memory 106, and communication interface 108 communicate with each other via bus 102. Computing device 100 may be a server or a terminal device. It should be understood that this disclosure does not limit the number of processors and memories in computing device 100.

[0161] The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 17 The fact that only one line is used in the figure does not mean that there is only one bus or only one type of bus. Bus 102 may include a path for transmitting information between various components of computing device 100 (eg, memory 106, processor 104, communication interface 108).

[0162] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0163] The memory 106 may include volatile memory, such as random access memory (RAM). The memory 106 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0164] Memory 106 stores executable program code. Processor 104 executes the executable program code to implement the functions of the aforementioned detection module 1610, acquisition module 1620, addition module 1630, processing module 1640, and replacement module 1650, thereby implementing the method for modifying test questions. In other words, memory 106 stores instructions for the method for modifying test questions.

[0165] Alternatively, the memory 106 stores executable code, which the processor 104 executes to implement the functions of the aforementioned detection module 1610, acquisition module 1620, addition module 1630, processing module 1640, and replacement module 1650, thereby implementing the method for modifying a test question. In other words, the memory 106 stores instructions for the method for modifying a test question.

[0166] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0167] The present disclosure also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0168] like Figure 18As shown, the computing device cluster includes at least one computing device 100. The memory 106 of one or more computing devices 100 in the computing device cluster may store the same instructions for the method for modifying test questions.

[0169] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for the method for modifying test questions. In other words, the combination of one or more computing devices 100 can jointly execute the instructions for the method for modifying test questions.

[0170] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions, each for performing a portion of the functions of the apparatus for modifying test questions. In other words, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more of the aforementioned detection module 1610, acquisition module 1620, addition module 1630, processing module 1640, and replacement module 1650.

[0171] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network, which may be a wide area network (WAN) or a local area network (LAN), and which may be a Transmission Control Protocol (TCP) network or a Remote Direct Memory Access (RDMA) network. Figure 19 A possible implementation is shown. Figure 19 As shown, two computing devices 100A and 100B are connected via a network. Specifically, the connection to the network is achieved through a communication interface in each computing device. In this possible implementation, the memory 106 in computing device 100A stores instructions for executing the functions of detection module 1610 and acquisition module 1620. Simultaneously, the memory 106 in computing device 100B stores instructions for executing the functions of addition module 1630, processing module 1640, and replacement module 1650.

[0172] Figure 19 The connection method between the computing device clusters shown may be based on the consideration that the method for modifying test questions provided in the present disclosure requires a large amount of data storage, and therefore it is considered to entrust the functions implemented by the query module 1430 to the computing device 100B for execution.

[0173] It should be understood that Figure 19 The functions of the computing device 100A shown in FIG. 1 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0174] The embodiment of the present disclosure also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similarly referred to as Figure 18 and Figure 19 The connection mode of the computing device cluster is different in that the memory 106 of one or more computing devices 100 in the computing device cluster may store the same instructions for the method of modifying test questions.

[0175] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for the method for modifying test questions. In other words, the combination of one or more computing devices 100 can jointly execute the instructions for the method for modifying test questions.

[0176] The present disclosure also provides a computer program product containing instructions. The computer program product may be software or a program product containing instructions that can be run on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to modify test questions.

[0177] The embodiments of the present disclosure also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disk (DVD)), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to modify a test question, or instruct the computing device to execute a method for modifying a test question.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present disclosure.

Claims

1. A method for modifying test questions, characterized in that: The method is applied to a cloud education platform and includes: detecting that the first knowledge content is changed to the second knowledge content; Obtaining test questions associated with the first knowledge content from a test question bank; Adding the first knowledge content, the second knowledge content, and the test question to a text template to obtain text description information, wherein the text description information is used to indicate modification of the test question so that the test question is changed from being adapted for the first knowledge content to being adapted for the second knowledge content; Processing the text description information based on the large language model to obtain a modified test question; The test questions in the test question bank are replaced by the modified test questions.

2. The method according to claim 1, characterized in that The detecting that the first knowledge content is changed to the second knowledge content includes: Crawling the first knowledge content from the historical web page data of the target URL; crawling second knowledge content having the same title as the first knowledge content from the current webpage data of the target website; The first knowledge content and the second knowledge content are compared, and if the two are different, it is determined that the first knowledge content is changed to the second knowledge content.

3. The method according to claim 1, characterized in that The detecting that the first knowledge content is changed to the second knowledge content includes: A knowledge change notification is received, wherein the knowledge change notification is used to indicate that the first knowledge content is changed to the second knowledge content.

4. The method according to any one of claims 1 to 3, characterized in that The test question database stores knowledge tags corresponding to each test question; The step of obtaining, from the question bank, questions associated with the first knowledge content includes: Determining, from among the sub-contents included in the first knowledge content, key sub-contents having a similarity greater than a first threshold to a knowledge information element in a knowledge graph, wherein the knowledge graph includes a plurality of nodes and edges between different nodes, each node and edge corresponding to a knowledge information element; In the question database, questions whose corresponding knowledge tags have a similarity with the key sub-content greater than a second threshold are determined as questions associated with the first knowledge content.

5. The method according to any one of claims 1 to 3, characterized in that The step of obtaining, from the question bank, questions associated with the first knowledge content includes: In the question database, questions whose similarity to the first knowledge content is greater than a third threshold are determined as questions associated with the first knowledge content.

6. The method according to any one of claims 1 to 5, characterized in that Before adding the first knowledge content, the second knowledge content, and the test question to a text template to obtain text description information, the method further includes: Based on the type of the test question, the text template is obtained.

7. The method according to any one of claims 1 to 6, characterized in that The step of adding the first knowledge content, the second knowledge content, and the test question to a text template to obtain text description information includes: The first knowledge content, the second knowledge content, the test questions and the test question modification examples are added to a text template to obtain text description information, wherein the test question modification examples include the third knowledge content, the fourth knowledge content, the example test questions and the modified example test questions, the example test questions are adapted to the third knowledge content, and the modified example test questions are adapted to the fourth knowledge content.

8. The method according to claim 7, characterized in that Before adding the first knowledge content, the second knowledge content, the test questions, and the test question modification examples to the text template to obtain the text description information, the method further includes: Based on the type of the test question, obtain the test question modification example.

9. The method according to any one of claims 1 to 8, characterized in that The step of adding the first knowledge content, the second knowledge content, and the test question to a text template to obtain text description information includes: The first knowledge content, the second knowledge content, the test questions and the thought chain are added to a text template to obtain text description information, wherein the thought chain is used to instruct the large language model on the steps of processing the text description information.

10. The method according to claim 9, characterized in that Before adding the first knowledge content, the second knowledge content, the test questions, and the thought chain to the text template to obtain text description information, the method further includes: The thought chain is obtained based on the type of the test question.

11. A device for modifying test questions, characterized in that: The device is applied to a cloud education platform and includes: A detection module, configured to detect that the first knowledge content has been changed to the second knowledge content; An acquisition module, configured to acquire, from a question bank, questions associated with the first knowledge content; an adding module, configured to add the first knowledge content, the second knowledge content, and the test question to a text template to obtain text description information, wherein the text description information is used to indicate modification of the test question so that the test question is changed from being adapted for the first knowledge content to being adapted for the second knowledge content; A processing module, configured to process the text description information based on a large language model to obtain a modified test question; The replacement module is used to replace the test questions in the test question library with the modified test questions.

12. The device according to claim 11, characterized in that The detection module is used to: Crawling the first knowledge content from the historical web page data of the target URL; crawling second knowledge content having the same title as the first knowledge content from the current webpage data of the target website; The first knowledge content and the second knowledge content are compared, and if the two are different, it is determined that the first knowledge content is changed to the second knowledge content.

13. The device according to claim 11, characterized in that The detection module is used to: A knowledge change notification is received, wherein the knowledge change notification is used to indicate that the first knowledge content is changed to the second knowledge content.

14. The device according to any one of claims 11 to 13, characterized in that The test question database stores knowledge tags corresponding to each test question; The acquisition module is used to: Determining, from among the sub-contents included in the first knowledge content, key sub-contents having a similarity greater than a first threshold to a knowledge information element in a knowledge graph, wherein the knowledge graph includes a plurality of nodes and edges between different nodes, each node and edge corresponding to a knowledge information element; In the question database, questions whose corresponding knowledge tags have a similarity with the key sub-content greater than a second threshold are determined as questions associated with the first knowledge content.

15. The device according to any one of claims 11 to 13, characterized in that The acquisition module is used to: In the question database, questions whose similarity to the first knowledge content is greater than a third threshold are determined as questions associated with the first knowledge content.

16. The device according to any one of claims 11 to 15, characterized in that The acquisition module is further configured to acquire the text template based on the type of the test question.

17. The device according to any one of claims 11 to 16, characterized in that The adding module is used to: The first knowledge content, the second knowledge content, the test questions and the test question modification examples are added to a text template to obtain text description information, wherein the test question modification examples include the third knowledge content, the fourth knowledge content, the example test questions and the modified example test questions, the example test questions are adapted to the third knowledge content, and the modified example test questions are adapted to the fourth knowledge content.

18. The device according to claim 17, characterized in that The acquisition module is further configured to acquire the test question modification example based on the type of the test question.

19. The device according to any one of claims 11 to 18, characterized in that The adding module is used to: The first knowledge content, the second knowledge content, the test questions and the thought chain are added to a text template to obtain text description information, wherein the thought chain is used to instruct the large language model on the steps of processing the text description information.

20. The device according to claim 19, characterized in that The acquisition module is further configured to acquire the thought chain based on the type of the test question.

21. A computing device cluster, characterized in that: comprising at least one computing device, each computing device including a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in a memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that The method comprises computer program instructions, and when the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 10.

23. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster is caused to perform the method according to any one of claims 1 to 10.

Images