Question knowledge point label recommendation method and system based on BERT and collaborative filtering

Abstract

The present invention discloses a method and system for recommending knowledge point labels for questions based on BERT and collaborative filtering, comprising the following steps: splicing the question stem and options to form an original text, wherein the formula is directly spliced ​​in the LateX format, and the picture uses [IMG] as a placeholder; establishing an inverted index for the above-spliced ​​original text of the question as the first feature F1 of the question; using a pre-trained BERT model to semantically vectorize the original text of the question, etc. The advantage of the present invention over the prior art is that the present invention can significantly reduce the complexity of knowledge point label recommendation, integrate keywords and semantic similarity algorithms, and significantly improve the accuracy of similar question calculations. A collaborative filtering algorithm is used to obtain new knowledge point labels for the question, and as the question bank feature library is continuously enriched, the accuracy of the knowledge point labels can be further improved. The question bank feature library established in the present invention can also be used for scenarios such as keyword retrieval and similar question recommendation, and has a wider range of adaptability scenarios.

Description

Technical Field

[0001] The present invention relates to the field of topic retrieval and knowledge point tagging in educational informatization, and specifically refers to a topic knowledge point tag recommendation method and system based on BERT and collaborative filtering. Background Art

[0002] In the current era of comprehensive advancement of educational informatization, question bank construction and management have undoubtedly become key links in improving teaching quality and optimizing learning outcomes.

[0003] The role of the question bank is crucial. On the one hand, it provides teachers with rich teaching materials, helping them to flexibly select appropriate questions and carefully design personalized teaching plans and evaluation methods based on different teaching objectives and student levels. On the other hand, for students, a precisely classified and labeled question bank is like a treasure trove of knowledge. They can quickly retrieve matching exercises based on their own learning progress and specific needs, so that they can study in a targeted manner and significantly improve their learning efficiency.

[0004] However, to achieve efficient use of the question bank, the accuracy and consistency of the question knowledge point labels are indispensable prerequisites. Only by scientifically, reasonably and accurately labeling the knowledge points of the questions can teachers and students quickly find the required questions in the massive question bank. But the reality is not optimistic. With the in-depth development of educational informatization, the scale of the question bank is becoming increasingly large, and the content is becoming more and more complicated. The traditional method of simply relying on experts to label questions one by one has exposed many drawbacks. This method not only has a huge workload and consumes a lot of manpower and time costs, but also due to factors such as human subjectivity and fatigue, the consistency of the labeling is difficult to guarantee, and it is impossible to meet the actual needs of teaching in a timely manner. Therefore, it is urgent to develop an efficient, accurate and intelligent question knowledge point label recommendation method and system.

[0005] In recent years, the booming development of natural language processing (NLP) technology has provided new ideas and possibilities for solving this problem. Pre-trained language models represented by BERT have pioneered the introduction of a deep learning framework, and successfully achieved semantic vectorization by conducting all-round context modeling of text. BERT's unique advantage lies in its ability to deeply understand and accurately capture the deep semantic structure in sentences, and is by no means limited to the surface matching of vocabulary. This significant feature enables BERT to demonstrate greater flexibility and amazing accuracy when dealing with complex matching relationships between questions and knowledge points.

[0006] The semantic vectorization technology based on BERT can cleverly convert the questions into vector representations, and then perform similarity queries in high-dimensional space. By accurately calculating the distance between these vectors, the system can effectively identify a series of questions similar to a specific question.

[0007] In addition, keyword full-text search also plays an important role in title search. It can quickly locate the part containing specific keywords in a large amount of text, and realize preliminary title search by extracting and matching keywords in the title. Compared with other methods, it has outstanding advantages such as simple operation and fast response, so it can be used as the first step of range screening algorithm.

[0008] Collaborative filtering algorithm is a technology widely used in recommendation systems. This technology predicts the content that users may be interested in and recommends this content to users by analyzing the similarities ("collaboration") between things. With the help of collaborative filtering algorithm, the multi-label classification problem in NLP can be transformed into a collaborative filtering problem of "what knowledge point labels are similar questions given?"

[0009] By calculating the angle or distance between the semantic vectors of the questions, similar questions can be calculated more accurately and efficiently. Finally, by combining the similarity score, the location of the knowledge point label and other information for fusion scoring, a highly accurate automatic labeling algorithm can be constructed.

[0010] In the tide of educational informatization, constantly exploring and innovating methods and techniques for labeling question knowledge points has far-reaching significance and value for improving the quality of education and optimizing the allocation of teaching resources. Summary of the invention

[0011] The problem to be solved by the present invention is to improve the performance and accuracy of automatic labeling of question knowledge points by integrating keyword full-text retrieval and BERT semantic information and adopting collaborative filtering algorithm to score and screen knowledge point labels of similar questions.

[0012] In order to solve the above technical problems, the technical solution provided by the present invention is: a method for recommending topic knowledge point labels based on BERT and collaborative filtering, comprising the following steps:

[0013] S1. Combine the question stem and options to form the original text. Formulas are directly inserted in LateX format, and images use [IMG] as placeholders;

[0014] S2. Create an inverted index for the concatenated original title text as the first feature F1 of the title;

[0015] S3. Use the pre-trained BERT model to semantically vectorize the original text of the above question, and use the mean method to calculate the whole sentence semantics [CLS] as the second feature F2 of the question;

[0016] S4. In the above semantic vectorization process, when the original text of the question exceeds 512 characters in length, the original text is split into segments of 512 characters in length, the semantics of each segment is calculated [CLS], and then the average of all segment semantics is taken as the second feature F2 of the entire question;

[0017] S5, storing the original text of the question, feature F1, feature F2, and original features such as the subject, question type, and score obtained in the above steps as a complete feature in a column-based database;

[0018] S6. Perform feature processing on all questions in the question bank according to the above steps to form a question feature library;

[0019] S7. Add a new question and obtain the original text of the question according to the above steps T n ;

[0020] S8, the new title original text T obtained in the above steps n , use word segmentation, use the TextRank algorithm to extract the topic keywords, and remove stop words to get the topic keyword Kw n ;

[0021] S9. Use the title keyword Kw obtained in the above steps n , the subject features of the questions themselves are filtered in the question feature library to obtain the question set S1;

[0022] S10, calculate the problem feature F2 according to the above n Method to calculate the semantic vector feature F2 of the new question n ;

[0023] S11. In the topic set S1, calculate the semantic vector feature F2 of the new topic n The cosine distance is obtained to obtain the cosine similarity Scos q ; Use the formula 1-abs(Scos q ) Calculate the similarity score Score q , ensure Score q The value range is 1 to 0;

[0024] S12, sort the question set S1 in descending order of similarity score, and extract the similarity score Score q >0.5, we get the similar question set S2;

[0025] S13, take the similarity set S2, remove the questions without knowledge point labels, and take the topN (n=3) questions as the label recommendation candidate set S3;

[0026] S14. Calculate the knowledge point label score of each question in the candidate set S3: i is the knowledge point label position, count is the number of knowledge point labels in the question;

[0027] S15. Combine the knowledge point label score rates in each question obtained in the above steps to obtain the final knowledge point label recommendation score:

[0028] S16: The recommendation score S of each knowledge point label is tag Sort from high to low and take the top 5 as the final recommended knowledge point labels. Correspondingly, the method also includes the system carried by the method, that is, the question knowledge point label recommendation system based on BERT and collaborative filtering, including keyword extraction module, BERT semantic vectorization module, inverted index construction and retrieval module, similarity calculation module, and collaborative filtering module.

[0029] The advantages of the present invention over the prior art are: the present invention can significantly reduce the complexity of knowledge point label recommendation, integrate keywords and semantic similarity algorithms, and significantly improve the accuracy of similar question calculation. A collaborative filtering algorithm is used to obtain new question knowledge point labels, and as the question bank feature library is continuously enriched, the accuracy of knowledge point labels can be further improved. The question bank feature library established in the present invention can also be used for scenarios such as keyword retrieval, similar question (learning from one example) recommendation, etc., and has a wider range of adaptability scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the method flow of the patent of this invention.

[0031] Figure 2 It is a schematic diagram of the system structure of the patent of this invention. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0033] Embodiment 1

[0034] like Figure 1 and Figure 2 As shown, the collaborative filtering knowledge point tag recommendation method proposed by the present invention, which integrates full-text retrieval and BERT semantic similarity, first needs to establish a question bank feature library, and the establishment method is as follows:

[0035] (1) Concatenate the question stem and options into the original text. The formula is directly inserted in LateX format, and the picture uses [IMG] as a placeholder. Keeping the LateX original text of the formula and the picture placeholder can retain the semantic information of the question to the greatest extent during BERT vectorization;

[0036] (2) The original text of the question stem and options is segmented and stop words are removed as the first feature F1 of the question, and an inverted index is established. Removing stop words can reduce the interference of meaningless words. Inverted index is a common data structure used for information retrieval, mainly used to improve the efficiency of text queries. Unlike the traditional forward index (which maps documents to the words they contain), the inverted index maps words to a list of documents containing the word. With this structure, when a user searches for a keyword, the retrieval system only needs to find a list of document IDs containing the keyword, so that relevant documents can be quickly returned. This method significantly improves retrieval efficiency, especially when processing large amounts of text. Therefore, inverted indexes are widely used in search engines and various document retrieval systems.

[0037] (3) Use the pre-trained BERT model to calculate the semantic vector of the question as feature F2. Specifically, the original text of the question obtained by the above splicing is split into multiple parts in units of 512 characters, and the vector of each part is calculated using the BERT model respectively. The [CLS] representing the complete features of each part is averaged and used as the semantic feature of the entire question. In this embodiment, except for questions in individual subjects that contain large sections of background materials, there are very few questions that need to be split into parts. Even if the question is too long, the core semantics of the question can still be captured and retained by the method of splitting and averaging.

[0038] All characteristics of the questions are stored in the column-based database, including stem, analysis, subject, score, keyword characteristics F1, and semantic information F2. A column-based database is a database that uses columns as the main data storage method. Compared with traditional row-based databases, it has a high compression ratio and saves space. The application of SIMD (single instruction multiple data) technology can significantly improve query processing efficiency. SIMD allows the same instruction to be executed on multiple data points in the same clock cycle, thereby improving throughput when processing column-based data.

[0039] Embodiment 2

[0040] Based on the first embodiment, Figure 1 and Figure 2 As shown, the fusion method proposed in this embodiment includes the following steps when calculating similar questions of a new question:

[0041] (1) First, we use the same method to build the question feature library to obtain the original question text T n and semantic feature F2 n ; After word segmentation, keyword extraction, and removal of stop words, the TextRank algorithm is used to extract the TopK (k=5) topic keywords Kw n. TextRank is a graph-based text summarization and keyword extraction algorithm. It is inspired by the PαgeRank algorithm and is suitable for unsupervised tasks. It performs well in summarizing and extracting keywords from natural language texts. The extracted keywords can express the core content of the text.

[0042] (2) The keyword feature Kw obtained according to the above steps n , and obtain the topic similarity set S1 by screening in the topic feature library. This step is performed in a column-based database, which can greatly reduce the amount of data for subsequent semantic similarity calculations and improve query performance. And based on the preliminary matching of keywords, the accuracy of similarity matching can also be improved. In the experiment of this embodiment, topics with similar semantics but different contents can be eliminated, thereby improving the accuracy of similarity matching;

[0043] (3) Calculate the semantic feature F2 of the new question n The cosine distance of each question in the similarity set S1 is obtained to obtain the cosine similarity Scos q ; Use the formula 1-Scos q Calculate the similarity score q , ensure Score q The value range is 1 to 0. Cosine Similarity is a commonly used indicator to measure the similarity between two vectors, especially in the fields of text analysis and information retrieval. It determines their similarity by calculating the cosine value of the angle between two vectors. Specifically, this method can effectively determine the directional similarity of two vectors in high-dimensional space, while ignoring their magnitude.

[0044] (4) After sorting the above cosine similarity scores from large to small, extract the score q Questions with a score of >0.5 form set S2. q >0.5 can exclude questions with high semantic similarity and low content similarity;

[0045] (5) In the above set S2, after eliminating the questions without knowledge point tags, the TopN (n=3) questions are extracted as the tag recommendation candidate set S3.

[0046] Embodiment 3

[0047] Based on the second embodiment, Figure 1 and Figure 2 As shown, the collaborative filtering tag recommendation algorithm proposed in this embodiment includes the following steps

[0048] (1) Calculate the similarity score of each knowledge point label in each question. Formula: i is the position of the knowledge point label, and count is the number of knowledge point labels in the question; this method takes into account both the overall semantic similarity score of the question and the order of the knowledge point labels. For questions containing less than 10 knowledge point labels, the importance of each knowledge point label gradually decreases according to the position, and the importance of labels exceeding 10 decays rapidly. In the experiment of this embodiment, the number of knowledge point labels is usually around 3-5, and the proportion of questions with more than 10 knowledge point labels is relatively small;

[0049] (2) Calculate the similarity score of each knowledge point label. q Add together to get the label score for each knowledge point The maximum value of i is 3 because at most 3 question knowledge point labels are extracted in the above steps. In this embodiment, TopN (n = 3) is an ideal value. If it is too small, there will be no collaborative filtering effect. If it is too large, the calculation amount will be amplified and the final recommendation accuracy will be reduced.

[0050] (3) The recommendation score S of each knowledge point label is tag Sort from high to low and take the top 5 as the final recommended knowledge point labels.

[0051] The above shows and describes the basic principles and main features of the present invention and the advantages of the invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for recommending topic knowledge point labels based on BERT and collaborative filtering, characterized by The following steps are involved: S1. Combine the question stem and options to form the original text. Formulas are directly inserted in LateX format, and pictures use [IMG] as placeholders. S2. Create an inverted index for the concatenated original title text as the first feature F1 of the title; S3. Use the pre-trained BERT model to semantically vectorize the original text of the above question, and use the meαn method to calculate the whole sentence semantics [CLS] as the second feature F2 of the question; S4. In the above semantic vectorization process, when the original text of the question exceeds 512 characters in length, the original text is split into segments of 512 characters in length, the semantics of each segment is calculated [CLS], and then the average of all segment semantics is taken as the second feature F2 of the entire question; S5, storing the original text of the question, feature F1, feature F2, and original features such as the subject, question type, and score obtained in the above steps as a complete feature in a column-based database; S6, performing feature processing on all questions in the question bank according to the above steps to form a question feature library; S7. Add a new question and obtain the original text of the question according to the above steps T n ; S8, the new title original text T obtained in the above steps n , use word segmentation, use the TextRank algorithm to extract the topic keywords, and remove stop words to get the topic keyword Kw n ; S9. Use the title keyword Kw obtained in the above steps n , the subject features of the questions themselves are filtered in the question feature library to obtain the question set S1; S10, calculate the problem feature F2 according to the above n Method to calculate the semantic vector feature F2 of the new question n ; S11. In the topic set S1, calculate the semantic vector feature F2 of the new topic n The cosine distance is obtained to obtain the cosine similarity Scos q ; Use the formula 1-abs(Scos q ) Calculate the similarity score Score q , ensure Score q The value range is 1 to 0; S12, sort the question set S1 in descending order of similarity score, and extract the similarity score Score q >0.5, we get the similar question set S2; S13, take the similarity set S2, remove the questions without knowledge point labels, and take the topN (n=3) questions as the label recommendation candidate set S3; S14. Calculate the knowledge point label score of each question in the candidate set S3: i is the knowledge point label position, count is the number of knowledge point labels in the question; S15. Combine the knowledge point label score rates in each question obtained in the above steps to obtain the final knowledge point label recommendation score: S16: The recommendation score S of each knowledge point label is tag Sort from high to low and take the top 5 as the final recommended knowledge point labels.

2. A topic knowledge point label recommendation system based on BERT and collaborative filtering, characterized by It includes keyword extraction module, BERT semantic vectorization module, inverted index construction and retrieval module, similarity calculation module, and collaborative filtering module.

Images