A query expansion method and device for retrieval of programming technical issues

Abstract

The present application discloses a query expansion method and device for retrieval of programming technical problems, the method comprising: identifying all technical tags contained in a technical query problem, adding tags associated with the tags of the problem to a tag set according to technical tag association rules, searching for several target programming technical problems that are most similar to the syntax of the problem through a Lucene index, inputting the target problem into the distributed word vector of each word output by a word2vec model, and retrieving several programming technical problems that are most semantically similar to the technical query problem from several target problems based on an inverse document frequency table, and recommending them to the user. It can be seen that using technical tag association rules to expand a user's technical query problem can effectively solve the problem of missing technical information in the user's query, and combining the word2vec model and the Lucene index can help users quickly retrieve similar technical problems that are more in line with their needs.

Description

Technical Field

[0001] The present application relates to the field of technical problem retrieval, and more specifically, to a query expansion method and device for programming technical problem retrieval. Background Art

[0002] With the rapid development of the information industry, more and more technical personnel are engaged in software development. Programming Q&A websites, such as Stack Overflow (SO) and CSDN, serve as platforms for developers to communicate and share. After more than a decade of development, they have accumulated a massive amount of programming Q&A data. For example, SO has over 23 million questions and 33 million answers. These technical questions and answers form a vast resource library, becoming an important source for developers to solve programming tasks. Therefore, helping users retrieve programming technical questions that meet their needs has become a fundamental research task, and is a prerequisite for knowledge discovery and recommendation (such as Application Programming Interface (API) recommendations, code search, answer summaries, etc.) from programming Q&A resource libraries.

[0003] In recent years, researchers have proposed a number of programming question retrieval methods, which can be categorized into two main types. The first type, based on grammatical matching (including the Lucene search engine), measures the similarity between user queries and programming questions by matching keywords. However, due to the vocabulary gap, it is difficult to effectively retrieve programming questions with semantic similarity to the query. The second type, based on word / sentence / document embedding technology, measures the semantic relevance between words / sentences / documents by mapping them into a low-dimensional vector space, effectively addressing the vocabulary gap. However, this method requires high computational overhead.

[0004] How to efficiently retrieve programming technology with similar query semantics is an issue that needs attention. Summary of the Invention

[0005] In view of the above problems, the present application provides a query expansion method and device for retrieval of programming technical issues, so as to efficiently retrieve programming technical issues with similar semantics to the query.

[0006] In order to achieve the above objectives, the following specific plans are proposed:

[0007] A query expansion method for programming technical question retrieval, comprising:

[0008] Obtaining a user's technical query question and identifying all tags contained in the technical query question;

[0009] According to pre-established tag association rules, tags associated with tags included in the technical query question are added to the tag set of the technical query question;

[0010] According to the tag set of the technical query question, searching for several target programming technical questions that are most similar in syntax to the technical query question through a pre-established Lucene index;

[0011] Input each target programming technical problem into the pre-established word2vec model and output the distributed word vector of each word in the target programming technical problem;

[0012] Retrieving, from the plurality of target programming technical questions, a plurality of semantically similar programming technical questions that are most semantically similar to the technical query question based on a pre-established inverse document frequency table and distributed word vectors of each word of each target programming technical question;

[0013] Recommending semantically similar programming technical problems to the user.

[0014] Optionally, the process of establishing the tag association rule includes:

[0015] Filtering out each technical tag whose usage frequency exceeds a preset number of times from a preset programming technical tag set;

[0016] Sampling a plurality of target technical labels from the various technical labels;

[0017] For each target technology tag, perform part-of-speech analysis on the regular expression of the target technology tag using the Stanford Part-of-Speech (POS) tagger tool to extract the category phrase of the target technology tag;

[0018] Determine all technical tag categories of each target technical tag based on the category phrase of each target technical tag;

[0019] Among the target technical labels, a target technical label belonging to any one of the technical label categories is determined as a standard technical label;

[0020] According to the transaction set of all programming technical problems in the preset programming technical problem set, label association rules of various standard technical labels are established. Each programming technical problem includes a label set consisting of multiple labels, and the transaction set is the collection of all label sets of all programming technical problems.

[0021] Optionally, the process of establishing the Lucene index includes:

[0022] Identify all the tags that each programming technical question contains;

[0023] According to the tag association rule, tags associated with tags included in each programming technical question are added to the tag set of the programming technical question;

[0024] Preprocessing the title, body, and tags of each programming technical question to obtain the preprocessed programming technical question;

[0025] Build a document for the programming technical problem based on the title and tags of the programming technical problem;

[0026] Use the index module of the Lucene library to create a Lucene index for documents on various programming technical issues.

[0027] Optionally, the process of establishing the word2vec model includes:

[0028] The sentence segmentation module of the NLTK library is used to process the title and body of each preprocessed programming technical question to obtain a sentence set of the programming technical question;

[0029] The sentence sets of various programming technical problems are used as training samples to train the word2vec module of the Gensim library to obtain the word2vec model.

[0030] Optionally, the process of establishing the inverse document frequency table includes:

[0031] After all programming technical problems are pre-processed, a word set of all programming technical problems is determined;

[0032] Calculate the inverse document frequency of each word in the word set;

[0033] Based on the inverse document frequency of each word, an inverse document frequency table is constructed.

[0034] Optionally, the step of retrieving, from the target programming technical issues, a number of programming technical issues that are most semantically similar to the technical query issue based on a pre-established inverse document frequency table and the distributed word vectors of each word in each target programming technical issue, includes:

[0035] Based on a pre-established inverse document frequency table and the distributed word vectors of each word in each target programming technical question, calculate the asymmetric similarity of the word set of each target programming technical question to the word set of the technical query question, and the asymmetric similarity of the word set of the technical query question to the word set of the target programming technical question;

[0036] Calculate the semantic similarity between the word set of the target programming technical problem and the word set of the technical query question based on the asymmetric similarity between the word set of each target programming technical problem and the word set of the technical query question, and the asymmetric similarity between the word set of the technical query question and the word set of the target programming technical problem;

[0037] Determining the semantic similarity between the technical query question and the target programming technical question based on the semantic similarity between the word set of each target programming technical question and the word set of the technical query question;

[0038] A preset number of target programming technical problems with the highest semantic similarity are determined from among the target programming technical problems, as semantically similar programming technical problems.

[0039] Optionally, the step of calculating the asymmetric similarity of the word set of each target programming technical question to the word set of the technical query question based on a pre-established inverse document frequency table and the distributed word vectors of each word of each target programming technical question includes:

[0040] The asymmetric similarity of the word set of each target programming technical question to the word set of the technical query question is calculated using the following formula:

[0041]

[0042] Among them, T1 is the technical vocabulary contained in the target programming technical problem, Q1 is the non-technical vocabulary contained in the technical query problem, w is a word in the technical vocabulary T1, idf(w) is the inverse document frequency of word w in the pre-established inverse document frequency table, sim(w,Q1) is the maximum semantic similarity between word w and any word w' in the technical vocabulary Q1, and sim(w,Q1) is calculated by the cosine similarity between the distributed word vector of word w' and the distributed word vector of word w.

[0043] Optionally, the calculating of the semantic similarity between the word set of the target programming technical problem and the word set of the technical query question based on the asymmetric similarity of the word set of each target programming technical problem to the word set of the technical query question, and the asymmetric similarity of the word set of the technical query question to the word set of the target programming technical problem, includes:

[0044] The semantic similarity between the word set of each target programming technical question and the word set of the technical query question is calculated using the following formula:

[0045]

[0046] Among them, sim(T1,Q1) is the semantic similarity between technical vocabulary T1 and technical vocabulary Q1, sim(T1→Q1) is the asymmetric similarity of technical vocabulary T1 to technical vocabulary Q1, and sim(Q1→T1) is the asymmetric similarity of technical vocabulary Q1 to technical vocabulary T1.

[0047] Optionally, determining the semantic similarity between the technical query question and the target programming technical question based on the semantic similarity between the word set of each target programming technical question and the word set of the technical query question includes:

[0048] The semantic similarity between the technical query question and the target programming technical question is calculated using the following formula:

[0049] sim(T,Q)=η*sim(T1,Q1)+(1-η)*sim(T2,Q2)

[0050] Among them, T is the target programming technical problem, Q is the technical query problem, T2 is the non-technical word set contained in the target programming technical problem, Q2 is the non-technical word set contained in the technical query problem, η∈[0,1] is the semantic similarity weight factor, and sim(T2,Q2) is the semantic similarity between the non-technical word set T2 and the non-technical word set Q2.

[0051] A query expansion device for retrieval of programming technical issues, comprising:

[0052] A technical question tag identification unit is used to obtain the user's technical query question and identify all tags contained in the technical query question;

[0053] an associated tag supplement unit, configured to supplement tags associated with tags included in the technical query question into the tag set of the technical query question according to a pre-established tag association rule;

[0054] a target programming technical problem determining unit, configured to search, based on the tag set of the technical query problem, through a pre-established Lucene index for a number of target programming technical problems that are most similar in syntax to the technical query problem;

[0055] A word2vec model analysis unit is used to input each target programming technical problem into a pre-established word2vec model and output a distributed word vector for each word in the target programming technical problem;

[0056] a similar question determining unit, configured to retrieve, from the plurality of target programming technical questions, a plurality of semantically similar programming technical questions that are most semantically similar to the technical query question based on a pre-established inverse document frequency table and a distributed word vector of each word in each target programming technical question;

[0057] The recommendation unit is used to recommend semantically similar programming technical problems to the user.

[0058] Optionally, the device further includes:

[0059] A technical tag screening unit, configured to screen out technical tags whose usage frequency exceeds a preset number of times from a preset programming technical tag set;

[0060] a target technology label sampling unit, configured to sample a plurality of target technology labels from the various technology labels;

[0061] A category phrase extraction unit is used to perform part-of-speech analysis on the regular expression sentence of each target technical tag using the Stanford POS tagger tool to extract the category phrase of the target technical tag;

[0062] a technical label category determination unit, configured to determine all technical label categories of each target technical label based on the category phrase of each target technical label;

[0063] a standard technical label determination unit, configured to determine, among the target technical labels, a target technical label belonging to any one of the technical label categories as a standard technical label;

[0064] The tag association rule establishing unit is used to establish tag association rules for each standard technical tag based on the transaction set of all programming technical problems in the preset programming technical problem set. Each programming technical problem includes a tag set consisting of multiple tags, and the transaction set is the collection of all tag sets of all programming technical problems.

[0065] Optionally, the device further includes:

[0066] A tag identification unit, used to identify all tags contained in each programming technical problem;

[0067] a tag set supplementing unit, configured to supplement tags associated with tags included in each programming technical question into the tag set of the programming technical question according to the tag association rule;

[0068] A preprocessing unit, configured to preprocess the title, body, and tags of each programming technical question to obtain the preprocessed programming technical question;

[0069] A document construction unit, configured to construct a document of the programming technical problem according to the title and tags of the programming technical problem;

[0070] The Lucene index building unit is used to build a Lucene index for documents related to various programming technical issues using the index module of the Lucene library.

[0071] Optionally, the device further includes:

[0072] A sentence segmentation unit is used to process the title and body of each pre-processed programming technical question using the sentence segmentation module of the NLTK library to obtain a sentence set of the programming technical question;

[0073] The model training unit is used to train the word2vec module of the Gensim library using sentence sets of various programming technical problems as training samples to obtain a word2vec model.

[0074] Optionally, the device further includes:

[0075] A word set determination unit, configured to determine a word set for all programming technical problems after all programming technical problems have been pre-processed;

[0076] an inverse document frequency calculation unit, configured to calculate the inverse document frequency of each word in the word set;

[0077] The inverse document frequency table construction unit is used to construct an inverse document frequency table based on the inverse document frequency of each word.

[0078] Optionally, the similar problem determining unit includes:

[0079] an asymmetric similarity calculation unit, configured to calculate, based on a pre-established inverse document frequency table and the distributed word vectors of each word of each target programming technical problem, an asymmetric similarity of the word set of each target programming technical problem to the word set of the technical query problem, and an asymmetric similarity of the word set of the technical query problem to the word set of the target programming technical problem;

[0080] a word set semantic similarity calculation unit, configured to calculate the semantic similarity between the word set of each target programming technical problem and the word set of the technical query question based on the asymmetric similarity of the word set of each target programming technical problem to the word set of the technical query question, and the asymmetric similarity of the word set of the technical query question to the word set of the target programming technical problem;

[0081] a question semantic similarity calculation unit, configured to determine the semantic similarity between the technical query question and the target programming technical question based on the semantic similarity between the word set of each target programming technical question and the word set of the technical query question;

[0082] The semantically similar programming technical problem determining unit determines a preset number of target programming technical problems with the highest semantic similarity from the target programming technical problems, as the semantically similar programming technical problems.

[0083] Optionally, the asymmetric similarity calculation unit includes:

[0084] The asymmetric similarity calculation subunit is used to calculate the asymmetric similarity of the word set of each target programming technical question to the word set of the technical query question using the following formula:

[0085]

[0086] Among them, T1 is the technical vocabulary contained in the target programming technical problem, Q1 is the non-technical vocabulary contained in the technical query problem, w is a word in the technical vocabulary T1, idf(w) is the inverse document frequency of word w in the pre-established inverse document frequency table, sim(w,Q1) is the maximum semantic similarity between word w and any word w' in the technical vocabulary Q1, and sim(w,Q1) is calculated by the cosine similarity between the distributed word vector of word w' and the distributed word vector of word w.

[0087] Optionally, the word set semantic similarity calculation unit includes:

[0088] The word set semantic similarity calculation subunit is used to calculate the semantic similarity between the word set of each target programming technical question and the word set of the technical query question using the following formula:

[0089]

[0090] Among them, sim(T1,Q1) is the semantic similarity between technical vocabulary T1 and technical vocabulary Q1, sim(T1→Q1) is the asymmetric similarity of technical vocabulary T1 to technical vocabulary Q1, and sim(Q1→T1) is the asymmetric similarity of technical vocabulary Q1 to technical vocabulary T1.

[0091] Optionally, the word set semantic similarity calculation unit includes:

[0092] The word set semantic similarity calculation subunit is used to calculate the semantic similarity between the technical query question and the target programming technical question using the following formula:

[0093] sim(T,Q)=η*sim(T1,Q1)+(1-η)*sim(T2,Q2)

[0094] Among them, T is the target programming technical problem, Q is the technical query problem, T2 is the non-technical word set contained in the target programming technical problem, Q2 is the non-technical word set contained in the technical query problem, η∈[0,1] is the semantic similarity weight factor, and sim(T2,Q2) is the semantic similarity between the non-technical word set T2 and the non-technical word set Q2.

[0095] By means of the above technical solution, the present application obtains the user's technical query questions and identifies all the tags contained in the technical query questions. According to the pre-established tag association rules, the tags associated with the tags contained in the technical query questions are added to the tag set of the technical query questions. According to the tag set of the technical query questions, a pre-established Lucene index is used to find several target programming technical problems that are most similar in syntax to the technical query questions. Each target programming technical problem is input into a pre-established word2vec model, and the distributed word vector of each word of the target programming technical problem is output. Based on the pre-established inverse document frequency table and the distributed word vector of each word of each target programming technical problem, several semantically similar programming technical problems that are most similar in semantics to the technical query questions are retrieved from the several target programming technical problems, and each semantically similar programming technical problem is recommended to the user. It can be seen that the use of technical tag association rules to expand the user's technical query questions can effectively solve the problem of missing technical information in the user's query. Combined with the word2vec model and the Lucene index, it can help users quickly retrieve similar technical problems that are more in line with their needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0097] Figure 1 A flowchart of a query expansion method for searching programming technical issues provided in an embodiment of the present application;

[0098] Figure 2 A framework diagram of the offline and online parts of a query expansion method for programming technical question retrieval provided in an embodiment of the present application;

[0099] Figure 3A schematic diagram of the structure of a device for implementing a query expansion method for retrieval of programming technical issues provided in an embodiment of the present application. DETAILED DESCRIPTION

[0100] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0101] The present application solution can be implemented based on a terminal with data processing capabilities, which can be applied to a computer, server, cloud, etc.

[0102] Next, combine Figure 1 The query expansion method for programming technical problem retrieval of the present application can be implemented when the terminal is online. The method may include the following steps:

[0103] Step S110: Obtain the user's technical query question and identify all tags contained in the technical query question.

[0104] Step S120: According to pre-established tag association rules, tags associated with tags included in the technical query question are added to the tag set of the technical query question.

[0105] Specifically, steps S110-S120 are the query expansion phase. For a given user query, it can be expanded to supplement any missing technical information, thereby accurately retrieving similar programming technical questions. Query expansion includes two sub-steps: tag identification and tag expansion, and is implemented in the same way as question expansion.

[0106] In addition, the expanded query is preprocessed by removing stop words and rooting.

[0107] Step S130 : searching, based on the tag set of the technical query question, for several target programming technical questions that are most similar in syntax to the technical query question using a pre-established Lucene index.

[0108] Specifically, the Lucene index can be pre-built when the terminal is offline. Using the Lucene search engine, the top-N questions with similar syntax to the query can be retrieved.

[0109] It is understood that step S130 is the stage of searching for Top-N grammatically similar questions based on Lucene. The BM25 matching mechanism used by the Lucene search engine can efficiently process large-scale question repositories. The resulting Top-N grammatically similar questions can be used as the question set for the next step, "Searching for Top-k Semantically Similar Questions Based on Word Embeddings," to ensure the efficiency of semantic retrieval. To ensure that the Top-k semantically similar questions are covered by the Top-N grammatically similar questions, N can be set to a large value, such as 10,000.

[0110] Step S140: Input each target programming technical problem into a pre-established word2vec model, and output the distributed word vector of each word of the target programming technical problem.

[0111] Specifically, the word2vec model can be pre-built when the terminal is offline. The word2vec model can accept programming technical questions as input and output a distributed word vector for each word in the programming technical questions.

[0112] Step S150: Based on the pre-established inverse document frequency table and the distributed word vectors of each word of each target programming technical problem, retrieve several semantically similar programming technical problems that are most similar to the technical query problem from the several target programming technical problems.

[0113] Specifically, the inverse document frequency table may be pre-built when the terminal is offline.

[0114] It can be understood that steps S110-S120 are the stage for retrieving the top-k semantically similar questions based on word embeddings. Using the Word2vec model and inverse document frequency table built offline, the top-k semantically similar questions are retrieved from the top-N syntactically similar questions retrieved by Lucene. Here, k can be smaller than N.

[0115] Step S160: recommending semantically similar programming technical problems to the user.

[0116] This embodiment provides a query expansion method for retrieval of programming technical problems, which obtains the user's technical query problem and identifies all the tags contained in the technical query problem. According to the pre-established tag association rules, the tags associated with the tags contained in the technical query problem are added to the tag set of the technical query problem. According to the tag set of the technical query problem, a pre-established Lucene index is used to find several target programming technical problems that are most similar to the syntax of the technical query problem. Each target programming technical problem is input into a pre-established word2vec model, and the distributed word vector of each word of the target programming technical problem is output. Based on the pre-established inverse document frequency table and the distributed word vector of each word of each target programming technical problem, several programming technical problems that are most similar to the semantics of the technical query problem are retrieved from the several target programming technical problems, and each semantically similar programming technical problem is recommended to the user. It can be seen that using technical tag association rules to expand the user's technical query problem can effectively solve the problem of missing technical information in the user's query. Combining the word2vec model and the Lucene index can help users quickly retrieve similar technical problems that are more in line with their needs.

[0117] In some embodiments of the present application, the process of establishing the tag association rules mentioned in the above embodiment is introduced. This process can be completed when the terminal is offline, and the process may include:

[0118] S1. Filter out each technical tag whose usage frequency exceeds a preset number of times from a preset programming technical tag set.

[0119] Specifically, the preset programming technology tag set may include 63,653 technology tags.

[0120] Understandably, high-quality classification using manual or semi-automatic methods is a difficult and time-consuming task. We observed that the usage frequency (i.e., the number of programming technical questions tagged) of many technical tags is very low (less than 1,000), indicating that these technical tags are not commonly used and rarely appear in user queries.

[0121] For example, you can specify a usage frequency of 1000 to filter out 5,284 technical tag categories with a frequency exceeding 1,000. This can reduce the classification burden while ensuring that commonly used technical tags are covered.

[0122] S2. Sampling a number of target technology labels from the various technology labels.

[0123] It is understandable that to construct a category set of technical labels, several target technical labels can be sampled from the various technical labels. Specifically, 356 technical labels were randomly selected from the 5,284 technical labels to be classified based on statistical sampling with a confidence level of 95% and a confidence interval of 5.

[0124] S3. For each target technical tag, perform part-of-speech analysis on the regular expression sentence of the target technical tag using the Stanford POS tagger tool to extract the category phrase of the target technical tag.

[0125] Specifically, for each sampled technical tag, a description sentence that conforms to the above sentence pattern is found using regular expressions, and then the Stanford POS tagger tool is used to perform part-of-speech analysis to extract the category phrase of the technical tag.

[0126] S4. Based on the category phrases of the target technical tags, determine all technical tag categories of the target technical tags.

[0127] It is understandable that by screening and optimizing the category phrases of each target technical tag, a variety of technical tag categories are obtained.

[0128] For example, the Stanford POS tagger tool was used to perform part-of-speech analysis on the regular expression statements of 356 sampled technical tags, extracting the category phrases of the technical tags. Finally, 20 technical tag categories were obtained through screening and optimization, as shown in the following table.

[0129]

[0130] 5,284 technical tags were categorized according to 20 technical tag categories. Ultimately, 2,420 technical tags were classified. The remaining uncategorized technical tags belonged to more abstract categories, such as "keyword" and "concept," and are rarely used in queries. The table above shows the number of tags classified into the 20 categories and some examples. The total number of tags in the 20 categories in the table is 2,507, slightly higher than the total number of categorized tags, 2,420. This is because some tags were classified into multiple categories. For example, "xml" was classified into both "non-PL Language" and "format."

[0131] S5. Among the target technical tags, determine a target technical tag belonging to any one of the technical tag categories as a standard technical tag.

[0132] S6. Establish tag association rules for various standard technical tags based on the transaction set of all programming technical issues in the preset programming technical issue set.

[0133] Each programming technical issue includes a tag set consisting of multiple tags, and the transaction set is a collection of all tag sets of all programming technical issues.

[0134] It's understandable that technical tags often have associations, such as numpy being a Python library. Leveraging this association between tags can more accurately identify and expand on missing technical information in user queries. For example, given a user query like "how to add two vectors with different sizes using numpy?", while the programming language isn't explicitly mentioned, since numpy is associated with Python, the query can be expanded to: "how to add two vectors with different sizes using numpy? python," better matching the technical question.

[0135] Programming technical problems will have several (0-5) technical tags to describe the technical context of the problem. If two technical tags appear simultaneously in many problems, it indicates that there is a strong association between them. Therefore, the association rule mining algorithm can be used to mine the association knowledge between tags from the tag set of programming technical problems. Specifically, the tag set of each programming technical problem is regarded as a transaction, and the transaction set of all programming technical problems is processed using the association rule mining algorithm (such as FP-Growth). Then, the association tag set that meets the set support threshold is obtained, from which tag association rules can be established. For example, given the association tag set (t1, t2), two tag association rules can be established. (meaning that t2 can be inferred from t1) and (Indicates that t1 can be inferred from t2). For each association rule, a confidence threshold δ∈(0,1] is further used for screening. Only association rules that meet the confidence threshold are sufficiently credible. For example, The confidence level of The calculation is as follows:

[0136]

[0137] Under different confidence thresholds, the number and accuracy of label association rules obtained are different, as shown in the following table.

[0138]

[0139] The table shows the number of label association rules and the accuracy of manual evaluation of a randomly sampled subset of rules for settings with confidence thresholds ranging from 0.1 to 1.0. Ultimately, 10,459 label association rules were selected at a confidence threshold of 0.5, with an accuracy of 98%.

[0140] In some embodiments of the present application, the process of establishing a Lucene index mentioned in the above embodiment is introduced. This process can be completed when the terminal is offline, and the process may include:

[0141] S1. Identify all the tags contained in each programming technical problem.

[0142] For example, from the programming technical question “How to add new column to DataFrame using alist?”, three tags can be identified: {dataframe, list, python}.

[0143] It's important to note that programming technical questions often have titles and tags in various forms, such as "java-8," "java8," and "Java8." Before tag recognition, titles and tags must be preprocessed by converting them to lowercase, removing version numbers, and replacing dashes with spaces.

[0144] S2. According to the tag association rule, add tags associated with tags included in each programming technical question to the tag set of the programming technical question.

[0145] Specifically, the programming technical question is expanded according to the tag association rule, that is, tags associated with the identified tags can be added to the tag set of the question.

[0146] It should be noted that in order to avoid expanding too many uncommon tags, which would increase the computational overhead when searching for technical issues, only tags that fall into several (20) technical categories can be considered during expansion.

[0147] For example, knowledge is associated with tags Pandas can be added to the above programming technical problem. After expansion, the tag set of this programming technical problem is {dataframe, list, python, pandas}.

[0148] S3. Preprocess the title, body, and tags of each programming technical question to obtain the preprocessed programming technical question.

[0149] Specifically, the preprocessing process includes three steps. Step 1: Remove HTML tags from the body <pre> <code>Identify long code snippets and all HTML tags. Step 2: Remove stopwords from the title, such as "a" and "an." Step 3: Stemming the words in the title, body, and tags. For example, the word "create," "created," and "creation" all have the root word "creat." Then, based on the preprocessed question set, build a Lucene index, a word embedding model, and an Inverse Document Frequency (IDF) table.

[0150] S4. Construct a document for the programming technical issue based on the title and tags of the programming technical issue.

[0151] S5. Use the index module of the Lucene library to create a Lucene index for the documents of each programming technical problem.

[0152] In some embodiments of the present application, the process of establishing the word2vec model mentioned in the above embodiment is introduced. This process can be completed when the terminal is offline, and the process may include:

[0153] S1. Using the sentence segmentation module of the NLTK library, the title and body of each preprocessed programming technical question are processed to obtain a sentence set of the programming technical question.

[0154] For example, the preprocessed programming problem is: Convert Bytes to Floating Point Numbers? The body is: I have a binary file that I have to parse and I'm using Python. Is there a way to take 4 bytes and convert it to a single precision floating point number? Using the sentence segmentation module of the NLTK library, we can process the title and body of this programming problem to obtain the following three sentences:

[0155] 1. Convert Bytes to Floating Point Numbers;

[0156] 2. I have a binary file that I have to parse and I'm using Python;

[0157] 3. Is there a way to take 4bytes and convert it to a single precision floating point number.

[0158] S2. Use the sentence sets of various programming technical problems as training samples to train the word2vec module of the Gensim library to obtain the word2vec model.

[0159] In some embodiments of the present application, the process of establishing the inverse document frequency table mentioned in the above embodiment is introduced. This process can be completed when the terminal is offline, and the process may include:

[0160] S1. After all programming technical problems are pre-processed, a word set of all programming technical problems is determined.

[0161] S2. Calculate the inverse document frequency of each word in the word set.

[0162] Specifically, the IDF of each word is the inverse of the number of documents containing the word. Inverse document frequency can be used to measure the importance of the word in a document set.

[0163] It is understandable that the smaller the IDF of a word, the more documents contain it (such as functional words such as the, a, an, to), the weaker its ability to express the content of the document, and the lower its importance.

[0164] S3. Based on the inverse document frequency of each word, an inverse document frequency table is constructed.

[0165] In some embodiments provided in this application, a framework diagram of the terminal in the offline and online parts is provided, such as Figure 2 shown.

[0166] In the offline phase, we collected two types of data from the programming Q&A community Stack Overflow (SO): 63,653 programming technology tags and 2,634,592 programming technology questions. The programming technology question set was extracted from SO's official backup data, while the programming technology tag set was obtained from the SO website using a crawler. We then performed four processing steps on these two types of data: tag classification, tag association mining, question expansion, and indexing and model building.

[0167] The online component includes query expansion, Lucene-based Top-N Lexically Similar Question Retrieval, and Word Embedding-based Top-k Semantically Similar Question Retrieval. First, the user query is expanded to provide any missing technical information. Then, a two-stage approach combining Lucene and word embedding technology is used to retrieve programming technical questions similar to the expanded query. The first stage, Lucene search, aims to quickly retrieve the top-N questions that are syntactically similar to the query, significantly reducing the search space for semantically similar questions. The second stage, based on word embedding technology, retrieves the top-k questions with semantic similarity from the top-N questions. Specifically, in the second stage, to accurately measure the functional and technical similarity between programming technical questions and queries, the technical and non-technical terms they contain are distinguished and weighted differently.

[0168] In some embodiments of the present application, the process of retrieving, from the target programming technical issues, several programming technical issues that are most semantically similar to the technical query issue based on the pre-established inverse document frequency table and the distributed word vectors of each word in each target programming technical issue in step S150 is introduced. This process may include:

[0169] S1. Based on the pre-established inverse document frequency table and the distributed word vectors of each word of each target programming technical problem, calculate the asymmetric similarity of the word set of each target programming technical problem to the word set of the technical query problem, and the asymmetric similarity of the word set of the technical query problem to the word set of the target programming technical problem.

[0170] Specifically, the asymmetric similarity between the word set of each target programming technical question and the word set of the technical query question can be calculated using the following formula:

[0171]

[0172] Among them, T1 is the technical vocabulary contained in the target programming technical problem, Q1 is the non-technical vocabulary contained in the technical query problem, w is a word in the technical vocabulary T1, idf(w) is the inverse document frequency of word w in the pre-established inverse document frequency table, sim(w,Q1) is the maximum semantic similarity between word w and any word w' in the technical vocabulary Q1, and sim(w,Q1) is calculated by the cosine similarity between the distributed word vector of word w' and the distributed word vector of word w.

[0173] S2. Calculate the semantic similarity between the word set of the target programming technical problem and the word set of the technical query problem based on the asymmetric similarity of the word set of each target programming technical problem to the word set of the technical query problem, and the asymmetric similarity of the word set of the technical query problem to the word set of the target programming technical problem.

[0174] Specifically, the semantic similarity between the word set of each target programming technical question and the word set of the technical query question can be calculated using the following formula:

[0175]

[0176] Among them, sim(T1,Q1) is the semantic similarity between technical vocabulary T1 and technical vocabulary Q1, sim(T1→Q1) is the asymmetric similarity of technical vocabulary T1 to technical vocabulary Q1, and sim(Q1→T1) is the asymmetric similarity of technical vocabulary Q1 to technical vocabulary T1.

[0177] S3. Determine the semantic similarity between the technical query question and the target programming technical question based on the semantic similarity between the word set of each target programming technical question and the word set of the technical query question.

[0178] Specifically, the semantic similarity between the technical query question and the target programming technical question can be calculated using the following formula:

[0179] sim(T,Q)=η*sim(T1,Q1)+(1-η)*sim(T2,Q2)

[0180] Among them, T is the target programming technical problem, Q is the technical query problem, T2 is the non-technical word set contained in the target programming technical problem, Q2 is the non-technical word set contained in the technical query problem, η∈[0,1] is the semantic similarity weight factor, and sim(T2,Q2) is the semantic similarity between the non-technical word set T2 and the non-technical word set Q2.

[0181] It is understandable that η is used to balance the importance of the two word sets. To determine the appropriate value of η, we first refer to the experimental evaluation methods of existing research work and randomly select the titles of 10 questions from the programming technology question library as experimental queries, as shown in the following table.

[0182]

[0183] Furthermore, we manually scored the relevance of the top 10 semantically similar questions retrieved by QE4PTQR for each experimental query on a five-point scale of 0-4, with η ranging from 0.1 to 1.0. Specifically, 0, 1, 2, 3, and 4 represent "strongly irrelevant," "irrelevant," "neutral," "relevant," and "strongly relevant," respectively.

[0184] Based on the relevance evaluation results, two commonly used indicators were used to evaluate the performance of the Top-k semantically similar question lists for each query: precision (Pre@k) and normalized discounted cumulative gain (NDCG@k). Precision (Pre@k) is used to calculate the proportion of truly relevant questions among the Top-k similar questions, while normalized discounted cumulative gain (NDCG@k) is used to comprehensively consider the position and specific score of each relevant question in the Top-k similar question list, which can reflect the overall quality.

[0185]

[0186]

[0187] Among them, rel i Score the relevance of the question at position i, IDCG k represents the maximum DCG score that can be achieved by the Top-k similar question list.

[0188] The following table shows the performance comparison between Pre@k and NDCG@k under different η settings.

[0189]

[0190] It can be seen that when η is 0.4, the performance reaches the best.

[0191] S4. Determine a preset number of target programming technical problems with the highest semantic similarity from among the target programming technical problems, and define them as semantically similar programming technical problems.

[0192] In some embodiments of the present application, in order to evaluate the performance of a query expansion method for programming technical problem retrieval provided in this application (denoted as QE4PTQR), two mainstream programming technical problem retrieval methods were selected: Lucene retrieval and a word embedding-based retrieval method, denoted as Lucene and WE respectively. In addition, two commonly used query expansion methods were also selected: QECK and WordNet (WN). By combining the two retrieval methods with the two query expansion methods, six benchmark programming technical problem retrieval methods were obtained: Lucene, WE, Lucene+QECK, WE+QECK, Lucene+WN, and WE+WN.

[0193] QE4PTQR and the six baseline methods were applied to the 10 experimental queries described above. The top 10 similar questions recommended for each query were manually scored for relevance on a five-point scale of 0 to 4. Based on the scoring results, the average performance of each method on the Pre@k and NDCG@k metrics was measured.

[0194] As shown in the following table, the performance results of QE4PTQR and 6 baseline methods are shown.

[0195]

[0196] It can be seen that QE4PTQR achieved the best performance in both Pre@k and NDCG@k indicators. In particular, QECK and WN had a negative impact on the results of programming technical problem retrieval. The reason is that neither of them accurately identified the missing information in the query, resulting in a lot of noise in the expanded vocabulary, which reduced the retrieval quality. In contrast, the query expansion method for programming technical problem retrieval proposed in this application can accurately identify the missing technical information in the query and expand it, ultimately effectively improving the performance of programming technical problem retrieval.

[0197] The following describes a query expansion device for retrieval of programming technical issues provided in an embodiment of the present application. The query expansion device for retrieval of programming technical issues described below and the query expansion method for retrieval of programming technical issues described above can be referenced to each other.

[0198] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of a query expansion device for implementing programming technology problem retrieval disclosed in an embodiment of the present application.

[0199] like Figure 3 As shown, the device may include:

[0200] A technical question tag identification unit 11 is used to obtain a user's technical query question and identify all tags contained in the technical query question;

[0201] The associated tag supplement unit 12 is configured to supplement tags associated with the tags included in the technical query question into the tag set of the technical query question according to a pre-established tag association rule;

[0202] A target programming technical problem determining unit 13 is configured to search for a number of target programming technical problems that are most similar in syntax to the technical query problem using a pre-established Lucene index based on the tag set of the technical query problem;

[0203] A word2vec model analysis unit 14 is configured to input each target programming technical problem into a pre-established word2vec model and output a distributed word vector for each word in the target programming technical problem;

[0204] A similar question determining unit 15 is configured to retrieve, from the target programming technical questions, a plurality of semantically similar programming technical questions that are most semantically similar to the technical query question based on a pre-established inverse document frequency table and distributed word vectors of each word in each target programming technical question;

[0205] The recommendation unit 16 is configured to recommend semantically similar programming technical problems to the user.

[0206] Optionally, the device further includes:

[0207] A technical tag screening unit, configured to screen out technical tags whose usage frequency exceeds a preset number of times from a preset programming technical tag set;

[0208] a target technology label sampling unit, configured to sample a plurality of target technology labels from the various technology labels;

[0209] A category phrase extraction unit is used to perform part-of-speech analysis on the regular expression sentence of each target technical tag using the Stanford POS tagger tool to extract the category phrase of the target technical tag;

[0210] a technical label category determination unit, configured to determine all technical label categories of each target technical label based on the category phrase of each target technical label;

[0211] a standard technical label determination unit, configured to determine, among the target technical labels, a target technical label belonging to any one of the technical label categories as a standard technical label;

[0212] The tag association rule establishing unit is used to establish tag association rules for each standard technical tag based on the transaction set of all programming technical problems in the preset programming technical problem set. Each programming technical problem includes a tag set consisting of multiple tags, and the transaction set is the collection of all tag sets of all programming technical problems.

[0213] Optionally, the device further includes:

[0214] A tag identification unit, used to identify all tags contained in each programming technical problem;

[0215] a tag set supplementing unit, configured to supplement tags associated with tags included in each programming technical question into the tag set of the programming technical question according to the tag association rule;

[0216] A preprocessing unit, configured to preprocess the title, body, and tags of each programming technical question to obtain the preprocessed programming technical question;

[0217] A document construction unit, configured to construct a document of the programming technical problem according to the title and tags of the programming technical problem;

[0218] The Lucene index building unit is used to build a Lucene index for documents related to various programming technical issues using the index module of the Lucene library.

[0219] Optionally, the device further includes:

[0220] A sentence segmentation unit is used to process the title and body of each pre-processed programming technical question using the sentence segmentation module of the NLTK library to obtain a sentence set of the programming technical question;

[0221] The model training unit is used to train the word2vec module of the Gensim library using sentence sets of various programming technical problems as training samples to obtain a word2vec model.

[0222] Optionally, the device further includes:

[0223] A word set determination unit, configured to determine a word set for all programming technical problems after all programming technical problems have been pre-processed;

[0224] an inverse document frequency calculation unit, configured to calculate the inverse document frequency of each word in the word set;

[0225] The inverse document frequency table construction unit is used to construct an inverse document frequency table based on the inverse document frequency of each word.

[0226] Optionally, the similar problem determining unit includes:

[0227] an asymmetric similarity calculation unit, configured to calculate, based on a pre-established inverse document frequency table and the distributed word vectors of each word of each target programming technical problem, an asymmetric similarity of the word set of each target programming technical problem to the word set of the technical query problem, and an asymmetric similarity of the word set of the technical query problem to the word set of the target programming technical problem;

[0228] a word set semantic similarity calculation unit, configured to calculate the semantic similarity between the word set of each target programming technical problem and the word set of the technical query question based on the asymmetric similarity of the word set of each target programming technical problem to the word set of the technical query question, and the asymmetric similarity of the word set of the technical query question to the word set of the target programming technical problem;

[0229] a question semantic similarity calculation unit, configured to determine the semantic similarity between the technical query question and the target programming technical question based on the semantic similarity between the word set of each target programming technical question and the word set of the technical query question;

[0230] The semantically similar programming technical problem determining unit determines a preset number of target programming technical problems with the highest semantic similarity from the target programming technical problems, as the semantically similar programming technical problems.

[0231] Optionally, the asymmetric similarity calculation unit includes:

[0232] The asymmetric similarity calculation subunit is used to calculate the asymmetric similarity of the word set of each target programming technical question to the word set of the technical query question using the following formula:

[0233]

[0234] Among them, T1 is the technical vocabulary contained in the target programming technical problem, Q1 is the non-technical vocabulary contained in the technical query problem, w is a word in the technical vocabulary T1, idf(w) is the inverse document frequency of word w in the pre-established inverse document frequency table, sim(w,Q1) is the maximum semantic similarity between word w and any word w' in the technical vocabulary Q1, and sim(w,Q1) is calculated by the cosine similarity between the distributed word vector of word w' and the distributed word vector of word w.

[0235] Optionally, the word set semantic similarity calculation unit includes:

[0236] The word set semantic similarity calculation subunit is used to calculate the semantic similarity between the word set of each target programming technical question and the word set of the technical query question using the following formula:

[0237]

[0238] Among them, sim(T1,Q1) is the semantic similarity between technical vocabulary T1 and technical vocabulary Q1, sim(T1→Q1) is the asymmetric similarity of technical vocabulary T1 to technical vocabulary Q1, and sim(Q1→T1) is the asymmetric similarity of technical vocabulary Q1 to technical vocabulary T1.

[0239] Optionally, the word set semantic similarity calculation unit includes:

[0240] The word set semantic similarity calculation subunit is used to calculate the semantic similarity between the technical query question and the target programming technical question using the following formula:

[0241] sim(T,Q)=η*sim(T1,Q1)+(1-η)*sim(T2,Q2)

[0242] Among them, T is the target programming technical problem, Q is the technical query problem, T2 is the non-technical word set contained in the target programming technical problem, Q2 is the non-technical word set contained in the technical query problem, η∈[0,1] is the semantic similarity weight factor, and sim(T2,Q2) is the semantic similarity between the non-technical word set T2 and the non-technical word set Q2.

[0243] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise," "include," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus comprising a set of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising the element.

[0244] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0245] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.< / code> < / pre>

Claims

1. A query expansion method for retrieving programming technical issues, characterized in that: include: Obtaining the user's technical query question and identifying all technical tags contained in the technical query question; According to pre-established tag association rules, tags associated with tags included in the technical query question are added to the tag set of the technical query question; According to the tag set of the technical query question, searching for several target programming technical questions that are most similar in syntax to the technical query question through a pre-established Lucene index; Input each target programming technical problem into the pre-established word2vec model and output the distributed word vector of each word in the target programming technical problem; Retrieving, from the plurality of target programming technical questions, a plurality of semantically similar programming technical questions that are most semantically similar to the technical query question based on a pre-established inverse document frequency table and distributed word vectors of each word of each target programming technical question; Recommending semantically similar programming technical problems to the user.

2. The method according to claim 1, characterized in that The process of establishing the tag association rule includes: Filtering out each technical tag whose usage frequency exceeds a preset number of times from a preset programming technical tag set; Sampling a plurality of target technical labels from the various technical labels; For each target technology tag, perform part-of-speech analysis on the regular expression sentence of the target technology tag using the Stanford Part-of-Speech tagger tool to extract the category phrase of the target technology tag; Determine all technical tag categories of each target technical tag based on the category phrase of each target technical tag; Among the target technical labels, a target technical label belonging to any one of the technical label categories is determined as a standard technical label; According to the transaction set of all programming technical problems in the preset programming technical problem set, label association rules of various standard technical labels are established. Each programming technical problem includes a label set consisting of multiple labels, and the transaction set is the collection of all label sets of all programming technical problems.

3. The method according to claim 2, characterized in that The Lucene index creation process includes: Identify all the tags that each programming technical question contains; According to the tag association rule, tags associated with tags included in each programming technical question are added to the tag set of the programming technical question; Preprocessing the title, body, and tags of each programming technical question to obtain the preprocessed programming technical question; Build a document for the programming technical problem based on the title and tags of the programming technical problem; Use the index module of the Lucene library to create a Lucene index for documents on various programming technical issues.

4. The method according to claim 3, characterized in that The process of establishing the word2vec model includes: The sentence segmentation module of the NLTK library is used to process the title and body of each preprocessed programming technical question to obtain a sentence set of the programming technical question; The sentence sets of various programming technical problems are used as training samples to train the word2vec module of the Gensim library to obtain the word2vec model.

5. The method according to claim 3, characterized in that The process of establishing the inverse document frequency table includes: After all programming technical problems are pre-processed, a word set of all programming technical problems is determined; Calculate the inverse document frequency of each word in the word set; Based on the inverse document frequency of each word, an inverse document frequency table is constructed.

6. The method according to any one of claims 1 to 5, characterized in that The method of retrieving a plurality of programming technical questions that are most semantically similar to the technical query question from the plurality of target programming technical questions based on the pre-established inverse document frequency table and the distributed word vectors of each word of each target programming technical question includes: Based on a pre-established inverse document frequency table and the distributed word vectors of each word in each target programming technical question, calculate the asymmetric similarity of the word set of each target programming technical question to the word set of the technical query question, and the asymmetric similarity of the word set of the technical query question to the word set of the target programming technical question; Calculate the semantic similarity between the word set of the target programming technical problem and the word set of the technical query question based on the asymmetric similarity between the word set of each target programming technical problem and the word set of the technical query question, and the asymmetric similarity between the word set of the technical query question and the word set of the target programming technical problem; Determining the semantic similarity between the technical query question and the target programming technical question based on the semantic similarity between the word set of each target programming technical question and the word set of the technical query question; A preset number of target programming technical problems with the highest semantic similarity are determined from among the target programming technical problems, as semantically similar programming technical problems.

7. The method according to claim 6, characterized in that The step of calculating the asymmetric similarity of the word set of each target programming technical question to the word set of the technical query question based on the pre-established inverse document frequency table and the distributed word vectors of each word of each target programming technical question includes: The asymmetric similarity of the word set of each target programming technical question to the word set of the technical query question is calculated using the following formula: Among them, T1 is the technical vocabulary contained in the target programming technical problem, Q1 is the non-technical vocabulary contained in the technical query problem, w is a word in the technical vocabulary T1, idf(w) is the inverse document frequency of word w in the pre-established inverse document frequency table, sim(w,Q1) is the maximum semantic similarity between word w and any word w' in the technical vocabulary Q1, and sim(w,Q1) is calculated by the cosine similarity between the distributed word vector of word w' and the distributed word vector of word w.

8. The method according to claim 7, characterized in that The calculating of the semantic similarity between the word set of the target programming technical problem and the word set of the technical query question based on the asymmetric similarity of the word set of each target programming technical problem to the word set of the technical query question, and the asymmetric similarity of the word set of the technical query question to the word set of the target programming technical problem, includes: The semantic similarity between the word set of each target programming technical question and the word set of the technical query question is calculated using the following formula: Among them, sim(T1,Q1) is the semantic similarity between technical vocabulary T1 and technical vocabulary Q1, sim(T1→Q1) is the asymmetric similarity of technical vocabulary T1 to technical vocabulary Q1, and sim(Q1→T1) is the asymmetric similarity of technical vocabulary Q1 to technical vocabulary T1.

9. The method according to claim 8, characterized in that Determining the semantic similarity between the technical query question and the target programming technical question based on the semantic similarity between the word set of each target programming technical question and the word set of the technical query question includes: The semantic similarity between the technical query question and the target programming technical question is calculated using the following formula: sim(T,Q)=η*sim(T1,Q1)+(1-η)*sim(T2,Q2) Among them, T is the target programming technical problem, Q is the technical query problem, T2 is the non-technical word set contained in the target programming technical problem, Q2 is the non-technical word set contained in the technical query problem, η∈[0,1] is the semantic similarity weight factor, and sim(T2,Q2) is the semantic similarity between the non-technical word set T2 and the non-technical word set Q2.

10. A query expansion device for searching programming technical problems, characterized in that: include: A technical question tag identification unit is used to obtain the user's technical query question and identify all tags contained in the technical query question; an associated tag supplement unit, configured to supplement tags associated with tags included in the technical query question into the tag set of the technical query question according to a pre-established tag association rule; a target programming technical problem determining unit, configured to search, based on the tag set of the technical query problem, through a pre-established Lucene index for a number of target programming technical problems that are most similar in syntax to the technical query problem; A word2vec model analysis unit is used to input each target programming technical problem into a pre-established word2vec model and output a distributed word vector for each word in the target programming technical problem; a similar question determining unit, configured to retrieve, from the plurality of target programming technical questions, a plurality of semantically similar programming technical questions that are most semantically similar to the technical query question based on a pre-established inverse document frequency table and a distributed word vector of each word in each target programming technical question; The recommendation unit is used to recommend semantically similar programming technical problems to the user.

Images