Method and system for identifying intention of programming technical problem

Abstract

The invention provides an intention recognition method and system for programming technical issues, and belongs to the technical field of software engineering.The method comprises the steps that the programming technical issues of a target platform are collected and screened; performing intention annotation on the programming technical problem by adopting an iterative statistical sampling mode, and constructing a multi-level intention classification system to obtain a basic data set; performing data enhancement on samples in the basic data set based on a large language model to obtain an enhanced data set; and training an intention recognizer through the enhanced data set, and carrying out intention recognition on the to-be-tested programming technical problem through the trained intention recognizer. According to the method, the multi-level intention classification system is constructed and the intention recognizer is trained, so that understanding of deep semantics of programming technical problems is realized, the problem of category imbalance in a data set can be relieved through a data enhancement mechanism, and the intention recognition accuracy is improved.

Description

Technical Field

[0001] This invention belongs to the field of software engineering technology, and in particular relates to an intent recognition method and system for programming technology problems. Background Technology

[0002] In their daily learning and work, software developers frequently encounter various programming technical problems. To facilitate problem-solving, crowdsourced question-and-answer platforms have been created, such as Stack Overflow (SO) and CSDN. Developers can post questions on these platforms to seek answers from other developers. Since their inception, these Q&A platforms have been widely used by developers, accumulating tens of millions of questions and answers. For both the platform's question dispatchers and answerers, accurately providing solutions hinges on accurately identifying the question's intent, such as concept explanation / comparison, task implementation, or anomaly diagnosis.

[0003] To address this need, some relevant research and practices have emerged. For example, Q&A platforms typically provide technical tags (such as programming languages, operating systems, etc.) to label questions, helping the platform better manage questions and assisting answerers in quickly identifying the technical background of the question. However, these technical tags are usually just single nouns or noun phrases (such as Java, Linux), which cannot accurately express the intent of the question. In addition, most current methods for classifying and identifying the intent of technical questions are mostly aimed at specific topics (such as API, Android), lacking effective coverage of a wide range of technical questions, and the granularity of the intent classification system is relatively coarse. Summary of the Invention

[0004] This invention provides a method and system for identifying the intent of programming technical problems, addressing the shortcomings of insufficient generalization ability in existing technologies for identifying programming technical problems, and achieving effective coverage of a wide range of technical issues. This invention also constructs a fine-grained intent classification system, which can accurately express the deep-seated needs and intents of programming technical problems.

[0005] In a first aspect, the present invention provides an intent recognition method for programming technology problems, comprising: Collect and filter programming technical issues for the target platform; An iterative statistical sampling method is used to annotate the programming technical problems and a multi-level intent classification system is constructed to obtain the basic dataset; Data augmentation is performed on samples in the base dataset based on a large language model to obtain an augmented dataset. The intent recognizer is trained using the augmented dataset, and the trained intent recognizer is used to identify the intent of the programming technique problem under test.

[0006] Based on the above technical solutions, preferably, the multi-level intent classification system includes a first-level intent category and a second-level intent category corresponding to each first-level intent category; The primary intent categories include general search, debugging and bug fixing, programming, third-party libraries / services / frameworks / platforms, tools, databases, and testing; The secondary intent categories corresponding to the regular search include: explanations of unknown terms, best practices in searching, differences between search technologies, and explanations of problems with search technologies; The secondary intent categories corresponding to debugging and bug fixing include: searching for explanations of anomalies / warnings / error messages, searching for solutions to common errors, searching for solutions to software configuration errors, searching for solutions to security-related errors, searching for solutions to performance problems, searching for solutions to multithreading problems, and searching for solutions to abnormal results; The secondary intent categories corresponding to the programming include: searching for examples or guidance on how to use specific features of a programming language, searching for standards, searching for examples or guidance on how to avoid anti-patterns, searching for pseudocode of an algorithm, code examples or algorithm principles, searching for alternatives or best solutions / techniques, searching for solutions to specific tasks, and searching for differences between methods. The secondary intent categories corresponding to the third-party libraries / services / frameworks / platforms include: searching for third-party libraries / services / frameworks / platforms, searching for usage examples or examples of third-party libraries / services / frameworks / platforms, and searching for examples of how to implement interactive operations between third-party libraries / services / frameworks / platforms. The secondary intent categories corresponding to the tools include: searching for examples and guidance on how to use the tools, searching for examples and guidance on how to configure the tools, searching for examples and guidance on how to upgrade / downgrade / install the tools, and searching for solutions to tool problems; The secondary intent categories corresponding to the database include: searching for examples or guidance on how to formulate structured query statements, searching for examples or guidance on how to use non-relational databases, and searching for database optimization solutions; The secondary intent categories corresponding to the tests include: searching for test method examples or guidance, searching for usage examples or guidance on how to use automated testing tools, and searching for test methods for specific tasks.

[0007] Based on the above technical solutions, preferably, the step of performing data augmentation on the samples in the basic dataset based on a large language model to obtain an augmented dataset includes: Extract the scarce categories from the basic dataset whose sample count is less than a preset threshold; Based on the text data in the samples corresponding to the scarce categories, new problem examples are generated using a large language model. Based on the problem examples, the scarce category is augmented to obtain an enhanced dataset.

[0008] Based on the above technical solutions, preferably, the step of generating new problem examples based on the text data in the samples corresponding to the scarce category and using a large language model includes: Extract the title and body text of each question from the samples corresponding to the scarce category; Design prompt text, and use the title and body of each question as examples in the large language model. Input the prompt text and the examples into the large language model to obtain multiple question examples generated by the large language model for each question.

[0009] Based on the above technical solutions, preferably, the design principles of the prompt text include: The generated problem examples strictly adhere to the core intent of the example problem, avoiding deviations from semantics and the introduction of irrelevant information; The generated sample questions maintain the same style as the example questions, including tone, structure, and technical expression. Ensure that the generated problem examples are natural, fluent, and grammatically correct, avoiding factual or logical errors; Limit the number of generated problem samples to a reasonable range.

[0010] Based on the above technical solutions, preferably, the title and body of the sample questions in the augmented dataset are embedded using a text embedding model to generate semantic vector representations; the semantic vector representations and the intent categories corresponding to the sample questions are input into an intent recognizer; the intent recognizer outputs the predicted intent categories; and the parameters of the text embedding model and the intent recognizer are jointly optimized using a backpropagation algorithm to obtain the trained intent recognizer.

[0011] Based on the above technical solutions, preferably, the text embedding model includes the BGE-M3 model.

[0012] Secondly, the present invention also provides an intent recognition system for programming technology problems, comprising: The data collection module is used to collect and filter programming technical issues on the target platform; The data annotation module is used to annotate the programming technical problems using iterative statistical sampling and to construct a multi-level intent classification system to obtain the basic dataset. The data augmentation module is used to augment the samples in the base dataset based on the large language model to obtain an augmented dataset; The intent recognition module is used to train an intent recognizer using the augmented dataset and to recognize the intent of the programming technology problem to be tested using the trained intent recognizer.

[0013] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the intent recognition method as described in any of the programming techniques above.

[0014] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intent recognition method for any of the programming techniques described above.

[0015] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the intent recognition method as described in any of the programming techniques above.

[0016] The beneficial effects of the technical solutions provided by some embodiments of the present invention include at least the following: 1) The programming technical problem intent recognition method and system provided by the present invention realizes the deep semantic understanding of programming technical problems by constructing a multi-level intent classification system and training an intent recognizer, and accurately expresses the deep-level needs and intents of the problem; in addition, the present invention performs data augmentation on the samples based on a large language model, which can effectively alleviate the problem of class imbalance in the dataset and improve the accuracy of intent recognition.

[0017] 2) The multi-level intent classification system constructed by this invention includes a first-level intent category and a second-level intent category corresponding to each first-level intent category. It realizes fine-grained and deep-level problem intent classification, and can support a wider range of technical problem intent classification and recognition. It can overcome the defects of existing intent classification systems that are coarse-grained and have insufficient coverage.

[0018] 3) This invention utilizes the learning and reasoning capabilities of large language models to expand samples in scarce categories, which can alleviate the problem of data scarcity, achieve data balance and correction, and at the same time reduce data annotation costs and improve data diversity and coverage.

[0019] 4) This invention ensures the quality of generated question examples by reasonably designing the prompt text and using the title and body of each question as examples of a large language model, thus avoiding illusions and data distortion.

[0020] 5) This invention utilizes a text embedding model to construct a text vector representation, which can fully capture the semantic features of the question text. Furthermore, joint training of the text embedding model and the intent recognizer can improve the stability and actual prediction performance of the intent recognizer. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is one of the flowcharts illustrating an intent recognition method for programming technology problems provided by the present invention; Figure 2 This is a general framework diagram of an intent recognition method for programming technology problems provided by the present invention; Figure 3 This is an example of a programming technique question for the SO Q&A platform provided by this invention; Figure 4 This is the second flowchart illustrating an intent recognition method for programming technology problems provided by the present invention; Figure 5 This is a pseudocode diagram illustrating the offline training phase of the intent recognizer provided by the present invention; Figure 6 This is an example of the prompt text provided by the present invention; Figure 7 This is a pseudocode diagram illustrating the online application phase of the intent recognizer provided by the present invention; Figure 8 This is a schematic diagram of the structure of an intent recognition system for programming technology problems provided by the present invention; Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] Developers frequently use Q&A platforms (such as SO) to find solutions to programming technical problems. With the widespread use of these platforms, a large number of programming-related questions have accumulated. However, the sheer volume, diverse topics, and loose structure of these questions pose a significant challenge to platform administrators and answerers in efficiently organizing and responding to them. Therefore, classifying and identifying the intent behind programming technical questions has become an important foundational task.

[0025] In existing technologies, intent classification methods for programming technical questions are mainly divided into classification methods based on general tags and classification methods oriented towards specific domains. Their main drawbacks are twofold: 1) The classification system is usually built around a macro theme (such as technical tags and API usage on SO Q&A platforms, error handling, etc.), lacking further division of the internal structure of the question intent. The granularity of the category system it relies on or customizes is relatively coarse, making it difficult to accurately express the deep-seated needs and intents of the question; 2) Classification methods oriented towards specific domains are mainly for technical questions in specific domains or themes. The scope of the technical questions they cover is relatively narrow (such as those related to APIs or Android), and their generalization ability is insufficient, making it difficult to support the classification and identification of a wide range of technical question intents.

[0026] Taking question-and-answer platforms such as SO as an example, existing intent classification and recognition methods can be mainly divided into classification methods based on general tags and classification methods oriented towards specific domains. The overall results are shown in Table 1.

[0027] Table 1 Analysis of Existing Programming Technology Problem Intent Classification Methods

[0028] To address the above problems, this invention proposes a method and system for intent recognition in programming technical questions. First, an iterative annotation process is used to construct a fine-grained intent classification system to overcome the shortcomings of existing systems, such as coarse granularity and insufficient coverage. Then, data augmentation techniques are used to generate a high-quality augmented dataset. Finally, an automated intent recognizer based on deep learning is designed to effectively identify the intent of a given technical question, providing support for the organization and answering of programming technical questions.

[0029] The key technical terms involved in this invention are explained below: Embedding Model: A type of machine learning model that transforms input (text, images, code snippets, etc.) into a low-dimensional, dense, continuous vector representation, facilitating downstream tasks such as similarity calculation, classification, or retrieval.

[0030] Large Language Model: A type of language model consisting of artificial neural networks with a large number of parameters (usually in the billions or more), trained on a large amount of unlabeled text using self-supervised or semi-supervised learning.

[0031] GPT-4o: A multimodal large language model that integrates multimodal data such as text and images during training, enabling it to process and generate cross-modal content simultaneously.

[0032] Data augmentation is a method that expands the data scale and improves the model's generalization ability by applying certain transformations or generation operations to existing training samples without collecting additional new data.

[0033] Multi-label classification: a classification task in which each sample can be assigned multiple labels simultaneously from a finite set of labels.

[0034] BGE-M3 (BAAI General Embeddings M3) model: A text embedding model used to embed text to generate semantic vectors.

[0035] Weighted Binary Cross-Entropy Loss (BCE With Logits Loss): A loss function used for binary classification, it is an extension of the Binary Cross-Entropy Loss (BCE With Logits Loss). When calculating the BCE loss, it multiplies the loss of positive samples by a specific weight to address the imbalance between positive and negative samples in the dataset.

[0036] Cosine Annealing Learning Rate: A technique that dynamically adjusts the learning rate according to the shape of a cosine function curve during model training, so that the learning rate smoothly decreases to a minimum value (usually 0).

[0037] Example 1 Please see Figure 1 , Figure 1 One of the flowcharts for an intent recognition method for programming technology problems provided by an embodiment of the present invention includes: S101. Collect and filter programming technical issues for the target platform; S102. An iterative statistical sampling method is used to annotate the intent of programming technical problems and a multi-level intent classification system is constructed to obtain the basic dataset. S103. Augment the samples in the basic dataset based on the large language model to obtain the augmented dataset; S104. Train the intent recognizer using an augmented dataset, and use the trained intent recognizer to identify the intent of the programming technology problem under test.

[0038] This invention achieves a deep understanding of the semantics of programming technical problems by constructing a multi-level intent classification system and training an intent recognizer, thus accurately expressing the deep-seated needs and intentions of the problems. In addition, this application uses a large language model to augment the samples, which can effectively alleviate the problem of class imbalance in the dataset and further improve the accuracy of intent recognition.

[0039] This embodiment uses the SO developer forum Q&A platform as an example to illustrate the intent recognition method for programming technical questions proposed in this invention. Figure 2 The diagram shown is an overall framework diagram of an intent recognition method for programming technology problems proposed in this invention. Figure 2 As shown, this intent recognition method mainly includes two stages: The first phase involves building an intent classification system: ① collecting and filtering programming technical questions from the target platform (SO Q&A platform); ② constructing a fine-grained intent classification system that can cover a wide range of programming technical questions, as well as a high-quality basic dataset with intent annotations, through iterative statistical sampling and intent labeling. The second stage, intent recognizer training and application, is divided into two phases: offline training and online application. In the offline training phase: ③ Preprocess the basic dataset; ④ Use LLM-based data augmentation techniques to augment the basic dataset, obtaining an augmented dataset; ⑤ Train an intent recognizer based on the augmented dataset. In the online application phase, when given a programming technical question without intent annotation (such as a newly posted question on the SO Q&A platform), execute ⑥ Intent recognizer application, that is, use the trained intent recognizer to automatically identify the intent category of the question to facilitate question management and answering.

[0040] The following is combined with Figure 1 and Figure 2 The technical solution of the present invention will be described.

[0041] In S101 of this embodiment, programming technical issues of the target platform are collected and screened.

[0042] Specifically, you can download the official Stack Exchange dataset from the Stack Exchange data backup website and extract all the programming technical questions. For example... Figure 3 The image shows an example of programming technical questions from the SO Q&A platform. Each question includes a title, body text, tags, and a score. To ensure data quality, vague, incomplete, or meaningless questions are filtered out. Only questions with a score higher than 5 and more than 500 view counts are retained, resulting in 1,503,584 high-quality questions, which form the programming technical question set for a multi-level intent classification system.

[0043] In S102 of this embodiment, an intent category system that can effectively cover the large-scale programming technical problem set collected in S101 is constructed by using iterative statistical sampling and manual collaborative annotation.

[0044] For example, the specific process of S102 may include: 1) Statistical Sampling: Using sampling parameters with a confidence level of 95% and a confidence interval of 5, a sample set is randomly drawn from a set of programming technical problems that have never been sampled before using statistical sampling methods. For example, the sample size in the first round of sampling can be 384.

[0045] 2) Intent Labeling: Two technical personnel familiar with the SO Q&A platform and with over 3 years of programming experience independently label the intent of each sample question. They then work together to semantically align the labeled intent categories and discuss any discrepancies in the labeling. Finally, a unified intent category system and question labeling results are established. It's worth noting that each question may have one or more intent categories. Before independent labeling, the two technical personnel must reach a consensus on the expression of the intent categories, using gerund phrases that accurately express the need (e.g., "search for explanations of unknown terms") to reduce the burden of subsequent semantic alignment.

[0046] 3) Iterative process: Since it is difficult for the results of one round of sampling and labeling to fully cover all intent categories of programming technical problems, this invention adopts a strategy of multiple rounds of sampling and labeling, that is, repeating the above steps 1) and 2) until no new intent categories are found.

[0047] Following the iterative statistical sampling and labeling process of steps 1) to 3), the iteration terminates after the fourth round of sampling and intent labeling, resulting in the basic dataset. A total of 1,536 questions were sampled and labeled, identifying 32 intent categories. These categories are divided into 7 intent dimensions. A multi-level intent classification system is constructed, using the 7 intent dimensions as primary intent categories and the 32 intent categories as secondary intent categories. Table 2 shows the multi-level intent classification system and data distribution table.

[0048] Table 2. Multi-level Intent Classification System and Data Distribution

[0049] In Table 2, the last column, "Number of Questions," gives the number of sample questions under each intent category. It is worth noting that the total number of questions in the "Number of Questions" column will be greater than 1,536 (specifically 2,534). This is because the two annotators used multi-intent annotation. For example, the intent "How to add a grid line at a specific location in matplotlib plot?" belongs to both "searching for solutions to a specific task" and "searching for usage or examples of third-party libraries / services / frameworks / platforms."

[0050] The multi-level intent classification system constructed in this invention achieves fine-grained and in-depth problem intent classification, while also supporting a wider range of technical problem intent classification and recognition. It can overcome the shortcomings of existing intent classification systems, such as coarse granularity and insufficient coverage.

[0051] In S103 of this embodiment, the samples in the basic dataset are preprocessed, and the samples in the basic dataset are augmented using a large language model to obtain an augmented dataset.

[0052] Specifically, preprocessing includes removing code snippets, images, etc., from the main text, retaining only the text content.

[0053] For example, data augmentation of samples in a basic dataset using a large language model can be achieved by designing diverse prompts to guide the large language model in generating problem description texts from different perspectives around a specific programming technology problem, thereby expanding the sample size for that problem and increasing sample diversity. Alternatively, around a specific category of programming technology problems, by using a small number of sample examples and specifying generation conditions (such as category labels, style, and keywords), the large language model can be adapted to the specific research domain of the programming technology problem, generating controllable new samples to balance data categories.

[0054] For example, back-translation enhancements can be applied to programming techniques in the base dataset, such as translating English text into Chinese and then back into English, introducing sentence variations, and enhancing sentence diversity to adapt to multilingual tasks.

[0055] For example, synonym substitution, sentence transformation, or restatement can be performed on programming techniques in the base dataset to maintain semantic integrity and alleviate overfitting in classification tasks.

[0056] Understandably, when performing data augmentation using large language models, it is necessary to filter the generated results to avoid semantic deviation, low-quality or duplicate data, and to control the degree of variation in the generated text to avoid deviating from the actual data distribution. If necessary, a manual review process can be added.

[0057] In S104 of this embodiment, an intent recognizer is constructed and trained by an augmented dataset for identifying the intent of the programming technology problem to be tested.

[0058] Since the multi-level intent classification system constructed in this invention has many secondary intent categories, the intent recognizer constructed needs to be a model capable of performing multi-label classification tasks.

[0059] For example, intent recognizers can be built based on deep learning models, such as Transformer-based models, Convolutional Neural Network (CNN)-based models, Recurrent Neural Network (RNN)-based models, etc. By directly training such deep learning models, multi-classification tasks can be performed.

[0060] For example, an ensemble learning strategy can be used, employing 3-5 models with different architectures to perform intent recognition separately, and then voting to determine the final intent recognition result. Alternatively, multiple base classifiers can be selected, and the XGBoost algorithm can be used for fusion prediction to determine the final intent recognition result.

[0061] The trained intent recognizer can then perform multi-label classification tasks for new programming technique problems. Specifically, the programming technique problem to be tested is preprocessed in the same way as in S103 and then input into the trained intent recognizer to obtain accurate secondary intent category recognition results.

[0062] Example 2 Please see Figure 4 , Figure 4 A second flowchart illustrating an intent recognition method for programming technology problems provided as an embodiment of the present invention includes: S201. Collect and filter programming technical issues for the target platform; S202. An iterative statistical sampling method is used to annotate the intent of programming technical problems and a multi-level intent classification system is constructed to obtain the basic dataset; S203. Extract the scarce categories in the basic dataset whose sample number is less than a preset threshold. S204. Based on the text data in the samples corresponding to the scarce categories, generate new problem examples using a large language model. S205. Based on the problem examples, the scarce categories are augmented to obtain an enhanced dataset; S206. Use a text embedding model to embed the titles and body text of the sample questions in the augmented dataset to generate semantic vector representations; S207. Input the semantic vector representation of the intent category corresponding to the sample question into the intent recognizer, output the predicted intent category through the intent recognizer, and jointly optimize the parameters of the text embedding model and the intent recognizer through the backpropagation algorithm to obtain the trained intent recognizer. S208. Perform intent recognition of the programming technology problem under test using the trained intent recognizer.

[0063] This invention leverages the learning and reasoning capabilities of a large language model to expand samples in scarce categories, alleviating data scarcity, achieving data balance and correction, while simultaneously reducing data annotation costs and improving data diversity and coverage. By designing appropriate prompt text and using the title and body of each question as examples from the large language model, the quality of the generated question examples is ensured, avoiding illusions and data distortion.

[0064] Furthermore, this invention constructs an intent recognizer through a text embedding model, which fully captures the semantic features of the question text, thereby improving the stability and actual prediction performance of the intent recognizer.

[0065] S201~202 of this embodiment can be referred to S101~S102 of Embodiment 1, and will not be repeated here.

[0066] like Figure 5 The diagram shown is a pseudocode illustration of the offline training stage of the intent recognizer provided in an embodiment of the present invention, which corresponds to the following steps S203 to S207.

[0067] In S203 of this embodiment, since the basic dataset obtained in S202 may have obvious class imbalance, this embodiment filters out the scarce classes in the basic dataset so that the samples of the scarce classes can be expanded in the future.

[0068] Taking the data distribution in Table 2 as an example, among the 1536 questions sampled and labeled, 11 categories have more than 20 samples, accounting for more than 90% of the total samples, while the combined sample size of the remaining 21 categories accounts for less than 10%. This long-tail distribution will severely restrict the ability of the training model to learn effectively from categories with small sample sizes.

[0069] Therefore, this embodiment first preprocesses the samples in the basic dataset by removing non-text content such as code snippets and images, resulting in samples containing only plain text data. Then, it counts the number of samples corresponding to each secondary intent category in the basic dataset and filters out rare categories with fewer than m samples. Here, m is a preset threshold, which can be set to m=20 or adjusted according to actual conditions.

[0070] In S204 of this embodiment, prompt text is designed. For the title and body of each programming technology question in a category with fewer than m samples, a large language model is used to generate n semantically consistent new questions for that programming technology question.

[0071] In some possible embodiments, new problem examples are generated based on the text data in the samples corresponding to the scarce category, using a large language model, including: Extract the title and body text of each question from the samples corresponding to the scarce category; Design prompt text, and use the title and body of each question as examples for the large language model. Input the prompt text and examples into the large language model to obtain multiple question examples generated by the large language model for each question.

[0072] Specifically, for samples corresponding to scarce categories, only the title and body of each question are retained. Based on the question title and body, prompt text is designed. This prompt text, along with the title and body of each question, are input into the large language model. In other words, the title and body of each question serve as input examples for the large language model. The prompt text constrains the generated question examples, and the large language model's learning and reasoning capabilities are used to generate n question examples semantically equivalent to the input questions. Here, n is a data augmentation parameter, i.e., the number of generated question examples. The default value is n=5, a number determined based on experiments. This avoids over-generating pseudo-samples that deviate from the true distribution while ensuring improved model performance. n can also be adjusted according to actual needs.

[0073] In some possible embodiments, the design guidelines for the prompt text include: The generated problem examples strictly adhere to the core intent of the example problem, avoiding deviations from semantics and the introduction of irrelevant information; The generated sample questions maintain the same style as the example questions, including tone, structure, and technical expression. Ensure that the generated problem examples are natural, fluent, and grammatically correct, avoiding factual or logical errors; Limit the number of generated problem samples to a reasonable range.

[0074] like Figure 6 The image shows an example of prompt text (including Chinese and English) provided by an embodiment of the present invention. The title and body of the sample question serve as example inputs for the large language model to learn from and reference, while the prompt text instructs the large language model to generate sample questions that meet the requirements.

[0075] Table 3 Examples of Problems Generated Based on Large Language Models

[0076] Table 3 shows examples of question samples generated based on a large language model (including Chinese and English translations) provided in the embodiments of the present invention. These question samples are generated based on GPT-4o, but can also be generated using large language models such as DeepSeek and Claude. Each question sample includes a title and body text.

[0077] In S205 of this embodiment, the problem example corresponding to each sample in S204 is added to its respective scarce category, and the intent category label of the scarce category is assigned to it, thereby expanding the samples of each scarce category in the basic dataset and obtaining the enhanced dataset.

[0078] Next, an intent recognizer applicable to the 32 secondary intent categories shown in Table 2 is trained using an augmented dataset. This invention treats the intent recognition of programming technical questions as a multi-label classification task. Before training the intent recognizer, multi-label encoding is performed for each question q. Specifically, the intent category label of question q is encoded into a 32-dimensional intent category vector, where each dimension represents whether q belongs to a certain intent category: if q belongs to the c-th intent category, the c-th dimension is marked as 1; otherwise, it is marked as 0.

[0079] In S206 of this embodiment, a text embedding model is used to embed the title and body of each question q to fully capture the semantic features of the question text and generate a semantic vector representation of the programming technology question.

[0080] For example, the text embedding model can use the pre-trained BGE-M3 (BAAI General Embedding-Multilingual, Multi-Functionality, Multi-Granularity) model. Other Transformer models can also be used, such as BERT (Bidirectional Encoder Representations from Transformers), RoBERTa (Robustly Optimized BERT Pretraining Approach), or other classic machine learning models, such as Random Forest and Support Vector Machine (SVM).

[0081] In S207 of this embodiment, the semantic vector representation of the programming technical problem and the corresponding secondary intent category are input into the intent recognizer. The parameters of the text embedding model and the intent recognizer are jointly optimized through the backpropagation algorithm. The mapping relationship between the text semantics of the programming technical problem and the secondary intent category is learned, and an intent recognizer capable of performing multi-label classification tasks is obtained.

[0082] During training, a weighted binary cross-entropy loss function is used, where the weights are calculated based on the sample distribution of each intent category to alleviate the class imbalance problem. At the same time, a cosine annealing learning rate and a weight decay strategy with a fixed coefficient of 0.001 are combined to prevent the model from overfitting and improve its stability.

[0083] In addition, an early stopping mechanism can be set up to terminate training when the performance of the validation set split from the augmented dataset does not improve after three rounds of training, thus obtaining the final programming technique problem intent recognizer based on the text embedding model.

[0084] For example, the intent recognizer may employ a sigmoid-based linear classification layer.

[0085] In S208 of this embodiment, the trained intent recognizer is applied to any given programming technical problem p to determine its intent category.

[0086] like Figure 7 The diagram shown is a pseudocode illustration of the online application stage of the intent recognizer provided in an embodiment of the present invention.

[0087] First, the text data of the title and body of the programming technical question p is extracted. Then, the extracted text data is embedded using BGE-M3 trained on S207 to obtain its semantic vector representation. This semantic vector representation is used as input to the intent recognizer to obtain the intent category of question p. It is worth noting that the final identified intent category of question p may be zero, one, or more. This result can support the organization or answering of questions. Questions with a predicted category of zero are often those with brief descriptions, vague semantics, or lacking obvious technical features, making it difficult for the intent recognizer to extract sufficient semantic information for accurate intent identification. Questions with a predicted category of one usually have clear goals and technical context. Questions with multiple predicted categories involve cross-intent content, reflecting the sensitivity and flexibility of the intent recognizer of this invention when facing multiple intent expressions.

[0088] The technical solution of the present invention will be verified below by combining benchmark comparison experiments and ablation experiments.

[0089] I. Benchmark Comparison Experiment To evaluate the effectiveness of the proposed method, 1,152 SO questions out of the 1,536 questions listed in Table 2 were used as the training set, and the remaining 384 questions were used as the test set. A programming technical question intent recognizer based on the text embedding model BGE-M3 was trained for identifying the intent of programming technical questions. The intent recognition method for programming technical questions proposed in this invention is denoted as TQIR (Technical Question Intent Recognition). Five commonly used and high-performing text classification models were selected as baseline models for comparison, including the traditional machine learning model SVM, the deep learning model LSTM, and pre-trained language models BERT, DeBERTa, and RoBERTa. Accuracy, precision, recall, and F1 score were used as evaluation metrics for the classification performance of these methods.

[0090] Considering the imbalanced class distribution in labeled datasets, weighted and micro forms of precision, recall, and F1-score are introduced, namely, weighted precision (…). ), micro-precision ( ), weighted recall rate ( Micro-recall rate ), weighted F1 score ( ), Microscopic F1-score ( This allows for a more comprehensive and reliable evaluation of the overall performance of the intent recognizer across different categories.

[0091] The calculation formulas for each evaluation indicator are as follows:

[0092]

[0093]

[0094]

[0095]

[0096]

[0097]

[0098] in, The total number of intent categories, These are the intent categories. The number of true positives, false positives, true negatives, and false negatives; Representative Intent Category The accuracy rate Representative Intent Category Recall rate, Representative Intent Category of Fraction:

[0099]

[0100]

[0101] This invention uses the Wilcoxon rank-sum test to examine whether the performance differences between TQIR and each baseline method are statistically significant, and the results are shown in Table 4. Table 4 shows the performance comparison between TQIR and the six baseline models, where statistical significance is marked by *, where * represents a significance index P<0.05, and ** represents P<0.02.

[0102] Table 4 Performance comparison between TQIR and baseline models

[0103] The experimental results in Table 4 demonstrate that TQIR outperforms traditional machine learning, deep learning, and pre-trained language models across all evaluation metrics. Compared to the best-performing baseline model, RoBERTa, TQIR achieves improvements of approximately 11.4%, 2.6%, and 7% in weighted precision, recall, and F1 score, respectively. In terms of micro-metrics, TQIR achieves improvements of approximately 14.6%, 2.6%, and 8.5% in precision, recall, and F1 score, respectively, validating the superior performance of the TQIR proposed in this invention.

[0104] II. Ablation Experiment To verify the effect of the LLM-based data augmentation mechanism on the proposed intent recognition method TQIR, ablation experiments were conducted. Table 5 shows the ablation experiment results for TQIR with and without data augmentation.

[0105] Table 5 Ablation Experiments of TQIR

[0106] As shown in Table 5, the TQIR data augmentation method using a large language model in this invention improves the performance of the intent recognition model across all evaluation metrics: the weighted F1 score increases from 0.865 to 0.874, and the micro F1 score increases from 0.858 to 0.869. Specifically, the model achieves relative improvements of 1.04% and 1.28% in the weighted F1 score and micro F1 score, respectively. The most significant improvement is in precision, with weighted precision and micro precision increasing by 2.04% and 2.21%, respectively. Recall and overall precision also steadily improve, with increases of 0.47% and 0.31%, respectively, demonstrating the effectiveness of the data augmentation mechanism proposed in this invention.

[0107] Example 3 Please see Figure 8 , Figure 8 A schematic diagram of an intent recognition system for programming technology problems provided as an embodiment of the present invention, the system comprising: The data collection module 810 is used to collect and filter programming technical issues on the target platform; The data annotation module 820 is used to construct a multi-level intent classification system. It uses iterative statistical sampling to annotate the intent of programming technical problems and obtain the basic dataset. The data augmentation module 830 is used to augment samples in the basic dataset based on a large language model to obtain an augmented dataset. The intent recognition module 840 is used to train an intent recognizer using an augmented dataset and to recognize the intent of programming technical questions using the trained intent recognizer.

[0108] The intent recognition system for a programming technology problem described above and the intent recognition methods for a programming technology problem described in the above method embodiments can be referred to each other, and will not be repeated here.

[0109] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include a processor 910, a communications interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communications interface 920, and the memory 930 communicate with each other via the communication bus 940. The processor 910 can call logical instructions in the memory 930 to execute an intent recognition method for a programming technical problem provided in the above-described method embodiments.

[0110] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to execute an intent recognition method for a programming technical problem provided by the above-described method embodiments.

[0112] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform an intent recognition method for a programming technique problem provided in the above-described method embodiments.

[0113] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying the intent of a programming technology problem, characterized in that, include: Collect and filter programming technical issues for the target platform; An iterative statistical sampling method is used to annotate the programming technical problems and a multi-level intent classification system is constructed to obtain the basic dataset; Data augmentation is performed on samples in the base dataset based on a large language model to obtain an augmented dataset. The intent recognizer is trained using the augmented dataset, and the trained intent recognizer is used to identify the intent of the programming technique problem under test.

2. The intent recognition method for programming technology problems according to claim 1, characterized in that, The multi-level intent classification system includes primary intent categories and secondary intent categories corresponding to each primary intent category; The primary intent categories include general search, debugging and bug fixing, programming, third-party libraries / services / frameworks / platforms, tools, databases, and testing; The secondary intent categories corresponding to the regular search include: explanations of unknown terms, best practices in searching, differences between search technologies, and explanations of problems with search technologies; The secondary intent categories corresponding to debugging and bug fixing include: searching for explanations of anomalies / warnings / error messages, searching for solutions to common errors, searching for solutions to software configuration errors, searching for solutions to security-related errors, searching for solutions to performance problems, searching for solutions to multithreading problems, and searching for solutions to abnormal results; The secondary intent categories corresponding to the programming include: searching for examples or guidance on how to use specific features of a programming language, searching for standards, searching for examples or guidance on how to avoid anti-patterns, searching for pseudocode of an algorithm, code examples or algorithm principles, searching for alternatives or best solutions / techniques, searching for solutions to specific tasks, and searching for differences between methods. The secondary intent categories corresponding to the third-party libraries / services / frameworks / platforms include: searching for third-party libraries / services / frameworks / platforms, searching for usage examples or examples of third-party libraries / services / frameworks / platforms, and searching for examples of how to implement interactive operations between third-party libraries / services / frameworks / platforms. The secondary intent categories corresponding to the tools include: searching for examples and guidance on how to use the tools, searching for examples and guidance on how to configure the tools, searching for examples and guidance on how to upgrade / downgrade / install the tools, and searching for solutions to tool problems; The secondary intent categories corresponding to the database include: searching for examples or guidance on how to formulate structured query statements, searching for examples or guidance on how to use non-relational databases, and searching for database optimization solutions; The secondary intent categories corresponding to the tests include: searching for test method examples or guidance, searching for usage examples or guidance on how to use automated testing tools, and searching for test methods for specific tasks.

3. The intent recognition method for programming technology problems according to claim 1, characterized in that, The data augmentation of samples in the base dataset based on the large language model yields an augmented dataset, including: Extract the scarce categories from the basic dataset whose sample count is less than a preset threshold; Based on the text data in the samples corresponding to the scarce categories, new problem examples are generated using a large language model. Based on the problem examples, the scarce category is augmented to obtain an enhanced dataset.

4. The intent recognition method for programming technology problems according to claim 3, characterized in that, The step of generating new problem examples based on the text data in the samples corresponding to the scarce category and using a large language model includes: Extract the title and body text of each question from the samples corresponding to the scarce category; Design prompt text, and use the title and body of each question as examples in the large language model. Input the prompt text and the examples into the large language model to obtain multiple question examples generated by the large language model for each question.

5. The intent recognition method for programming technology problems according to claim 4, characterized in that, The design principles for the prompt text include: The generated problem examples strictly adhere to the core intent of the example problem, avoiding deviations from semantics and the introduction of irrelevant information; The generated sample questions maintain the same style as the example questions, including tone, structure, and technical expression. Ensure that the generated problem examples are natural, fluent, and grammatically correct, avoiding factual or logical errors; Limit the number of generated problem samples to a reasonable range.

6. The intent recognition method for programming technology problems according to claim 1, characterized in that, The process of training the intent recognizer using the augmented dataset includes: The title and body of the sample questions in the augmented dataset are embedded using a text embedding model to generate semantic vector representations. The semantic vector representing the intent category corresponding to the sample question is input into the intent recognizer. The intent recognizer outputs the predicted intent category, and the parameters of the text embedding model and the intent recognizer are jointly optimized through the backpropagation algorithm to obtain the trained intent recognizer.

7. The intent recognition method for programming technology problems according to claim 6, characterized in that, The text embedding model includes the BGE-M3 model.

8. A method for identifying the intent of a programming technology problem, characterized in that, include: The data collection module is used to collect and filter programming technical issues on the target platform; The data annotation module is used to annotate the programming technical problems using iterative statistical sampling and to construct a multi-level intent classification system to obtain the basic dataset. The data augmentation module is used to augment the samples in the base dataset based on the large language model to obtain an augmented dataset; The intent recognition module is used to train an intent recognizer using the augmented dataset and to recognize the intent of the programming technology problem to be tested using the trained intent recognizer.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the intent recognition method for programming technical problems as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the intent recognition method for programming technical problems as described in any one of claims 1 to 7.

Images