A knowledge-adaptive code retrieval model, method and system
By building a knowledge-adaptive code retrieval model, the problem of ignoring code language-specific knowledge in the existing technology is solved, and more accurate and efficient code retrieval and generation are achieved.
Patent Information
- Application Number
- CN202510617011.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-14
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2045-05-14
AI Technical Summary
In existing code retrieval tasks, retrieval methods for multiple code languages ignore the unique knowledge of each code language, resulting in insufficient retrieval accuracy and consistency, and fail to select the appropriate code language for user questions, affecting the accuracy of code generation.
Build a knowledge-adaptive code retrieval model, including a code language selection module, a code knowledge module, a general knowledge module, and a feature fusion module. Select the appropriate code language by training the model, learn the unique and general knowledge of each code language, and combine the feature fusion module to improve retrieval accuracy.
By adaptively selecting code languages and learning unique knowledge, the accuracy and efficiency of code retrieval are improved, the relevance of generated code to natural language is ensured, and the accuracy of code generation is improved.
Smart Images
Figure CN120470108B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of code generation, and in particular to a knowledge-adaptive code retrieval model, method and system. Background Art
[0002] While pre-trained large language models, such as the GPT series, demonstrate impressive code generation capabilities, they are prone to "hallucination" issues when dealing with proprietary, recent, or domain-specific code, resulting in a lack of accuracy and consistency in generated content. To address these issues, code retrieval augmented generation technology retrieves code snippets highly relevant to natural language from a code knowledge base. These snippets are then incorporated into the prompts of the large language model to assist in generating more accurate and contextually relevant code. Code retrieval augmented generation technology addresses the limitations of the large model's own knowledge while simultaneously introducing the latest and proprietary information in real time during the code generation process, thereby improving code generation accuracy. Therefore, in code retrieval, accurate code retrieval in real time is crucial, especially in code retrieval tasks involving multiple languages. The more relevant the retrieved code is to the natural language, the more accurate the code generated by the large language model will be.
[0003] Existing code retrieval tasks typically use the same embedding model to extract feature embeddings for multiple code languages, neglecting to learn the unique knowledge of each code language, such as its unique grammatical structure and logical functions, which affects the accuracy of code retrieval tasks. Furthermore, each code language excels at solving different problem domains. Existing code retrieval methods fail to select the appropriate code language for the user's question, which indirectly affects the accuracy of code retrieval tasks. Summary of the Invention
[0004] In order to overcome the defect of low code retrieval accuracy in the above-mentioned prior art, the present invention proposes a training method for a knowledge-adaptive code retrieval model, which can be trained to obtain a code retrieval model that takes into account the differences of each code language and the common knowledge of different languages, greatly improving the accuracy and efficiency of code retrieval.
[0005] The present invention proposes a training method for a knowledge-adaptive code retrieval model. First, a retrieval model is constructed. The retrieval model includes a code language selection module, a code knowledge module, a general knowledge module, and a feature fusion module. The code language selection module and the feature fusion module are respectively connected to the code knowledge module and the general knowledge module.
[0006] The code language selection module outputs the probability distribution of user questions adapted to each code language based on the text features of the user questions; the code knowledge module corresponds to each code language one by one and is used to process code features and text features by combining the knowledge of the corresponding language; the general knowledge module combines general knowledge to process code features and text features;
[0007] The feature fusion module combines the code features output by the code knowledge module with the probability distribution, and then fuses them with the question features output by the general knowledge module to output text fusion features. It also fuses the code features with the code features output by the general knowledge module to output code fusion features.
[0008] The retrieval model is trained on the dataset {user questions, code snippets}. First, the loss is calculated by combining the true code language labels with the probability distribution output by the code language selection module to train the code language selection module.
[0009] Then fix the code language selection module, combine the distance loss L1 of general knowledge learning and code knowledge learning, and the contrast loss L2 of text feature fusion and code feature fusion to update the code knowledge module, general knowledge module and text feature fusion module.
[0010] Preferably, the distance loss is:
[0011]
[0012] Where N is a type of all code languages, and The training samples of the i-th code language are output through the text features of the corresponding code knowledge module and general knowledge module respectively. and The training samples of the i-th code language are respectively output by the code features of the corresponding code knowledge module and the general knowledge module.
[0013] Preferably, the contrast loss is:
[0014]
[0015] Where K represents the training sample set of the current round, |K| represents the total number of samples in K; N is the total number of code languages in K; t i The true label is code snippet c i User issues; The feature fusion module is used to solve the user question t i Output text fusion features, For the feature fusion module, target code snippet c i Output code fusion features; The feature fusion module is used to solve the user question t mOutput text fusion features, For the feature fusion module, target code snippet c m Output code fusion features, t m and c m From the positive sample (t i ,c i )’s negative sample set M; Represents code snippet c i The initial code features are processed by the code knowledge module and the general knowledge module corresponding to language j and then fused into code fusion features, which are used as negative examples; the initial code features are code features extracted from the code fragment through the pre-trained text embedding model.
[0016] Preferably, token-level features of user questions are extracted through a pre-trained text embedding module; the code language selection module includes sequentially connected: a correlation attention network, an overlay network, and a classification unit;
[0017] For each learnable parameter corresponding to a specific code language, a related attention network is used to calculate the similarity between each token in the token-level text features and the learnable parameter, extracting multiple activations with the largest similarity as the enhancement coefficient. The output of the related attention network for the learnable parameter is the product of the enhancement coefficient and the text feature. The initial value of the learnable parameter is the mean code feature of the code snippet in the specified code language.
[0018] The superposition network superimposes the learnable parameters and the output of the relevant attention network and inputs them into the classification unit. The classification unit outputs the probability distribution of user questions applicable to various code languages.
[0019] Preferably, the classification unit is composed of multiple layers of fully connected layers.
[0020] Preferably, cross entropy loss is used to update learnable parameters and classification units during the training process of the code language selection module.
[0021] The present invention proposes a knowledge-adaptive code retrieval method, comprising the following steps:
[0022] Constructing a code generation model, wherein the code generation model generates a code snippet representing the user problem based on the user problem and the reference code; and constructing a code retrieval model using the training method of the knowledge-adaptive code retrieval model;
[0023] Obtain user questions. The code retrieval model processes the user questions and outputs text fusion features through the feature fusion module. The code languages corresponding to the larger H1 probabilities are extracted from the probability distribution output by the code language selection module of the code retrieval model as candidate languages.
[0024] Retrieve H2 code snippets with the greatest similarity to the text fusion features from the code snippets corresponding to each candidate language as reference codes;
[0025] Input user questions and reference codes into the code generation model, obtain the code snippets output by the code generation model and execute them.
[0026] Preferably, the code generation model is a neural network model trained using a machine learning method.
[0027] The present invention proposes a knowledge adaptive code retrieval system, which includes a memory and a processor. The memory stores a computer program. The processor is connected to the memory and is used to execute the computer program to implement the knowledge adaptive code retrieval method.
[0028] The present invention proposes a storage medium storing a computer program, which is used to implement the knowledge adaptive code retrieval method when executed.
[0029] The advantages of the present invention are:
[0030] (1) In the knowledge-adaptive code retrieval model proposed in this invention, the code language selection module selects the appropriate code language for different user questions, the various code knowledge modules learn the unique knowledge of each code language, and the general knowledge module captures user intent or general knowledge of various code languages. In this way, through the interaction between the general knowledge module and the various code knowledge modules, the learned feature embedding is ensured to be richer and more professional, thereby improving the accuracy of the code retrieval task. The code language selection module adaptively selects the appropriate code language for user questions and considers the applicability and degree of applicability of each code language, which is conducive to improving the accuracy of code retrieval.
[0031] (2) The present invention takes into account the differences between each code language and designs a general knowledge module and a code knowledge module. Through the effect of distance loss, it ensures that these two knowledge modules learn general knowledge and unique knowledge respectively. That is, the learned knowledge is independent of each other, thereby ensuring comprehensive reasoning of user questions and improving the accuracy and efficiency of code retrieval;
[0032] (3) The code language selection module adaptively selects the appropriate code language for each user question. This module implicitly learns a feature for each code language and enhances the code feature by correlating the code feature with the token in the user question, which helps to select a more appropriate code language for each question.
[0033] (4) The feature fusion module considers the applicability of all code languages when fusing text features, and further promotes the learning of code knowledge modules and general knowledge modules through contrast loss in the fused feature space.
[0034] (5) The code language selection module of the present invention introduces learnable parameters and combines them with the text features extracted by the pre-trained text embedding module, which greatly improves the effect of code retrieval.
[0035] (6) The knowledge-adaptive code retrieval method proposed in the present invention combines a knowledge-adaptive code retrieval model to retrieve reference code, and a code generation model combines the reference code to generate code. The present invention improves the code retrieval effect and ensures the accuracy of code generation. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 This is the structural diagram of the knowledge adaptive code retrieval model proposed by the present invention;
[0037] Figure 2 This is a flow chart of the training method for the code language selection module proposed in the present invention;
[0038] Figure 3 This is a flow chart of the knowledge-adaptive code retrieval model training method proposed in the present invention. DETAILED DESCRIPTION
[0039] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.
[0040] Reference Figure 1 、 Figure 2 ,The present invention proposes a knowledge adaptive code retrieval model including: a text embedding module, a code language selection module, a code knowledge module, a general knowledge module and a feature fusion module;
[0041] The text embedding module is used to extract the text features of user questions and the code features of code snippets in the code library; the code library corresponds one-to-one with the code language type.
[0042] In this invention, the text embedding module adopts the general text embedding model BGE-M3, which is trained on large-scale text datasets, supports multi-language and cross-language retrieval, achieves excellent performance in various retrieval tasks, and has strong generalization.
[0043] Assume there are {1,…,i,…N} coding languages, such as go, java, javascript, php, python, ruby, etc. For any user question t and code snippet c i ,have:
[0044]
[0045] f cp =W BGE-M3 (c i ); (1.2)
[0046] Among them, the code fragment c i The subscript i indicates the code language; W BGE-M3 represents the model weight of the pre-trained BGE-M3, f tp Indicates that user problem t passes through W BGE-M3 After that, the text features Indicates that user problem t passes through W BGE-M3 The subsequent token-level text features, f cp Indicates that the code snippet passes through W BGE-M3 After the initial code features, M1 is the number of tokens in the user question, D is the feature dimension; N is the number of code language types.
[0047] The code language selection module adaptively selects the appropriate code language based on the user's question, thereby quickly matching it to the corresponding code library and improving the accuracy and efficiency of retrieval.
[0048] The code language selection module includes sequentially connected: correlation attention network, superposition network and classification unit;
[0049] The relevant attention network is combined with learnable parameters to analyze the text features f of the user question. tp Enhancement is performed, and the enhanced features and learnable parameters are superimposed through the superposition network and then converted into language probability distribution through the classification unit, that is, the probability that the user question is applicable to each code language.
[0050] The learnable parameters are the reference arrays to be trained for the relevant attention network and correspond one-to-one to the code language. The initial value of the learnable parameters is the initial code feature f extracted by the text embedding module from the code snippet in the specified code language. cp The mean of .
[0051] Related attention network is used to calculate token-level text features The similarity between each token and the learnable parameters in , for each learnable parameter f cl , extract the similarity array with similarity greater than the threshold δ and activate it as the enhancement coefficient; then calculate the enhancement coefficient and token-level text features The product of is used as the enhanced feature of the output of the relevant attention network.
[0052] Specifically, learnable parameters N is the number of code languages, and D is the feature dimension; M1 is the number of tokens in the user's question.
[0053] In this embodiment, the learnable parameter f cl The introduction of enables the code language selection module to adaptively select the appropriate code language based on user questions, thereby quickly matching it to the corresponding code knowledge base and improving the accuracy and efficiency of retrieval.
[0054] The processing of the relevant attention network is expressed as follows:
[0055]
[0056] in, express An array of values selected from the set that are greater than the threshold δ, Represents token-level text features Each token and the learnable parameter f cl An array of similarity values, sigmoid represents the activation function; Token-level text features representing user questions The transpose of Indicates the combination of f cl right Enhanced features.
[0057] The processing process of the overlay network is expressed as follows:
[0058]
[0059] in, Represents the output of the superposition network; α is the set balance parameter used to balance and The importance of the function is between 0 and 1.
[0060] The classification unit is implemented using a 3-layer FC network. The activation function of the FC network can be LeakyReLU, represented by ActFun(). The working principle of the classification unit is as follows:
[0061]
[0062] in, and Represent the learnable parameters of the 3-layer FC network, Fit learnable features f to user questions cp The probability of the corresponding code language;
[0063] The code knowledge modules correspond to the code languages one by one, that is, the code language corresponding to each code knowledge module is determined, and each code fragment has a unique corresponding code knowledge module.
[0064] The code knowledge module processes the initial code features f by learning the knowledge of the corresponding language cp and text features f tp .
[0065] The code knowledge module consists of a 2-layer FC network, and the initial code feature f cp The code knowledge module processes the initial code feature f cp The processing process is expressed as follows:
[0066]
[0067] and are the weights of the 2-layer FC network in the code knowledge module of the i-th code language, i is the initial code feature f cp The corresponding language number, f cp Code features processed by the corresponding code knowledge module.
[0068] The code knowledge module is used to analyze the text features f tp The processing process is expressed as follows:
[0069]
[0070] in, f tp The text features after passing through the code knowledge module of the i-th code language.
[0071] The general knowledge module is used to learn general knowledge about user questions and code snippets. It is also composed of a two-layer FC network and is expressed as follows:
[0072]
[0073] in, and represents the weight of the general knowledge module, is f cp The code features after the general knowledge module, is f tp Problem characteristics after the general knowledge module.
[0074] The code language selection module is used to select the appropriate code language for different problems, the various knowledge networks are used to learn the unique knowledge of each code language, and the general knowledge module is used to capture user intentions or the general knowledge of various code languages. In this way, through the interaction between the general knowledge module and various knowledge networks, the learning of general knowledge and unique knowledge can be guaranteed.
[0075] The feature fusion module is used to combine the text features output by each code knowledge module Combine the probability distribution output by the code language module for weighted fusion and then combine it with the problem characteristics output by the general knowledge module Fusion is performed; the formula is expressed as:
[0076]
[0077] in, is the text feature output by the code knowledge module of the i-th code language, is the probability that the user question corresponds to code language i, The output of each code knowledge module is combined with the weighted fusion result of probability distribution; Problem features output by the general knowledge module; It is the text fusion feature output by the feature fusion module.
[0078] For any user question, it has a clear code label during training, and it is known which code knowledge module is used to obtain feature embedding. However, when inferring the code label, the code knowledge module is uncertain, and a user question is not limited to one code language. Therefore, during training, the text features output by all code knowledge modules are integrated, and the importance of different code language knowledge modules in the final question features is determined based on the output of the code language selection module, which is conducive to improving the comprehensiveness and correctness of reasoning. At the same time, considering that user questions may contain some common information, such as user intentions, and that the code may contain common knowledge of different code languages, the present invention introduces a common knowledge module to independently process common information and common knowledge, ensuring that each code knowledge module can learn the unique knowledge of each code while not being affected by these common information or common knowledge, further improving the correctness of code retrieval.
[0079] The feature fusion module is also used to fuse the code features of known languages with the code features output by the general knowledge module. The formula is expressed as:
[0080]
[0081] in, The code features output by the code knowledge module corresponding to the code language i; is the code feature after the general knowledge module, The code fusion features output by the feature fusion module.
[0082] The feature fusion module fuses the output of the code knowledge module with the output of the general knowledge module to ensure that the learned features contain both general knowledge and unique knowledge.
[0083] The model training method includes the following steps:
[0084] First, we train the code language selection module on the dataset {user question, code language} as follows:
[0085] S11. Select training samples from the data set and let the code language selection module select the code language based on the text feature f tp and token-level text features Output probability distribution, combined with the probability of the code language of the true label to calculate the loss function L CE , and update the relevant attention network of the code language selection module and the 3-layer FC network of the classification unit according to the loss function. The updated parameters include: f cl 、 and
[0086]
[0087] in, is a binary number. If the true label is code language i, then Otherwise it is 0; The probability that the user question output by the code language selection module corresponds to code language i;
[0088] S12. Repeat the above step S11 until the code language selection module converges.
[0089] After the code language selection module is fixed, the code knowledge module and the general knowledge module are trained on the dataset {user question, code snippet}. The specific steps are as follows:
[0090] Under the action of βL1+γL2 loss function, each code knowledge module and general knowledge modules The parameters are iteratively updated until the model converges and the optimal model parameters are obtained. β,γ∈[0,1] are balance parameters.
[0091] In the second stage, the model processes the samples as follows: extract the text features f of the user questions in the training samples through the text embedding module. tp and token-level text features Let the code language selection module output the probability distribution of each code type based on text features Each code knowledge module has a certain effect on the text feature f tp Process and output features The text feature fusion module combines the code type probability distribution with the output of each code knowledge module Do weighted fusion and then add the general knowledge module to the text feature f tp Output As the final text fusion feature The initial code features f of the code snippet of the sample are extracted through the text embedding module cp , according to the code type, use the corresponding code knowledge module to process the initial code features to obtain features At the same time, the general knowledge module is also called to analyze the initial code features f cp Processing to obtain features Superimpose the above two processing results to obtain the final code fusion feature
[0092] The loss function L1 is used to characterize the distance loss between general knowledge learning and code knowledge learning, ensuring that the knowledge learned by the general knowledge module and the code knowledge module are independent of each other, and that the input features are distanced from each other after being processed by the general knowledge module and the code knowledge module, thereby improving the learning accuracy of the code knowledge module for specific knowledge and the learning accuracy of the general knowledge module for shared information.
[0093] L2 is used to characterize the contrast loss of text feature fusion and code feature fusion. In the feature space of the fused features, the distance between positive sample pairs needs to be shortened, while the distance between negative sample pairs needs to be increased. Negative sample pairs here include two types: the first type has the same problem but different code; the second type has the same problem and code but different code knowledge modules. These are code knowledge-level negative samples, which ensure the professionalism of the code knowledge module.
[0094] The loss function is:
[0095]
[0096] Where N is the total number of code languages. is the text feature output by the code knowledge module of the training sample of the i-th code language, is the problem feature output by the general knowledge module, is the code feature output by the corresponding code knowledge module of the training sample corresponding to the code language i, Text features output by the general knowledge module;
[0097] It is worth noting that when there are multiple training samples corresponding to the same code language, the loss function L1 is the mean text feature of the training sample of the labeled code language i, and the loss function L1 is the mean code feature value of the training samples labeled with code language i;
[0098] The loss function L1 and Corresponding to each other, that is Represents the text features of the training sample of the labeled code language i output by the corresponding code knowledge module Mean and text features after general knowledge module product of means; Represents the code features output by the corresponding code knowledge module for the training sample of the labeled code language i Mean and code features through general knowledge modules The product of the means.
[0099] Let the model predict labels into two categories: (t i ,c i ) represents the positive sample pair with label i, user question t i The true label is code ci, i is the code type; (t i ,c m ) represents a negative sample pair with label m and m≠i;
[0100]
[0101] Where K represents the training sample set of the current batch, |K| represents the total number of samples in K; N is the total number of code languages; t i The true label is code snippet c i User issues; The feature fusion module is used to solve the user question t i Output text fusion features, For the feature fusion module, target code snippet c i Output code fusion features; M represents t i The negative sample set, that is, M is the set of training samples (t i ,c i ) other than the sample set; The feature fusion module is used to solve the user question t m Output text fusion features, For the feature fusion module, target code snippet c m Output code fusion features, t m and c m From the negative sample set M; For code snippet c iNegative examples at the knowledge network level are constructed using features obtained from code knowledge modules in different languages. Represents code snippet c i The initial code features of language j are processed by the code knowledge module and the general knowledge module respectively, and then fused into code fusion features. The code fusion features are used as negative examples. The initial code features are the code features f extracted from the code fragment through the pre-trained text embedding model. cp .
[0102] N is the total number of code languages.
[0103] t i The negative sample set contains the negative samples relative to t i Different codes of the same code language also include codes of different languages; the formula is expressed as:
[0104] t i The positive sample is (t i ,c i )∈K, its corresponding negative sample set Indicates that the code type is also i but c i Different codes.
[0105] During the second phase of training:
[0106] S21, initializing the code knowledge module and general knowledge module of the model;
[0107] S22. Select samples from the dataset {user questions, code snippets} and input them into the model, calculate the loss function βL1+γL2, and update each code knowledge module and general knowledge module according to the loss function;
[0108] S23. Repeat step S22 until the model converges.
[0109] Reference Figure 2 ,This embodiment also proposes a knowledge-adaptive code retrieval method, comprising the following steps:
[0110] Construct a code generation model and the above-mentioned code retrieval model; the code generation model generates a code snippet representing the user problem based on the user problem and the reference code; the code generation model is a neural network model trained using a machine learning method.
[0111] A corresponding code library is established for all codes. N code libraries are established for N code languages. When establishing the library for each code language, text features are extracted based on the text embedding module (i.e., the BGE-M3 pre-trained model). The corresponding code knowledge module and the general knowledge module extract code features and fuse them to obtain code fusion features. For details, see formulas 1.2, 3.1, 4.1, and 5.3.
[0112] For any input user question, we first extract pre-trained features based on the text embedding module (see Formula 1.1). Then, we use the code language selection module to obtain the probability of each code language (see Formula 2.3). Based on this probability, we select the top three corresponding code languages.
[0113] The feature fusion module fuses the code features output by the code knowledge module according to the probability of each code language, and then fuses them with the text features output by the general knowledge module to output text fusion features. See formulas 5.1 and 5.2;
[0114] Fusion of text features Search the code bases corresponding to the top 3 respectively, and obtain the top 3 similar codes in each code base as reference codes based on cosine similarity;
[0115] Input user questions and reference codes into the code generation model, obtain the code snippets output by the code generation model and execute them.
[0116] Of course, it will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, but also encompasses the same or similar structures that can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and it is intended that all variations that fall within the meaning and range of equivalents of the claims be encompassed within the present invention. Any reference signs in the claims should not be construed as limiting the claim to which they relate.
[0117] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.
[0118] The technology, shape, and structure not described in detail in the present invention are all well-known technologies.
Claims
1. A training method for a knowledge-adaptive code retrieval model, characterized in that: First, a retrieval model is constructed, which includes a code language selection module, a code knowledge module, a general knowledge module, and a feature fusion module. The code language selection module and the feature fusion module are connected to the code knowledge module and the general knowledge module respectively. The code language selection module outputs the probability distribution of user questions adapted to each code language based on the text features of the user questions; the code knowledge module corresponds to each code language one by one and is used to process code features and text features by combining the knowledge of the corresponding language; the general knowledge module combines general knowledge to process code features and text features; The feature fusion module combines the code features output by the code knowledge module with the probability distribution, and then fuses them with the question features output by the general knowledge module to output text fusion features; It also fuses the code features with the code features output by the general knowledge module to output code fusion features; The retrieval model is trained on the dataset {user questions, code snippets}. First, the loss is calculated by combining the true code language labels with the probability distribution output by the code language selection module to train the code language selection module. Then fix the code language selection module, combine the distance loss L1 of general knowledge learning and code knowledge learning, and the contrast loss L2 of text feature fusion and code feature fusion to update the code knowledge module, general knowledge module and text feature fusion module; Where N is a type of all code languages, and The training samples of the i-th code language are output through the text features of the corresponding code knowledge module and general knowledge module respectively. and The training samples of the i-th code language are output by the code features of the corresponding code knowledge module and the general knowledge module respectively; Where K represents the training sample set of the current round, |K| represents the total number of samples in K; N is the total number of code languages in K; t i The true label is code snippet c i User issues; The feature fusion module is used to solve the user question t i Output text fusion features, For the feature fusion module, target code snippet c i Output code fusion features; The feature fusion module targets user questions t m Output text fusion features, For the feature fusion module, target code snippet c m Output code fusion features, t m and c m From the positive sample (t i ,c i )’s negative sample set M; Represents code snippet c i The initial code features are processed by the code knowledge module and the general knowledge module corresponding to language j and then fused into code fusion features, which are used as negative examples; the initial code features are code features extracted from the code fragment through the pre-trained text embedding model.
2. The method for training a knowledge-adaptive code retrieval model according to claim 1, wherein: User questions are trained to extract token-level features through a pre-trained text embedding module. The code language selection module consists of a sequentially connected: correlation attention network, overlay network, and classification unit. For each learnable parameter corresponding to a coding language, the relevant attention network is used to calculate the similarity between each token in the token-level text features and the learnable parameter, and multiple similarity activations greater than a threshold δ are extracted as enhancement coefficients. The output of the relevant attention network for the learnable parameter is the product of the enhancement coefficient and the text feature. The initial value of the learnable parameter is the mean value of the code features of the code snippet in the specified code language; The superposition network superimposes the learnable parameters and the output of the relevant attention network and inputs them into the classification unit. The classification unit outputs the probability distribution of user questions applicable to various code languages.
3. The training method of the knowledge-adaptive code retrieval model according to claim 2, characterized in that: The classification unit consists of multiple layers of fully connected layers.
4. The method for training a knowledge-adaptive code retrieval model according to claim 2, wherein: During the training process of the code language selection module, cross entropy loss is used to update the learnable parameters and classification units.
5. A knowledge adaptive code retrieval method using the training method of the knowledge adaptive code retrieval model according to any one of claims 1 to 4, characterized in that: The following steps are involved: Build a code generation model. The code generation model generates code snippets that represent user problems based on user problems and reference codes. Constructing a code retrieval model using the training method of the knowledge-adaptive code retrieval model according to any one of claims 1 to 4; Obtain user questions. The code retrieval model processes the user questions and outputs text fusion features through the feature fusion module. Extract the code languages corresponding to the top H1 probabilities arranged from the largest to the smallest from the probability distribution output by the code language selection module of the code retrieval model as candidate languages; Retrieve H2 code snippets with the greatest similarity to the text fusion features from the code snippets corresponding to each candidate language as reference codes; Input user questions and reference codes into the code generation model, obtain the code snippets output by the code generation model and execute them.
6. The knowledge adaptive code retrieval method according to claim 5, characterized in that: The code generation model is a neural network model trained using machine learning methods.
7. A knowledge-adaptive code retrieval system, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, the processor is connected to the memory, and the processor is used to execute the computer program to implement the knowledge adaptive code retrieval method as claimed in claim 5 or 6.
8. A storage medium, characterized in that: A computer program is stored, and when the computer program is executed, it is used to implement the knowledge adaptive code retrieval method according to claim 5 or 6.