Cross-natural language code retrieval model training method, cross-natural language code retrieval method, device, equipment and medium

Abstract

The application discloses a cross-natural language code retrieval model training method, a cross-natural language code retrieval method, a device, equipment and a medium, and relates to the technical field of artificial intelligence and software engineering. The cross-natural language code retrieval model training method comprises the following steps: obtaining an original corpus database, and constructing training data according to the original corpus database; performing confusion and inversion on main language codes to obtain main language code samples, wherein the main language code samples comprise main language code positive samples and main language code negative samples; and training an initial model through a gradient inversion layer according to the training data and the main language code samples to obtain a target model. According to the application, the natural language-specific "fingerprint" features in the codes can be removed, the embedding space alignment direction can be unified, the sampling distribution deviation in the training process can be reduced, and the consistency and generalization capability of cross-language code retrieval can be improved.

Description

Technical Field

[0001] This application relates to the interdisciplinary field of artificial intelligence and software engineering, and in particular to a cross-natural language code retrieval model training method, cross-natural language code retrieval method, device, equipment and medium. Background Technology

[0002] As enterprise software systems continue to grow in scale, multi-language collaborative development has become the mainstream model. Developers need to perform semantic searches across heterogeneous code repositories to achieve tasks such as function reuse, interface location, and logic lookup. This places higher demands on the accuracy and consistency of cross-natural language code retrieval technologies.

[0003] Currently, deep learning technology has made significant progress in the field of code retrieval, with contrastive learning frameworks being a common method for cross-natural language code retrieval. However, existing methods have significant limitations: on the one hand, models easily learn language-specific "fingerprint" features of natural languages. These features are unrelated to the core semantics of the code, leading to independent clusters of code with the same function in the vector space due to natural language categories. This results in inconsistent alignment directions in the embedding space, creating language barriers and poor consistency in cross-natural language retrieval results. On the other hand, existing methods mostly employ a single-language batch training paradigm. The empirical distribution of training samples deviates significantly from the real multilingual joint distribution. Models are prone to overfitting to high-resource languages ​​and experiencing performance degradation in low-resource languages. Furthermore, the lack of global consistency constraints in the embedding space further exacerbates ranking fluctuations and result biases in cross-natural language retrieval. Summary of the Invention

[0004] The main objective of this application is to provide a method for training a cross-natural language code retrieval model, a cross-natural language code retrieval method, apparatus, device, and medium.

[0005] To address the aforementioned technical problems, this application provides a method for training a cross-natural language code retrieval model, the method comprising:

[0006] Obtain the original corpus database, and construct training data based on the original corpus database. The original corpus database includes multiple code segments and natural language descriptions containing multiple natural languages ​​corresponding to each code segment. The training data includes the main language query text, the target language query text, and the main language code.

[0007] The host language code is obfuscated and reversed to obtain host language code samples, wherein the host language code samples include positive host language code samples and negative host language code samples;

[0008] Based on the training data and the main language code samples, the initial model is trained through a gradient inversion layer to obtain the target model.

[0009] In one embodiment, the step of constructing training data based on the original corpus database includes:

[0010] The main language is determined from the original corpus database, and the natural language description of the main language is used as the main language query text, wherein the main language is a natural language;

[0011] Select the code fragment corresponding to the main language query text from the original corpus database as the main language code;

[0012] The target language query text is obtained based on the main language query text, wherein the main language and the target language are different natural languages, and the target language includes at least one natural language;

[0013] Training data is constructed based on the main language query text, the target language query text, and the main language code.

[0014] In one embodiment, the step of obfuscating and reversing the host language code to obtain a host language code sample includes:

[0015] The main language code is parsed to obtain a structured abstract syntax tree;

[0016] Obfuscate the abstract syntax tree to obtain a positive sample of the main language code;

[0017] Reverse the abstract syntax tree to obtain negative samples of the main language code;

[0018] Based on the positive and negative samples of the main language code, a main language code sample is obtained.

[0019] In one embodiment, the initial model includes a first initial model and a second initial model, and the target model includes a first target model. The step of training the initial model using a gradient reversal layer based on the training data and the main language code samples to obtain the target model includes:

[0020] The main language code sample is encoded using the second initial model to obtain the main language code embedding vector;

[0021] The main language code embedding vector is forward-propagated through a gradient inversion layer to obtain the transformed main language code embedding vector.

[0022] The transformed main language code embedding vector is input into the first initial model to obtain the predicted language category;

[0023] Calculate the gradient based on the predicted language category and the preset language categories of the original corpus database;

[0024] The parameters of the first initial model are updated based on the gradient to obtain the first target model.

[0025] In one embodiment, the target model further includes a second target model, and the step of training the initial model using a gradient reversal layer based on the training data and the main language code samples to obtain the target model includes:

[0026] The inverted gradient is calculated based on the gradient through the gradient inversion layer;

[0027] The parameters of the second initial model are updated based on the inverted gradient to obtain the second target model.

[0028] In one embodiment, the initial model further includes a third initial model, and the target model further includes a third target model. The step of training the initial model using a gradient reversal layer based on the training data and the main language code samples to obtain the target model includes:

[0029] The main language code sample is input into the second target model to obtain the main language target code embedding vector;

[0030] The main language query text and the target language query text are respectively input into the third initial model to obtain the main language query vector and the target language query vector;

[0031] Based on the host language target code embedding vector, the host language query vector, and the target language query vector, minimize the composite retrieval loss;

[0032] The weights of the third initial model are updated based on the composite retrieval loss to obtain the third target model.

[0033] In one embodiment, after the step of obtaining the third target model, the method further includes:

[0034] The training data is sampled to construct a validation set, wherein the validation set includes language validation query text and language validation code, the language validation query text includes language query text in multiple natural languages, and the language validation code includes code snippets corresponding to multiple natural languages;

[0035] The validation set is processed by the third target model and the second target model to obtain the validation language query vector and the validation code embedding vector for each natural language.

[0036] A comprehensive quantitative index is calculated based on the validation set, the validation language query vector for each natural language, and the validation code embedding vector for each natural language.

[0037] If the comprehensive quantitative index does not meet the preset conditions, then the compensation weight is calculated based on the comprehensive quantitative index and the validation set.

[0038] The parameters of the third objective model are updated based on the compensation weights to obtain the optimized third objective model.

[0039] In one embodiment, the step of obtaining the verification language query vector for each natural language and the verification code embedding vector for each natural language by passing the third target model and the second target model respectively includes:

[0040] Obtain all validation query texts and corresponding language validation codes for each natural language in the validation set;

[0041] The verification query text for each natural language is input into the third target model to obtain the verification language query vector for each natural language.

[0042] The language verification code for each natural language is input into the second target model to obtain the verification code embedding vector for each natural language.

[0043] In one embodiment, the step of calculating the comprehensive quantitative index based on the validation set, the validation language query vector for each natural language, and the validation code embedding vector for each natural language includes:

[0044] Calculate the similarity between the verification language query vector and the corresponding verification code embedding vector for each natural language and sort them to obtain a sorted list for each natural language;

[0045] A comprehensive quantitative index is calculated based on any two sorted lists in the sorted lists of each natural language and the validation set.

[0046] In one embodiment, the step of calculating the comprehensive quantitative index based on any two sorted lists in the sorted lists of each natural language and the validation set includes:

[0047] Calculate the similarity between any two natural languages ​​based on any two sorted lists in the sorted lists of each natural language;

[0048] A comprehensive quantitative index is calculated based on the number of natural languages ​​in the validation set and the similarity scores of each language.

[0049] In one embodiment, the step of calculating compensation weights based on the comprehensive quantitative index and the validation set if the comprehensive quantitative index does not meet the preset conditions includes:

[0050] If the comprehensive quantitative index does not meet the preset conditions, then the similarity between any two natural languages ​​in the verification set is obtained;

[0051] The compensation weight is calculated based on the similarity between any two natural languages ​​in the validation set and the comprehensive quantitative index.

[0052] In one embodiment, the step of calculating the compensation weight based on the similarity between any two natural languages ​​in the validation set and the comprehensive quantitative index includes:

[0053] The deviation factor is calculated based on the difference between the similarity of any two natural languages ​​and the comprehensive quantitative index.

[0054] The deviation factor is used to calculate the activation value through an activation function;

[0055] Based on the language query texts of the various natural languages, the third initial model is used to obtain the initial verification language query vector for each natural language;

[0056] The compensation weight is calculated based on the activation value and the initial verification language query vector.

[0057] Furthermore, to address the aforementioned technical problems, this application provides a cross-natural language code retrieval method, which includes:

[0058] In response to a user query command, obtain the user query language text corresponding to the user query command;

[0059] The user query language text is input into the third target model to obtain the user query language vector;

[0060] Determine the user query code based on the user query language text;

[0061] The user query code is input into the second target model to obtain the user query code embedding vector;

[0062] Calculate and sort the similarity between the user query language vector and the user query code embedding vector;

[0063] The code retrieval results are obtained based on the sorted similarity and the multilingual code corpus.

[0064] Furthermore, to address the aforementioned technical problems, this application also provides a cross-natural language code retrieval model training device, which includes:

[0065] The sample construction module is used to acquire the original corpus database and construct training data based on the original corpus database. The original corpus database includes multiple code segments and natural language descriptions containing multiple natural languages ​​corresponding to each code segment. The training data includes the main language query text, the target language query text, and the main language code.

[0066] The sample conversion module is used to obfuscate and reverse the main language code to obtain main language code samples, wherein the main language code samples include positive main language code samples and negative main language code samples;

[0067] The training module is used to train the initial model using the training data and the main language code samples through a gradient inversion layer to obtain the target model.

[0068] Furthermore, to address the aforementioned technical problems, this application also provides a cross-natural language code retrieval device, which includes:

[0069] The response module is used to respond to a user query command and obtain the user query language text corresponding to the user query command;

[0070] The query encoding module is used to input the user query language text into the third target model to obtain the user query language vector;

[0071] The determination module is used to determine the user query code based on the user query language text;

[0072] The code encoding module is used to input the user query code into the second target model to obtain the user query code embedding vector;

[0073] The calculation module is used to calculate and sort the similarity between the user query language vector and the user query code embedding vector.

[0074] The retrieval module is used to obtain code retrieval results based on the sorted similarity and the multilingual code corpus.

[0075] In addition, to solve the above-mentioned technical problems, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the cross-natural language code retrieval model training method and / or the cross-natural language code retrieval method as described above.

[0076] In addition, to solve the above-mentioned technical problems, this application also provides a storage medium, which is a computer-readable storage medium, and stores a program for implementing a cross-natural language code retrieval model training method. The program for implementing the cross-natural language code retrieval model training method is executed by a processor to implement the steps of the cross-natural language code retrieval model training method and / or the cross-natural language code retrieval method as described above.

[0077] This application provides a method for training a cross-natural language code retrieval model. First, it acquires a raw corpus database containing multilingual code fragments and corresponding natural language descriptions to construct training data including host language query text, target language query text, and host language code. Then, it obfuscates and inverts the host language code to obtain positive and negative host language code samples. Finally, it trains the initial model using a gradient inversion layer to obtain the target model. Obfuscation suppresses the model's dependence on surface features, while inversion allows the model to focus on the logical relationships within the code. Training the initial model using the gradient inversion layer enables the target model to remove functionally irrelevant grammatical "fingerprint" features from the code, allowing the first and second target models to train adversarially. Specifically, the gradient inversion layer calculates the gradient during forward propagation, causing the first target model to... On the one hand, it can accurately classify languages. On the other hand, by calculating the inverted gradient during backpropagation through the gradient inversion layer, the second objective model cannot classify languages. This forces the second objective model to strip away the grammatical "fingerprint" features of specific natural languages, avoiding the formation of independent clusters in the vector space of the same functional code due to different natural languages. This unifies the alignment direction of the embedding space of different languages ​​and breaks down language barriers. At the same time, based on the adversarial training mechanism of constructing multilingual training data and gradient inversion layer, it can effectively reduce the sampling distribution bias caused by single-language batch training, alleviate the overfitting problem of the model on high-resource languages, improve the retrieval performance of the model on low-resource languages, enhance the global consistency of the embedding space, and ultimately significantly improve the consistency and generalization ability of cross-language code retrieval results. It effectively improves the defects of large fluctuations in cross-language retrieval ranking and significant result bias in traditional methods, and achieves accurate and stable retrieval matching of the same functional code in multiple natural languages. Attached Figure Description

[0078] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0079] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0080] Figure 1 A flowchart illustrating the cross-natural language code retrieval model training method provided in this application embodiment;

[0081] Figure 2 A schematic diagram of the training process of the cross-natural language code retrieval model provided in this application embodiment;

[0082] Figure 3 A flowchart illustrating the cross-natural language code retrieval method provided in this application embodiment;

[0083] Figure 4 A schematic diagram of the module structure of the cross-natural language code retrieval model training device provided in the embodiments of this application;

[0084] Figure 5 A schematic diagram of the module structure of the cross-natural language code retrieval device provided in the embodiments of this application;

[0085] Figure 6 A schematic diagram of the device structure for the hardware operating environment of the cross-natural language code retrieval model training method and / or the cross-natural language code retrieval method provided in the embodiments of this application.

[0086] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0087] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0088] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0089] Currently, deep learning technology has made significant progress in the field of code retrieval, with contrastive learning frameworks being a common method for cross-natural language code retrieval. However, existing methods have significant limitations: on the one hand, models easily learn language-specific "fingerprint" features of natural languages. These features are unrelated to the core semantics of the code, leading to independent clusters of code with the same function in the vector space due to natural language categories. This results in inconsistent alignment directions in the embedding space, creating language barriers and poor consistency in cross-natural language retrieval results. On the other hand, existing methods mostly employ a single-language batch training paradigm. The empirical distribution of training samples deviates significantly from the real multilingual joint distribution. Models are prone to overfitting to high-resource languages ​​and experiencing performance degradation in low-resource languages. Furthermore, the lack of global consistency constraints in the embedding space further exacerbates ranking fluctuations and result biases in cross-natural language retrieval.

[0090] The main solution of this application is as follows: First, obtain an original corpus database. Second, construct training data based on the original corpus database. The original corpus database includes multiple code segments and corresponding natural language descriptions containing multiple natural languages ​​for each code segment. The training data includes host language query text, target language query text, and host language code. Third, obfuscate and invert the host language code to obtain host language code samples. These host language code samples include positive host language code samples and negative host language code samples. Fourth, train the initial model using a gradient inversion layer based on the training data and the host language code samples to obtain the target model.

[0091] To address the aforementioned issues, this application acquires a raw corpus database containing multilingual code fragments and their corresponding natural language descriptions. It then constructs training data comprising main language query text, target language query text, and main language code. The main language code is then obfuscated and inverted to obtain positive and negative samples. These samples are then used to train the initial model using a gradient inversion layer to obtain the target model. Specifically, the gradient inversion layer assists the first target model in accurately classifying languages ​​through forward propagation, while the reversed gradient output during backpropagation prevents the second target model from classifying languages. This process effectively removes grammatical "fingerprints" of specific natural languages ​​unrelated to functionality from the code, preventing the formation of independent vector clusters for codes with the same function due to different natural language languages. It also unifies the alignment direction of the embedding space and breaks down language barriers. Furthermore, the multilingual training data and adversarial training mechanism reduce the batch sampling distribution bias of single languages, alleviate overfitting in high-resource languages, and improve retrieval performance and global consistency of the embedding space for low-resource languages. Ultimately, this addresses the problems of large fluctuations in ranking and significant result bias in traditional cross-language code retrieval, achieving accurate and stable retrieval and matching of codes with the same function across multiple natural languages.

[0092] Specifically, this application relates to a cross-natural language code retrieval technique that optimizes the embedded spatial geometry under non-independent and identically distributed data conditions through anchor constraints. This application abandons the traditional "query-code" binary alignment model and proposes a ternary anchor architecture based on "main language query-target language query-code," forcibly constraining the consistency of the high-dimensional spatial structure of each natural language within the same Hilbert space. Simultaneously, a structured abstract syntax tree is obtained by parsing the code, and then the code text is derived from the abstract syntax tree. By obfuscating and reversing the code text, two types of samples with opposite facts are generated. The obfuscation operation suppresses the model's dependence on surface features, while the reversal operation forces the model to focus on the true logical causal relationships of the code. Furthermore, by introducing a gradient reversal layer between the first and second target models, a bidirectional adversarial game is constructed between the two models. The core objective is to force the second target model to eliminate the syntactic fingerprint of a specific natural language, making the encoding vector approach a "language-inseparable" distribution, thereby obtaining a unified language-independent semantic subspace. Ultimately, this ensures that the semantic topology of code fragments remains consistent during cross-natural language mapping.

[0093] It should be noted that the execution subject of the method in the various embodiments of the cross-natural language code retrieval model training method of this application can be a code retrieval system, or a computing service device with data processing, network communication and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of realizing the above functions, etc. This embodiment does not specifically limit it. The following uses a code retrieval system as the execution subject as an example to describe this embodiment and the following embodiments.

[0094] Based on this, this application proposes a first embodiment of a cross-natural language code retrieval model training method, please refer to... Figure 1 The cross-natural language code retrieval model training method includes steps S10 to S30:

[0095] Step S10: Obtain the original corpus database and construct training data based on the original corpus database. The original corpus database includes multiple code segments and natural language descriptions containing multiple natural languages ​​corresponding to each code segment. The training data includes the main language query text, the target language query text, and the main language code.

[0096] In this embodiment, the original corpus database originates from a pre-defined multilingual code corpus, which contains multiple code fragments. Several code fragments are selected from the multilingual code corpus as code fragments in the original corpus database. The original corpus database includes multiple code fragments and their corresponding natural language descriptions. Each code fragment is associated with a natural language description, meaning each code fragment corresponds to a natural language description containing multiple natural languages. The natural language descriptions are extracted from the docstrings, comments, or function declarations of the code fragments using natural language processing techniques (such as regular expression extraction and semantic segmentation). These descriptions accurately represent the core functional semantics of the corresponding code fragments, and the multiple natural language description formats can cover both the host language and the target language. This provides a multilingual data foundation for training cross-natural language code retrieval models, ensuring the diversity and representativeness of the training data. The training data is the fundamental data set supporting the entire model training process. A training dataset consists of a triple vector comprising a main language query text, a target language query text, and a main language code. The main language code is a code snippet from the original corpus database, the main language query text is a natural language description of the main language code, and the target language query text is another natural language description of the main language code. The main language query text and the target language query text correspond to different natural language types.

[0097] In this embodiment, step S10 may include:

[0098] Step S101: Determine the main language from the original corpus database, and use the natural language description of the main language as the main language query text, wherein the main language is a natural language;

[0099] It should be noted that the main language is determined from the original corpus database. The main language is a natural language, such as English, which can be pre-set or randomly selected.

[0100] For example, natural language descriptions (such as English) are extracted from the Docstrings, comments, or function declarations of code snippets in the original corpus using natural language processing techniques (such as regular expression extraction, semantic segmentation, etc.), and the extracted natural language descriptions of the main language are used as the query text of the main language.

[0101] Step S102: Select the code fragment corresponding to the main language query text from the original corpus database as the main language code;

[0102] Based on the function name, call relationship, or position anchor of the main language, the code fragment corresponding to the main language query text is determined in the original corpus data through the main language query text, and the code fragment is used as the main language code.

[0103] Step S103: Obtain the target language query text based on the main language query text, wherein the main language and the target language are different natural languages, and the target language includes at least one natural language;

[0104] It should be noted that, since the target language includes at least one natural language, the target language query text includes natural language descriptions in multiple natural languages ​​corresponding to the main language code.

[0105] This application does not limit how the target language query text is determined based on the main language query text. For example, in one feasible implementation, the main language query text can be translated into the target language to obtain the target language query text. In another feasible implementation, BERT technology can be used to perform multilingual alignment indexing on the main language query text in the original corpus database to obtain the target language query text. For example, the target language query text can be Chinese, Japanese, Korean, etc.

[0106] For example, if the primary language query text is "Spring Boot MultipartFile uploadcontroller", and a multi-language aligned index is performed on the primary language query text, the target language query text can be "Spring Boot uses MultipartFile to implement file upload controller".

[0107] Step S104: Construct training data based on the main language query text, the target language query text, and the main language code.

[0108] Training data is constructed based on the host language query text, the target language query text, and the host language code. Each training data point consists of the host language query text. The main language code C and the corresponding target language query text. composition.

[0109] For example, a training data set includes a primary language query text, a target language query text, and a primary language code. For instance, if the primary language is determined to be English, the primary language query text is "Calculate the factorial of a given number using recursion," and the primary language code is "def factorial(n):return 1 if n==0 else n...". If the target language is Chinese, the corresponding target language query text can be "Calculate the factorial of a given number using a recursive method". If the target language includes multiple natural languages, the target language query text can be one of the target language query texts in multiple natural languages. For ease of description, they will not be listed one by one here.

[0110] Thus, the embodiments of this application not only ensure the accurate correspondence between the main language code and the query text, but also construct cross-natural language query text pairs, helping the model learn the mapping relationship between code semantics and cross-language natural language semantics, further improving the adaptability of training data to cross-natural language code retrieval scenarios, assisting in the goal of stripping away the language-specific "fingerprint" features of natural languages, and effectively improving the consistency of cross-natural language code retrieval.

[0111] Step S20: Obfuscate and reverse the main language code to obtain a main language code sample, wherein the main language code sample includes a positive main language code sample and a negative main language code sample;

[0112] For example, when obfuscating the host language code, variable name obfuscation can be performed to obtain code with the same semantics as the host language code but different natural language features, and this code is used as a positive host language code sample. Simultaneously, when reversing the host language code, logical inversion can be performed to obtain code with different semantics but the same natural language features, and this code is used as a negative host language code sample. Both positive and negative host language code samples are collectively referred to as host language code samples.

[0113] In this embodiment, step S20 may include:

[0114] Step S201: Parse the main language code to obtain a structured abstract syntax tree;

[0115] It's important to note that parsing the main language code refers to the specialized process of using code parsing tools compatible with multiple programming languages ​​to perform syntactic analysis and structural extraction on the main language code. The core objective is to convert linearly arranged code snippets into a structured data representation, clearly presenting the code's syntactic structure, logical hierarchy, and core functional elements in the form of hierarchical nodes. Only the core content of the code's syntax and logic is retained, while redundant formatting is eliminated. An abstract syntax tree (AST) is a structured data structure obtained after code parsing. It can clearly present the code's syntactic structure, logical hierarchy, and core functional elements in the form of hierarchical nodes, retaining only the core content of the code's syntax and logic while removing irrelevant formatting redundancy.

[0116] For example, parsers such as Tree-sitter can be used to extract the main language code. It is mapped to a structured abstract syntax tree (AST).

[0117] Step S202: Obfuscate the abstract syntax tree to obtain a positive sample of the main language code;

[0118] In one feasible implementation, when obfuscating the abstract syntax tree, positive samples of the main language code can be obtained through variable name obfuscation. This method replaces variable names without changing the core logic and syntactic features of the code. It is used to eliminate the surface semantic interference of variable naming and cut off the false causal path between variable naming and functional semantics. Specifically, it identifies and replaces identifiers such as variable names and function names in the abstract syntax tree with random strings. The code after replacement is the positive sample of the main language code. For example, it identifies and replaces function names in the abstract syntax tree with random strings (such as var_1).

[0119] Step S203: Invert the abstract syntax tree to obtain negative samples of the main language code;

[0120] When reversing an abstract syntax tree, negative samples of the main language code can be obtained through logical reversal. This involves adjusting the core logic in reverse without changing the syntax features of the code, generating samples with opposite functional semantics. Specifically, this can involve identifying and reversing logical operators or control flow keywords in the abstract syntax tree to obtain negative samples of the main language code that are logically opposite but textually very similar. For example, identifying "==" in the abstract syntax tree and reversing it to "!=".

[0121] Step S204: Based on the positive sample of the main language code and the negative sample of the main language code, obtain the main language code sample.

[0122] Positive samples and negative samples of the main language code constitute the main language code sample.

[0123] Step S30: Based on the training data and the main language code sample, train the initial model through a gradient inversion layer to obtain the target model.

[0124] It's important to note that the Gradient Reversal Layer (GRL) is structurally a custom layer. During forward propagation, it performs identity transformations to calculate the gradient, and during backpropagation, it calculates the inverse gradient based on the initial gradient. Specifically, training the initial model with the GRL allows the target model to remove the language-specific grammatical "fingerprint" features from the code that are irrelevant to the function. This enables the first and second target models to train adversarially. On one hand, the gradient reversal layer calculates gradients during forward propagation, allowing the first target model to accurately classify natural language. On the other hand, the inverse gradient calculation during backpropagation prevents the second target model from classifying natural language. This forces the second target model to remove the language-specific grammatical "fingerprint" features, preventing identical code from forming independent clusters in the vector space due to different natural language categories. This unifies the embedding space alignment of different natural languages, breaking down language barriers.

[0125] Based on the training data and main language code samples constructed through the above steps, the initial model is trained through a gradient inversion layer to obtain the target model.

[0126] Thus, this embodiment of the application acquires an original corpus database containing multilingual code fragments and corresponding natural language descriptions, constructs training data containing main language query text, target language query text, and main language code, then obfuscates and inverts the main language code to obtain positive and negative main language code samples, and trains the initial model using a gradient inversion layer to obtain the target model. Specifically, the gradient inversion layer calculates gradients during forward propagation to help the first target model accurately classify languages, and outputs inverted gradients during backpropagation to prevent the second target model from classifying languages. This removes the grammatical "fingerprints" of specific natural languages ​​unrelated to function from the code, preventing the formation of independent vector clusters for codes with the same function due to different natural language languages, unifying the alignment direction of the embedding space, and breaking down language barriers. Simultaneously, the construction of multilingual training data and the model adversarial training mechanism can reduce the batch sampling distribution bias of single languages, alleviate overfitting in high-resource languages, improve the retrieval performance and global consistency of the embedding space for low-resource languages, and ultimately improve the problems of large fluctuations in ranking and significant result bias in traditional cross-natural language code retrieval, achieving accurate and stable retrieval matching of codes with the same function across multiple natural languages.

[0127] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment can be referred to the above description and will not be repeated hereafter. Based on this, the initial model includes a first initial model and a second initial model, and the target model includes a first target model. It should be noted that the first initial model can be an initial language discriminator, i.e., a language discriminator to be trained, which can be composed of a multilayer perceptron (MLP) classifier, used to receive code embedding vectors and predict their natural language category. The second initial model can be an initial code encoder, i.e., a code encoder to be trained, which can be built based on the Transformer / CodeBERT architecture and is the core semantic extraction module of the initial model, used to extract features from the input code and generate code embedding vectors. The code embedding vectors are high-dimensional vectors that can represent the core features of the code. The first target model can be a target language discriminator, i.e., a trained language discriminator.

[0128] Step S30 may include:

[0129] Step S301: Encode the main language code sample using the second initial model to obtain the main language code embedding vector;

[0130] The positive and negative samples of the main language code are input into the second initial model, and the two samples are encoded by the second initial model to obtain the embedding vectors of the positive and negative samples of the main language code. The embedding vectors of the positive and negative samples of the main language code are collectively referred to as the main language code embedding vector, i.e., a high-dimensional dense vector, which is the output of the code encoder and is used to represent the semantics of the code.

[0131] Step S302: The main language code embedding vector is forward propagated through a gradient inversion layer to obtain the transformed main language code embedding vector.

[0132] It should be noted that the gradient reversal layer is a custom layer inserted between the first initial model and the second initial model. During forward propagation, it is used to train the first initial model to accurately classify the language, thereby adversarial to the training of the second initial model, which strips away the grammatical fingerprint of a specific natural language.

[0133] During forward propagation, taking the second initial model as the initial code encoder as an example, the initial code encoder is the code encoder to be trained, and the main language code embedding vector output by the initial code encoder is used. The input is fed into the GRL and subjected to an identity transformation (i.e., without any modification, the main language code embedding vector is directly output), resulting in the transformed main language code embedding vector. This process can be represented as = .

[0134] Step S303: Input the transformed main language code embedding vector into the first initial model to obtain the predicted language category;

[0135] Taking the first initial model as the initial language discriminator as an example, the initial language discriminator is the language discriminator to be trained. After performing the above identity transformation, the transformed main language code embedding vector is input into the Softmax of the initial language discriminator to obtain the predicted language category (predicted language category). This predicted language category is the natural language category corresponding to the main language code predicted by the initial language discriminator, such as Chinese.

[0136] Step S304: Calculate the gradient based on the predicted language category and the preset language categories of the original corpus database;

[0137] The actual natural language category (i.e., the preset language category) of the current main language code is obtained from the original corpus database and used as the label benchmark for training the first initial model. The error between the predicted language category and the preset language category is calculated using a preset cross-entropy loss function to quantify the bias of the first initial model's prediction (hereinafter referred to as classification loss); the gradient is determined based on this classification loss. This gradient is the partial derivative of the cross-entropy loss function with respect to the parameters of the first initial model and is used to update the parameters of the first initial model.

[0138] In one feasible implementation, taking the first initial model as the initial language discriminator as an example, the true natural language category (i.e., the preset language category) corresponding to the current main language code embedding vector is obtained from the original corpus database. The error between the predicted language category and the preset language category is calculated, and this error is input into the initial language discriminator D. The cross-entropy loss is calculated using the cross-entropy loss function, and then the gradient is calculated based on the cross-entropy loss. The parameters of the initial language discriminator are updated to optimize its language discrimination ability, thereby obtaining the target language discriminator. Specifically, the cross-entropy loss is calculated using the cross-entropy loss function as follows:

[0139] ;

[0140] Where B refers to the current batch size, which is a constant. One batch size consists of multiple samples taken from the original corpus database, and each batch size consists of b data points selected from the original corpus database; y represents the preset natural language label, i.e., the preset language category. The predicted language category probability, i.e., the probability that the sample belongs to the i-th natural language in the j-th batch of samples, satisfies the following condition: ∈[0,1] and ∑ =1; K represents the total number of natural language types in the original corpus database. For example, if the original corpus database contains English, Chinese, and Malay, then K=3.

[0141] Step S305: Update the parameters of the first initial model based on the gradient to obtain the first target model.

[0142] For example, partial derivatives can be calculated based on gradients, and the model parameters of the first initial model can be updated according to the partial derivatives to obtain the trained first target model.

[0143] Taking the first initial model as the initial language discriminator as an example, the gradient is calculated based on the cross-entropy loss. By updating the parameters of the initial language discriminator, the target language discriminator can be obtained, specifically as follows:

[0144] ;

[0145] in, These are the parameters for the initial language discriminator.

[0146] In this embodiment, it is determined whether the first initial model meets the preset first termination condition. The first termination condition can be the convergence of the cross-entropy loss function in S305, or it can be the repeated execution of the above steps S301-S305 until the preset number of times is reached. When the first termination condition is met, the training of the first initial model is stopped, and the first target model is obtained.

[0147] It should be noted that, to ensure sufficient and efficient training of the language discriminator, the first termination condition is a preset standard for judging whether the language discriminator training is complete. This standard can be based on "loss function convergence" or "meeting of update counts," or a combination of both, or other termination conditions; no specific limitation is made here. Cross-entropy loss function convergence means that the loss value calculated by the cross-entropy loss function tends to stabilize, with fluctuations below the preset convergence threshold, indicating that the prediction accuracy of the language discriminator has reached the expected requirements. A first preset threshold is pre-set to determine whether the initial language discriminator training has reached the optimal number of iterations. This threshold is used to forcibly terminate training if the loss function does not converge in time, avoiding overtraining that could lead to overfitting of the language discrimination model.

[0148] In this embodiment, the target model further includes a second target model, and step S30 may include:

[0149] Step S306: Calculate the inverted gradient based on the gradient through the gradient inversion layer;

[0150] In one feasible implementation, the gradient can be multiplied by a negative constant to obtain the inverted gradient.

[0151] For example, the negative constant used in the gradient reversal layer can be expressed as The gradient is inverted, and the resulting inverted gradient is: .

[0152] Step S307: Update the parameters of the second initial model based on the reversed gradient to obtain the second target model.

[0153] For example, the inverted gradient can be calculated based on the gradient, and the model parameters of the second initial model can be updated according to the inverted gradient to obtain the trained second target model.

[0154] Taking the second initial model as the initial code encoder as an example, through gradient... Calculate the inverted gradient The target code encoder (the trained code encoder) is obtained by training the initial code encoder to update its parameters.

[0155] In this embodiment, it is determined whether the second initial model meets the preset second termination condition. The second termination condition may be to repeat the above steps S301-S307 until a preset number of times is reached. When the second termination condition is met, the training of the second initial model is stopped, and the second target model is obtained.

[0156] It should be noted that the second termination condition is a preset standard used to determine whether the initial code encoder training is complete. A second preset threshold can be set in advance to determine whether the initial code encoder training has terminated. The second preset threshold is the preset optimal number of iterations for the initial code encoder.

[0157] Thus, this embodiment employs Language-Agnostic Adversarial Domain Adaptation (LADA), which trains a first initial model and a second initial model through a gradient inversion layer. This process is a game theory process, where the first target model strives to accurately distinguish codes from different natural languages, while the second target model strives to generate encoded features that cannot be distinguished by the language discriminator. The two continuously iterate and optimize during adversarial training, ultimately enabling the second target model to effectively remove grammatical features bound to specific natural languages ​​from the code. As a result, the output encoded features no longer carry unique information related to natural languages, thereby improving the universality and generalization ability of the second target model across different natural languages. This allows the model to operate without being affected by differences in natural languages ​​in scenarios such as cross-natural language code understanding, code retrieval, and code transfer learning, improving consistency and stability when processing codes from different natural languages. Specifically, the gradient inversion layer calculates the gradient during forward propagation and trains the first initial model by minimizing the classification loss, enabling the first target model to accurately classify language. The gradient inversion layer calculates the inverted gradient during backpropagation and trains the second initial model, preventing the second target model from classifying language. This forces the second target model to strip away the grammatical "fingerprint" features of a specific natural language, avoiding the formation of independent clusters in the vector space of the same functional code due to different natural language categories.

[0158] In this embodiment, the initial model further includes a third initial model, and the target model further includes a third target model. Step S30, which involves training the initial model using a gradient inversion layer based on the training data and the main language code samples to obtain the target model, may further include:

[0159] Step A10: Input the main language code sample into the second target model to obtain the main language target code embedding vector;

[0160] Positive and negative samples of the main language code are input together into the second target model to obtain the semantic features of the main language code, also known as the main language code-target code embedding vector. The main language code-target code embedding vector represents the core semantics of the code, retaining only the functional semantics. The second target model can be a trained code encoder, i.e., the target code encoder.

[0161] Step A20: Input the main language query text and the target language query text into the third initial model respectively to obtain the main language query vector and the target language query vector;

[0162] It should be noted that the initial model also includes a third initial model, which can be the initial query encoder, i.e. the query encoder to be trained. It can be built based on the Transformer / CodeBERT architecture and has the ability to extract semantics from natural language text.

[0163] The main language query text and the target language query text are input into the third initialization model, respectively, to obtain the encoding vectors corresponding to the main language query text (i.e., the main language query vector) and the target language query text (i.e., the target language query vector). These can be collectively referred to as query embedding vectors. The main language query vector and the target language query vector are high-dimensional vectors output by the third initialization model after semantic extraction of the main language query text and the target language query text, respectively, used to represent the core semantics of the corresponding query text.

[0164] Step A30: Based on the main language target code embedding vector, the main language query vector, and the target language query vector, minimize the composite retrieval loss;

[0165] The host language target code embedding vector, host language query vector, and target language query vector are combined into a triplet embedding vector. Then, local losses are calculated for any two vectors in the triplet embedding vector to obtain the first local loss, second local loss, and third local loss, respectively. Finally, a composite retrieval loss is obtained based on the local losses. The composite retrieval loss is used to achieve code retrieval consistency across natural languages. In cross-natural language scenarios, descriptions of the same functional code in different natural languages ​​are often distributed in different regions.

[0166] In one feasible implementation, firstly, the training data, negative samples of the main language code, and positive samples of the main language code are mapped to the same vector space (such as Hilbert space). The output triplet embedding vector enables alignment between different modalities. Including language target code embedding vectors Main language query vector and target language query vector Then, the composite retrieval loss is calculated based on the triplet embedding vectors. Specifically, for any two vectors in the triplet embedding vectors, local losses are calculated using the triplet embedding vectors to obtain the first, second, and third local losses, respectively. Finally, the composite retrieval loss is obtained based on each local loss. This process can be performed by the loss calculation unit of the Compression Loss-based Spatial-Temporal Attention (CL-STA) module, specifically as follows:

[0167] Based on the language target code embedding vector and main language query vector The first local loss was calculated. (Code - Main Query), based on the language target code embedding vector and target language query vector The second local loss was calculated. (Code - Target Query), based on the main language query vector and target language query vector The third local loss was calculated. (Main query - target query); Each local loss is calculated using Information Noise-Contrastive Estimation Loss (InfoNCE), and the calculation formula is as follows:

[0168] ;

[0169] Where B refers to the current batch size, which is a constant. One batch size consists of multiple samples sampled from the original corpus database, and each batch size consists of b data points selected from the original corpus database. Used for calculation and The cosine similarity, where, when calculating the first local loss, , , Let represent the main language target code embedding vector, the main language query vector, and the main language query vector of the k-th sample in the current batch, respectively; when calculating the second local loss... , , Let represent the main language target code embedding vector, the target language query vector, and the target language query vector of the k-th sample in the current batch, respectively; when calculating the third local loss... , , These represent the main language query vector, the target language query vector, and the target language query vector of the kth sample in the current batch, respectively. The temperature parameter, a value within the range of 0-1, is used to adjust the second objective model. Finally, the sum of the local losses is used as the composite retrieval loss, calculated using the following formula:

[0170] .

[0171] Step A40: Update the weights of the third initial model based on the composite retrieval loss to obtain the third target model.

[0172] With minimizing the composite retrieval loss as the core objective (also known as the third termination condition), the weights of the third initial model are iteratively updated based on the composite retrieval loss to continuously optimize the semantic extraction capability of the third initial model until the composite retrieval loss is minimized, thus obtaining the trained third target model. The third target model can be a trained query encoder, i.e., the target query encoder.

[0173] For example, such as Figure 2 The diagram illustrates the model training process. First, training data containing the main language query text, target language query text, and main language code is extracted from the original corpus data. The main language code is parsed to obtain a structured abstract sequence (AST). Variable name obfuscation is applied to the AST to obtain positive main language code samples with the same semantics as the AST. Logical inversion is then performed on the AST to obtain negative main language code samples with the opposite semantics. The main language code samples, composed of the positive and negative samples, are input into the second initial model to obtain the main language code embedding vector. This embedding vector is then input into the first initial model to obtain the predicted language category. A classification loss is calculated based on the preset and predicted language categories of the embedding vector. Finally, a gradient inversion layer is used to calculate the gradient based on the classification loss, and the gradient is then used to... The parameters of the first initial model are trained until the first termination condition is met, resulting in the first target model. Then, the gradient inversion layer is used to calculate the inverted gradient, and the second initial model is trained based on the inverted gradient until the second termination condition is met, resulting in the second target model. Next, the main language code samples are input into the second target model to obtain the main language target code embedding vector. The main language query text and the target language query text are input into the third initial model to obtain the main language query vector corresponding to the main language query text and the target language query vector corresponding to the target language query text. Based on the main language target code embedding vector, the main language query vector, and the target language query vector, the composite retrieval loss is calculated. The parameters of the third initial model are trained with the goal of minimizing the composite retrieval loss (i.e., the third termination condition), resulting in the third target model.

[0174] Thus, the embodiments of this application, through the above-described training method for the third initial model, avoid the high computational complexity and large GPU memory consumption caused by traditional parallel training of multiple natural languages; by increasing the computational resource consumption by only about 5% for triple encoding, the joint distribution of multiple natural language codes can be approximately reconstructed with lower computational complexity, significantly reducing the GPU memory occupation during the training process of the third initial model. This allows a single GPU to complete the training of a query encoder supporting more than 10 natural languages. While effectively solving the technical problems of large differences in the representation of different natural language codes, weak model generalization, and excessive training resource requirements, this embodiment significantly improves the feasibility and resource utilization efficiency of multi-natural language code model training.

[0175] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and second embodiments described above can be referred to the above description and will not be repeated hereafter. Based on this, after step A40, the cross-natural language code retrieval model training method of this application further includes:

[0176] Step B10: Sample the training data and construct a validation set, wherein the validation set includes language validation query text and language validation code, the language validation query text includes language query text in multiple natural languages, and the language validation code includes code snippets corresponding to multiple natural languages;

[0177] The training data is sampled, and a validation set is constructed based on the sampled data. The validation set includes language validation query text and language validation code. The language validation query text includes language query text in multiple natural languages, and the language validation code includes code snippets corresponding to multiple natural languages.

[0178] In one feasible implementation, the number of samples for validation data is pre-set to ensure the reliability and accuracy of the validation results and to avoid validation bias caused by insufficient sample size. A preset number of sample data are sampled from multiple training data sets to construct a validation set. For example, 10% to 20% of the data is sampled from each training data set as the validation set. Each validation sample data in the validation set includes the main language code, the corresponding main language query text, and the corresponding target language query text. In the validation set, the main language query text and the target language query text are collectively referred to as language validation query text, and the code fragments corresponding to each query text are referred to as language validation codes.

[0179] Step B20: The verification set is processed by the third target model and the second target model to obtain the verification language query vector and the verification code embedding vector for each natural language, respectively.

[0180] The validation set is processed through the second objective model to obtain the processing result, namely the validation code embedding vector; the validation set is processed through the third objective model to obtain the processing result, namely the validation language query vector.

[0181] In this embodiment, step B20 may include:

[0182] Step B201: Obtain all verification query texts and corresponding language verification codes for each natural language in the verification set;

[0183] Step B202: Input the verification query text for each natural language into the third target model to obtain the verification language query vector for each natural language.

[0184] Step B203: Input the verification code for each natural language into the second target model to obtain the verification code embedding vector for each natural language.

[0185] Obtain all validation query texts and corresponding language validation codes for each natural language in the validation set; input all language validation query texts for each natural language into the third target model to obtain the validation language query vector for each natural language; and input the language validation code for each natural language into the second target model to obtain the validation code embedding vector for each natural language.

[0186] Step B30: Calculate a comprehensive quantitative index based on the validation set, the validation language query vector for each natural language, and the validation code embedding vector for each natural language.

[0187] It should be noted that the comprehensive quantitative indicator can be the Rank Distance Matrix (RDM) indicator.

[0188] In this embodiment, step B30 may include:

[0189] Step B301: Calculate and sort the similarity between the verification language query vector and the corresponding verification code embedding vector for each natural language to obtain a sorted list for each natural language.

[0190] In one feasible implementation, the similarity between the verification language query vector and the verification code embedding vector for each natural language is calculated, and they are arranged in descending order of similarity to obtain an initial sorted list for each natural language; the top-ranked verification data in each initial sorted list are selected to form the final sorted list for each natural language.

[0191] For example, taking the second target model as the target code encoder and the third target model as the target query encoder, a natural language i is randomly selected from the validation set. Based on natural language i, all corresponding language validation codes and language validation query texts for natural language i are obtained from the validation set. Then, the language validation codes and language validation query texts for natural language i are input into the second and third target models respectively to obtain the validation language query vector and validation code embedding vector for natural language i. Next, the cosine similarity between the validation language query vector and validation code embedding vector for natural language i is calculated. The vectors are then sorted from high to low based on their cosine similarity, and the top K vectors are selected to obtain the sorted list for natural language i. Similarly, we obtain the sorted list of natural language j. Here, natural language i and natural language j are different natural languages ​​in the validation set. All natural languages ​​in the validation set are traversed, and the search result list for each natural language is calculated using the aforementioned steps. It is understood that natural languages ​​i and j are merely two different natural languages ​​illustrated above; in practice, this application embodiment does not limit the number of natural languages ​​in the validation set.

[0192] Step B302: Calculate the comprehensive quantitative index based on any two sorted lists in the sorted list of each natural language and the validation set.

[0193] After obtaining the sorted list corresponding to each natural language in the validation set, the RDM index of each sorted list is determined by calculating a comprehensive quantitative index based on any two sorted lists in the sorted list of each natural language and the validation set.

[0194] In this embodiment, step B302 may include:

[0195] Step B3021: Calculate the similarity between any two natural languages ​​based on any two sorted lists in the sorted lists of each natural language.

[0196] Calculate the similarity between any two sorted lists in each natural language sorted list to obtain multiple similarity scores.

[0197] In one feasible implementation, a preset rank correlation coefficient calculation method is used to perform pairwise matching calculations on all sorted lists to obtain the rank correlation coefficient between each pair of lists, thereby quantifying the sorting similarity of each pair of lists.

[0198] Step B3022: Calculate a comprehensive quantitative index based on the number of natural languages ​​in the validation set and the similarity of each language.

[0199] Based on the number of natural languages ​​and their similarities in the validation set, a comprehensive quantitative index is calculated.

[0200] In one feasible implementation, the Spearman rank correlation coefficient is used. calculate and The similarity is calculated, and the average similarity of all languages ​​in the validation set is used to obtain a comprehensive quantitative index. The formula for calculating the comprehensive quantitative index is as follows:

[0201] ;

[0202] Here, query Q refers to the query text of each natural language in the validation set; To verify the total number of natural languages ​​in the set, for example, supporting 4 languages, then ; and Let i and j represent sorted lists of natural language i and natural language j, respectively. is the Spearman rank correlation coefficient, used to quantify the similarity between two sorted lists.

[0203] Understandably, the RDM metric is used to monitor cross-natural language alignment in real time. If the RDM metric is low, it indicates that the codes of different natural languages ​​have drifted in the vector space.

[0204] Step B40: If the comprehensive quantitative index does not meet the preset conditions, then calculate the compensation weight based on the comprehensive quantitative index and the validation set.

[0205] It should be noted that the preset condition can be a threshold value that meets the comprehensive quantitative index. This threshold is used to determine whether the performance of the third objective model meets the target requirements, and it is the core basis for triggering the secondary optimization of the third objective model. The preset condition can be that the comprehensive quantitative index is greater than or equal to the preset judgment threshold, wherein the judgment threshold is preferably set to 0.6.

[0206] If the comprehensive quantitative index does not meet the preset conditions, it means that the performance of the third objective model has not met the target requirements. In this case, the third objective model is optimized a second time based on the comprehensive quantitative index to obtain the optimized third objective model. If the comprehensive quantitative index meets the preset conditions, there is no need to perform additional optimization of the third objective model.

[0207] In this embodiment, step B40 may include:

[0208] Step B401: If the comprehensive quantitative index does not meet the preset conditions, then obtain the similarity between any two natural languages ​​in the verification set.

[0209] If the comprehensive quantitative index is less than the preset judgment threshold, the similarity between any two natural languages ​​in the validation set is obtained. It can be understood that the similarity between any two natural languages ​​refers to the similarity between the sorted lists corresponding to the two natural languages.

[0210] Step B402: Calculate the compensation weight based on the similarity between any two natural languages ​​in the verification set and the comprehensive quantitative index.

[0211] Compensation weights are calculated based on the similarity and comprehensive quantitative index of any two natural languages ​​in the validation set, and these compensation weights are used to optimize the third objective model.

[0212] In this embodiment, step B402 may include:

[0213] Step B4021: Calculate the deviation factor based on the difference between the similarity of the two natural languages ​​and the comprehensive quantitative index;

[0214] Calculate the difference between the similarity and the comprehensive quantitative index of any two natural languages, and use the absolute value of the difference as a deviation factor to obtain multiple deviation factors.

[0215] For example, assuming the validation set includes three natural languages, A, B, and C, then the calculated similarity between any two natural languages ​​includes... , and Calculate the difference between each similarity score and the comprehensive quantitative index. Based on this difference, obtain the deviation factor. The deviation factor may include three components: , and .

[0216] Step B4022: Calculate the activation value of the deviation factor using an activation function;

[0217] Each deviation factor is input into a preset activation function to calculate the activation value, resulting in multiple activation values.

[0218] It should be noted that the activation function can be a preset nonlinear mapping function. Its core function is to normalize and nonlinearly transform the bias factor and output the activation value that adapts to the weight update.

[0219] Step B4023: Based on the language query texts of the multiple natural languages, the initial verification language query vectors for each natural language are obtained through the third initial model.

[0220] The language query texts of multiple natural languages ​​are input into the third initial model (i.e., the untrained query encoder) to obtain the encoding results of the language query texts by the third initial model, which are called the initial validation language query vectors.

[0221] Step B4024: Calculate the compensation weight based on the activation value and the initial verification language query vector.

[0222] In one feasible implementation, the similarity between any two natural languages ​​in the validation set is first calculated. Based on the differences between each similarity score and a comprehensive quantitative index, multiple deviation factors are determined, denoted as […]. ;Calculate the compensation weights using the ReLU activation function The compensation weights are then directly applied to the composite retrieval loss in step A30 above. In the calculation, the adjusted compensation weights are used to train the third objective model, and the calculation formula is as follows:

[0223] ,

[0224] in, The base weight is usually set to 1.0; This is the adjustment coefficient (hyperparameter), used to control the degree of compensation; The calculated deviation factor represents the magnitude of the alignment error between any two natural languages; This represents the activation function, ensuring that weights are increased only when there is a positive bias (large error), and no adjustment is made when the error is small; The similarity score represents the similarity between sample i in a batch of samples (B) and the query embedding vector obtained by encoding the third initial model, used to distinguish the difficulty of samples.

[0225] Step B50: Update the parameters of the third objective model based on the compensation weights to obtain the optimized third objective model.

[0226] After obtaining each compensation weight, the parameters of the third objective model are updated based on each compensation weight to obtain the optimized third objective model.

[0227] This application's embodiments introduce a dynamic error compensation mechanism combining the CL-STA algorithm and the RDM feedback mechanism. This enables the third objective model to automatically identify representational discrepancies between low-resource languages ​​(such as query descriptions in minority languages) and code snippets, and dynamically adjust their training weights. On the one hand, when searching for the same code function in different natural languages, a highly consistent ranking experience can be obtained. On the other hand, experiments show that after adopting the above-mentioned dynamic error compensation mechanism, the mean reciprocal rank (MRR) of low-resource natural language queries is improved by more than 15%, effectively alleviating the performance degradation of code retrieval caused by imbalanced natural language query data. This allows this application to further ensure the accuracy and consistency of code retrieval across different natural languages, while achieving the extraction of natural language syntactic fingerprints and improving the universality of cross-natural language code representation. This effectively improves the ranking stability and user experience in cross-natural language code retrieval scenarios.

[0228] This application proposes a cross-natural language code retrieval method, which can be applied to code retrieval systems.

[0229] In the first embodiment of the cross-natural language code retrieval method of this application, please refer to Figure 3 The cross-natural language code retrieval method includes steps S40-S90:

[0230] Step S40: In response to the user query command, obtain the user query language text corresponding to the user query command;

[0231] It should be noted that a user query command is the instruction information entered by the user when initiating a code search. It includes user query language text that represents the search request and can correspond to any of the natural language types mentioned above, either the main language or the target language. The user query language text is the core component of the user query command and is the natural language representation of the user's search request.

[0232] Receive user query instructions, obtain the natural language description in the user query instructions, process the natural language description using natural language processing technology, and obtain the user query language text.

[0233] Step S50: Input the user query language text into the third target model to obtain the user query language vector;

[0234] The user query language text is input into the third-objective model or an optimized third-objective model to obtain the processing result, namely the user query language vector. This user query language vector is the core basis for retrieval.

[0235] Step S60: Determine the user query code based on the user query language text;

[0236] Based on the user query language text, at least one relevant code fragment is identified in the original corpus database, which is called the user query code.

[0237] Step S70: Input the user query code into the second target model to obtain the user query code embedding vector;

[0238] The user query code is input into the second target model to obtain a vector representation after different natural language features have been removed, which is the user query code embedding vector.

[0239] Step S80: Calculate and sort the similarity between the user query language vector and the user query code embedding vector;

[0240] Calculate the similarity between the user query language vector and the embedding vector of each user query code, and sort them to obtain the sorted similarity scores. Specifically, the similarity can be sorted from high to low.

[0241] Step S90: Obtain code retrieval results based on the sorted similarity and the multilingual code corpus.

[0242] It should be noted that the multilingual code corpus includes multiple code fragments and serves as the data source for code retrieval.

[0243] Finally, the highest similarity scores are selected from the sorted similarity scores to determine the user query codes corresponding to these similarities. These user query codes are then retrieved from the multilingual code corpus as the code retrieval results.

[0244] In one feasible implementation, after obtaining the embedding vector of each user's query code, the embedding vector of each user's query code is stored in an efficient vector database; the user query vector is calculated respectively. Embedded vectors for user query codes in a high-efficiency vector database The cosine similarity score between them is calculated using the following formula:

[0245]

[0246] Then, the similarity scores are sorted from high to low to determine the user query code embedding vectors corresponding to the top K similarity scores. Based on these user query code embedding vectors, the corresponding K code fragments are retrieved from the multilingual code corpus as the retrieval results.

[0247] This application embodiment generates a user query vector through a third target model or an optimized third target model. It combines the user query text with a pre-screening of candidate code fragments in a multilingual code corpus to reduce redundant calculations and improve retrieval efficiency. Then, the natural language features of the candidate code fragments are extracted by a second target model, and the optimal results are selected based on semantic similarity matching. This not only breaks down the barriers between different natural languages ​​and achieves accurate cross-language retrieval, but also solves the problems of inefficiency and result bias in traditional retrieval. It realizes the training value of the target model, balances retrieval accuracy and efficiency, and improves the practicality and user experience of cross-natural language code retrieval.

[0248] For example, taking the query text "retrieve HTTP file upload function" as an example, the whole process is divided into three stages: training, dynamic error compensation and evaluation, and inference and retrieval.

[0249] The first phase is the training phase. Initially, the training data triplets include the English main language query text ("upload file to server via HTTP POST"), the Chinese target language query text ("upload file to server via HTTP POST request"), and the main language code (Java source code snippets) implementing this function. Then, a Tree-sitter is used to map the main language code to an Abstract Syntax Tree (AST). Variable name obfuscation is applied to the AST to obtain positive main language code samples, and logical inversion is performed on the AST to obtain negative main language code samples. An initial code encoder maps all samples to main language code embedding vectors. Next, the main language code embedding vectors are input to an initial language discriminator, which predicts the natural language category of the main language code embedding vectors, thus obtaining the predicted language category. The gradient between the predicted language category and the main language code embedding vector is calculated through a gradient reversal layer to optimize the parameters of the initial language discriminator, resulting in the target language discriminator. The code encoder then eliminates the "language-specific fingerprints" between different natural languages ​​in the main language code embedding vector through the gradient reversal layer, resulting in the target code encoder, whose output retains only abstract logical semantics. Finally, the triples are mapped to the same Hilbert space to obtain triple embedding vectors, and the composite loss is calculated using the CL-STA loss function to train the initial query encoder, resulting in the target query encoder, ensuring the consistency of the high-dimensional semantic space structure across natural languages.

[0250] The second stage is the dynamic error compensation and evaluation stage. This stage focuses on consistency monitoring and adaptive optimization. It evaluates the consistency of cross-language retrieval ranking in real time on the validation set. For the "file upload" function, it generates ranking lists for two different natural languages. If the RDM index corresponding to the Spearman rank correlation coefficient between the two lists is <0.6, a cross-language distribution bias is identified. Subsequently, through the RDM feedback mechanism, based on the bias factor and sample difficulty, a compensation weight is calculated using the ReLU activation function and applied to the next round of CL-STA loss calculation. This forcibly corrects the embedding space offset of the corresponding natural language, resulting in the optimized target query encoder.

[0251] The third stage is the reasoning and retrieval stage. In this stage, the implementation scheme is deployed. After the user inputs a natural language query, the target query encoder generates a corresponding user query language vector. The target code encoder, utilizing its learned adversarial coding and feature purification capabilities, maps the code in the multilingual code corpus into user query code embedding vectors that have been freed from grammatical interference from different natural languages. Finally, the cosine similarity between the user query language vector and the user query code embedding vector is calculated, and the Top-K best-matching code fragments across natural languages ​​are retrieved in real time, enabling accurate retrieval of code related to the HTTP file upload function.

[0252] This application also provides a cross-natural language code retrieval model training device. Please refer to... Figure 4 The cross-natural language code retrieval model training device includes:

[0253] The sample construction module 10 is used to acquire the original corpus database and construct training data based on the original corpus database. The original corpus database includes multiple code segments and natural language descriptions containing multiple natural languages ​​corresponding to each code segment. The training data includes the main language query text, the target language query text, and the main language code.

[0254] The sample conversion module 20 is used to obfuscate and reverse the main language code to obtain a main language code sample, wherein the main language code sample includes a positive main language code sample and a negative main language code sample;

[0255] The training module 30 is used to train the initial model through a gradient inversion layer based on the training data and the main language code samples to obtain the target model.

[0256] This application also provides a cross-natural language code retrieval device. Please refer to... Figure 5 The cross-natural language code retrieval device includes:

[0257] Response module 40 is used to respond to a user query command and obtain the user query language text corresponding to the user query command;

[0258] The query encoding module 50 is used to input the user query language text into the third target model to obtain the user query language vector;

[0259] The determining module 60 is used to determine the user query code based on the user query language text;

[0260] The code encoding module 70 is used to input the user query code into the second target model to obtain the user query code embedding vector;

[0261] The calculation module 80 is used to calculate and sort the similarity between the user query language vector and the user query code embedding vector;

[0262] The retrieval module 90 is used to obtain code retrieval results based on the sorted similarity and the multilingual code corpus.

[0263] The cross-natural language code retrieval model training device provided in this application adopts the cross-natural language code retrieval model training method in the above embodiments. It can solve the technical problem of how to remove the natural language-specific "fingerprint" features in the code, unify the embedding space alignment direction, and reduce the sampling distribution deviation during the training process, so as to improve the consistency and generalization ability of cross-language code retrieval. Compared with the prior art, the beneficial effects of the cross-natural language code retrieval model training device provided in this application are the same as the beneficial effects of the cross-natural language code retrieval model training method provided in the above embodiments, and other technical features in the cross-natural language code retrieval model training device are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0264] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the cross-natural language code retrieval model training method in Embodiment 1 above.

[0265] The following is for reference. Figure 6 It shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of this application. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0266] like Figure 6As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the electronic device to exchange data with other devices wirelessly or via wired communication. Although the diagrams show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0267] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0268] The electronic device provided in this application employs the cross-natural language code retrieval model training method described in the above embodiments. This method addresses the technical problem of how to remove language-specific "fingerprint" features from code, unify the embedding space alignment direction, and reduce sampling distribution bias during training, thereby improving the consistency and generalization ability of cross-language code retrieval. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the cross-natural language code retrieval model training method described in the above embodiments, and other technical features of this electronic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0269] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0270] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0271] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the cross-natural language code retrieval model training method described in the above embodiments.

[0272] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0273] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0274] The aforementioned computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device causes the following actions: to acquire an original corpus database; to construct training data based on the original corpus database, wherein the original corpus database includes multiple code segments and natural language descriptions corresponding to each code segment, each containing multiple natural languages; the training data includes a main language query text, a target language query text, and main language code; to obfuscate and invert the main language code to obtain main language code samples, wherein the main language code samples include positive and negative main language code samples; and to train an initial model using a gradient inversion layer based on the training data and the main language code samples to obtain a target model.

[0275] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages.

[0276] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application.

[0277] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0278] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., computer programs) for executing the aforementioned cross-natural language code retrieval model training method. This solves the technical problem of how to remove language-specific "fingerprint" features from code, unify the embedding space alignment direction, and reduce sampling distribution bias during training, thereby improving the consistency and generalization ability of cross-language code retrieval. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the cross-natural language code retrieval model training method provided in the above embodiments, and will not be repeated here.

[0279] This application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the cross-natural language code retrieval model training method described above.

[0280] The computer program product provided in this application can remove language-specific "fingerprint" features from the code, unify the alignment direction of the embedding space, and reduce sampling distribution bias during training, thereby improving the consistency and generalization ability of cross-language code retrieval. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the cross-natural language code retrieval model training method provided in the above embodiments, and will not be repeated here.

[0281] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A method for training a cross-natural language code retrieval model, characterized in that, The training method for the cross-natural language code retrieval model includes: Obtain the original corpus database, and construct training data based on the original corpus database. The original corpus database includes multiple code segments and natural language descriptions containing multiple natural languages ​​corresponding to each code segment. The training data includes the main language query text, the target language query text, and the main language code. The main language code is obfuscated and reversed to obtain a main language code sample, wherein the main language code sample includes a positive main language code sample and a negative main language code sample. The obfuscation refers to variable name substitution processing based on the abstract syntax tree, and the reversal refers to logical reversal processing based on the abstract syntax tree. Based on the training data and the main language code samples, the initial model is trained through a gradient inversion layer to obtain the target model.

2. The cross-natural language code retrieval model training method as described in claim 1, characterized in that, The step of constructing training data based on the original corpus database includes: The main language is determined from the original corpus database, and the natural language description of the main language is used as the main language query text, wherein the main language is a natural language; Select the code fragment corresponding to the main language query text from the original corpus database as the main language code; The target language query text is obtained based on the main language query text, wherein the main language and the target language are different natural languages, and the target language includes at least one natural language; Training data is constructed based on the main language query text, the target language query text, and the main language code.

3. The cross-natural language code retrieval model training method as described in claim 1, characterized in that, The step of obfuscating and reversing the main language code to obtain a main language code sample includes: The main language code is parsed to obtain a structured abstract syntax tree; Obfuscate the abstract syntax tree to obtain a positive sample of the main language code; Reverse the abstract syntax tree to obtain negative samples of the main language code; Based on the positive and negative samples of the main language code, a main language code sample is obtained.

4. The cross-natural language code retrieval model training method as described in claim 1, characterized in that, The initial model includes a first initial model and a second initial model, and the target model includes a first target model. The step of training the initial model using a gradient inversion layer based on the training data and the main language code samples to obtain the target model includes: The main language code sample is encoded using the second initial model to obtain the main language code embedding vector; The main language code embedding vector is forward-propagated through a gradient inversion layer to obtain the transformed main language code embedding vector. The transformed main language code embedding vector is input into the first initial model to obtain the predicted language category; Calculate the gradient based on the predicted language category and the preset language categories of the original corpus database; The parameters of the first initial model are updated based on the gradient to obtain the first target model.

5. The cross-natural language code retrieval model training method as described in claim 4, characterized in that, The target model further includes a second target model. The step of training the initial model using a gradient reversal layer based on the training data and the main language code samples to obtain the target model includes: The inverted gradient is calculated based on the gradient through the gradient inversion layer; The parameters of the second initial model are updated based on the inverted gradient to obtain the second target model.

6. The cross-natural language code retrieval model training method as described in claim 5, characterized in that, The initial model further includes a third initial model, and the target model further includes a third target model. The step of training the initial model using a gradient reversal layer based on the training data and the main language code samples to obtain the target model includes: The main language code sample is input into the second target model to obtain the main language target code embedding vector; The main language query text and the target language query text are respectively input into the third initial model to obtain the main language query vector and the target language query vector; Based on the host language target code embedding vector, the host language query vector, and the target language query vector, minimize the composite retrieval loss; The weights of the third initial model are updated based on the composite retrieval loss to obtain the third target model.

7. The cross-natural language code retrieval model training method as described in claim 6, characterized in that, After obtaining the third target model, the method further includes: The training data is sampled to construct a validation set, wherein the validation set includes language validation query text and language validation code, the language validation query text includes language query text in multiple natural languages, and the language validation code includes code snippets corresponding to multiple natural languages; The validation set is processed by the third target model and the second target model to obtain the validation language query vector and the validation code embedding vector for each natural language. A comprehensive quantitative index is calculated based on the validation set, the validation language query vector for each natural language, and the validation code embedding vector for each natural language. If the comprehensive quantitative index does not meet the preset conditions, then the compensation weight is calculated based on the comprehensive quantitative index and the validation set. The parameters of the third objective model are updated based on the compensation weights to obtain the optimized third objective model.

8. The cross-natural language code retrieval model training method as described in claim 7, characterized in that, The steps of obtaining the verification language query vector and the verification code embedding vector for each natural language by passing the verification set through the third target model and the second target model respectively include: Obtain all validation query texts and corresponding language validation codes for each natural language in the validation set; The verification query text for each natural language is input into the third target model to obtain the verification language query vector for each natural language. The language verification code for each natural language is input into the second target model to obtain the verification code embedding vector for each natural language.

9. The cross-natural language code retrieval model training method as described in claim 7, characterized in that, The step of calculating the comprehensive quantitative index based on the validation set, the validation language query vector for each natural language, and the validation code embedding vector for each natural language includes: Calculate the similarity between the verification language query vector and the corresponding verification code embedding vector for each natural language and sort them to obtain a sorted list for each natural language; A comprehensive quantitative index is calculated based on any two sorted lists in the sorted lists of each natural language and the validation set.

10. The cross-natural language code retrieval model training method as described in claim 9, characterized in that, The step of calculating the comprehensive quantitative index based on any two sorted lists in the sorted lists of each natural language and the validation set includes: Calculate the similarity between any two natural languages ​​based on any two sorted lists in the sorted lists of each natural language; A comprehensive quantitative index is calculated based on the number of natural languages ​​in the validation set and the similarity scores of each language.

11. The cross-natural language code retrieval model training method as described in any one of claims 7 to 10, characterized in that, The step of calculating compensation weights based on the comprehensive quantitative index and the validation set if the comprehensive quantitative index does not meet the preset conditions includes: If the comprehensive quantitative index does not meet the preset conditions, then the similarity between any two natural languages ​​in the verification set is obtained; The compensation weight is calculated based on the similarity between any two natural languages ​​in the validation set and the comprehensive quantitative index.

12. The cross-natural language code retrieval model training method as described in claim 11, characterized in that, The step of calculating the compensation weight based on the similarity between any two natural languages ​​in the validation set and the comprehensive quantitative index includes: The deviation factor is calculated based on the difference between the similarity of any two natural languages ​​and the comprehensive quantitative index. The deviation factor is used to calculate the activation value through an activation function; Based on the language query texts of the various natural languages, the third initial model is used to obtain the initial verification language query vector for each natural language; The compensation weight is calculated based on the activation value and the initial verification language query vector.

13. A cross-natural language code retrieval method, characterized in that, Based on the cross-natural language code retrieval model training method according to any one of claims 1 to 12, a second target model and a third target model of the target model are determined, wherein the cross-natural language code retrieval method includes: In response to a user query command, obtain the user query language text corresponding to the user query command; The user query language text is input into the third target model to obtain the user query language vector; Determine the user query code based on the user query language text; The user query code is input into the second target model to obtain the user query code embedding vector; Calculate and sort the similarity between the user query language vector and the user query code embedding vector; The code retrieval results are obtained based on the sorted similarity and the multilingual code corpus.

14. A training device for a cross-natural language code retrieval model, characterized in that, The cross-natural language code retrieval model training device includes: The sample construction module is used to acquire the original corpus database and construct training data based on the original corpus database. The original corpus database includes multiple code segments and natural language descriptions containing multiple natural languages ​​corresponding to each code segment. The training data includes the main language query text, the target language query text, and the main language code. The sample conversion module is used to obfuscate and reverse the main language code to obtain main language code samples. The main language code samples include positive main language code samples and negative main language code samples. The obfuscation refers to variable name substitution processing based on the abstract syntax tree, and the reversal refers to logical reversal processing based on the abstract syntax tree. The training module is used to train the initial model using the training data and the main language code samples through a gradient inversion layer to obtain the target model.

15. A cross-natural language code retrieval device, characterized in that, Based on the cross-natural language code retrieval model training method according to any one of claims 1 to 12, a second target model and a third target model of the target model are determined, and the cross-natural language code retrieval device comprises: The response module is used to respond to a user query command and obtain the user query language text corresponding to the user query command; The query encoding module is used to input the user query language text into the third target model to obtain the user query language vector; The determination module is used to determine the user query code based on the user query language text; The code encoding module is used to input the user query code into the second target model to obtain the user query code embedding vector; The calculation module is used to calculate and sort the similarity between the user query language vector and the user query code embedding vector. The retrieval module is used to obtain code retrieval results based on the sorted similarity and the multilingual code corpus.

16. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the cross-natural language code retrieval model training method as described in any one of claims 1 to 12 or the cross-natural language code retrieval method as described in claim 13.

17. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the cross-natural language code retrieval model training method as described in any one of claims 1 to 12 or the cross-natural language code retrieval method as described in claim 13.

Images