Code intent recognition model training method, code intent recognition method and device

Abstract

A training method and device of a code intent recognition model, the code intent recognition model comprising a first graph neural network and a first large language module, the method comprising: obtaining a first code in a text form, and extracting a first vector representation of a code attribute graph corresponding to the first code; inputting the first vector representation into the first graph neural network to obtain a second vector representation; extracting a third vector representation corresponding to the first code, obtaining a fusion vector representation based on the second vector representation and the third vector representation, and inputting the fusion vector representation into the first large language module to obtain a first output result indicating a code intent; determining a first difference loss according to a preset label and the first output result, obtaining a second output result indicating the code intent based on the second vector representation and a preset classifier, determining a second difference loss according to the second output result and the preset label, and determining a target loss based on the first difference loss and the second difference loss, and updating network parameters of the first graph neural network with the purpose of reducing the target loss.

Description

Technical Field

[0001] This specification relates to the field of neural network technology in one or more embodiments, and particularly to a training method for a code intent recognition model, a code intent recognition method and apparatus. Background Technology

[0002] Code intent recognition refers to identifying the function, purpose, or intent of code snippets through program analysis or machine learning methods. In recent years, with the rapid development of Large Language Models (LLMs) technology, code intent recognition using LLMs has become a common approach. However, existing code intent recognition schemes typically process the code as plain text input into the LLM, which leads to poor recognition performance in some scenarios. Summary of the Invention

[0003] The embodiments in this specification aim to provide a method for executing operational commands on a confidential container. It can extract features of a complete code attribute graph through a graph neural network and combine them with features of the code text to input into a large language model, thereby providing the large language model with complete structured information and textual description information in the code, which greatly improves the accuracy of the code intent recognition results output by the large language model and solves the shortcomings of the prior art.

[0004] According to the first aspect, a training method for a code intent recognition model is provided, the code intent recognition model comprising a first graph neural network and a first large language module, the method comprising:

[0005] Obtain the first code in text form, determine the code attribute map corresponding to the first code, and extract the first vector representation corresponding to the code attribute map;

[0006] The first vector representation is input into the first graph neural network to obtain the second vector representation;

[0007] Extract the third vector representation corresponding to the first code, and obtain the fused vector representation based on the second and third vector representations. Input the fused vector representation into the first large language module to obtain the first output result indicating the code intent of the first code.

[0008] The first difference loss is determined based on the preset label corresponding to the first code and the first output result;

[0009] Based on the second vector representation and a preset classifier, a second output result indicating the code intent of the first code is determined, and a second difference loss is determined based on the second output result and a preset label;

[0010] The target loss is determined based on the first difference loss and the second difference loss; the network parameters of the first graph neural network are updated with the aim of reducing the target loss.

[0011] In one possible implementation, the code intent recognition model further includes a projector;

[0012] The step of obtaining the fused vector representation based on the second vector representation and the third vector representation includes: inputting the second vector representation into the projector to obtain a fourth vector representation, wherein the dimension of the fourth vector representation is smaller than that of the second vector representation;

[0013] Based on the second and third vector representations, a fused vector representation is obtained, including: based on the fourth and third vector representations, a fused vector representation is obtained;

[0014] With the aim of reducing the target loss, the network parameters of the first graph neural network are updated, including:

[0015] With the aim of reducing the target loss, the network parameters of the projector and the first graph neural network are updated.

[0016] In one possible implementation, the dimension represented by the third vector is a preset dimension of the input vector of the first large language module; the dimension represented by the fourth vector is smaller than the preset dimension.

[0017] Extract the third vector representation corresponding to the first code. Based on the second and third vector representations, obtain the fused vector representation, including:

[0018] Obtain a first prompt word, which contains a first code and also includes a first identifier represented by the second vector;

[0019] Extract the target vector representation corresponding to the first prompt word. The target vector contains a first part corresponding to the first code and a second part corresponding to the first identifier. This means that the second part in the target vector representation is replaced with the second vector representation to obtain the fusion vector.

[0020] In one possible implementation, determining the code attribute map corresponding to the first code includes:

[0021] An abstract syntax tree corresponding to the first code is determined. Based on the abstract syntax tree, a control flow graph and a data flow graph corresponding to the first code are determined. Based on the abstract syntax tree, the control flow graph, and the data flow graph, the code attribute graph is obtained.

[0022] In one possible implementation, the extraction of the third vector representation corresponding to the first code includes:

[0023] The third vector representation corresponding to the first code is extracted through the text embedding layer.

[0024] In one possible implementation, the extraction of the third vector representation corresponding to the first code includes:

[0025] A truncation operation is applied to the first code to obtain the first truncated part of the first code. The vector representation of the first truncated part is extracted and used as the third vector representation.

[0026] According to the second aspect, a code intent recognition method is provided, including:

[0027] Obtain the second code in text form, determine the second code attribute map corresponding to the second code, and extract the fifth vector representation corresponding to the second code attribute map;

[0028] The fifth vector representation is input into the first graph neural network included in the code intent recognition model trained by the method described in the first aspect, to obtain the sixth vector representation;

[0029] Extract the seventh vector representation corresponding to the second code. Based on the sixth vector representation, obtain the second fusion vector representation. Input the second fusion vector representation into the preset large language model to obtain the third output result indicating the code intent of the second code.

[0030] According to a third aspect, a training apparatus for a code intent recognition model is provided, the code intent recognition model comprising a first graph neural network and a first large language module, the apparatus comprising:

[0031] The first acquisition unit is configured to acquire a first code in text form, determine a code attribute map corresponding to the first code, and extract a first vector representation corresponding to the code attribute map;

[0032] The first processing unit is configured to input the first vector representation into the first graph neural network to obtain the second vector representation;

[0033] The second processing unit is configured to extract the third vector representation corresponding to the first code, obtain a fused vector representation based on the second vector representation and the third vector representation, input the fused vector representation into the first large language module, and obtain a first output result indicating the code intent of the first code;

[0034] The training unit is configured to: determine a first difference loss based on the preset label corresponding to the first code and the first output result; determine a second output result indicating the code intent of the first code based on the second vector representation and the preset classifier; determine a second difference loss based on the second output result and the preset label; determine a target loss based on the first difference loss and the second difference loss; and update the network parameters of the first graph neural network with the aim of reducing the target loss.

[0035] According to the fourth aspect, a code intent recognition device is provided, comprising:

[0036] The second acquisition unit is configured to acquire the second code in text form, determine the second code attribute map corresponding to the second code, and extract the fifth vector representation corresponding to the second code attribute map;

[0037] The third processing unit is configured to input the fifth vector representation into the first graph neural network included in the code intent recognition model trained by the first aspect method, and obtain the sixth vector representation.

[0038] The recognition unit is configured to extract the seventh vector representation corresponding to the second code, obtain the second fusion vector representation based on the sixth vector representation and the sixth vector representation, input the second fusion vector representation into a preset large language model, and obtain a third output result indicating the code intent of the second code.

[0039] According to a fifth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the methods described in the first and second aspects.

[0040] According to a sixth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the methods described in the first and second aspects.

[0041] By utilizing one or more of the methods, systems, computing devices, and storage media mentioned above, features of the complete code attribute graph can be extracted through graph neural networks (GNNs) and combined with the features of the code text, which are then input into a large language model. This provides the large language model with complete structured information and textual description information from the code, greatly improving the accuracy of the large language model's output in recognizing the code intent. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A schematic diagram illustrating a scheme for identifying code intent using a large language model is shown.

[0044] Figure 2 A schematic diagram illustrating a scheme for identifying the intent of a code is shown;

[0045] Figure 3 A schematic diagram illustrating an alternative scheme for identifying code intent is shown;

[0046] Figure 4 A schematic diagram illustrating a training method for a code intent recognition model according to an embodiment of this specification is shown.

[0047] Figure 5 A flowchart illustrating a training method for a code intent recognition model according to an embodiment of this specification is shown.

[0048] Figure 6 A schematic diagram illustrating the acquisition of code attribute graphs according to embodiments of this specification;

[0049] Figure 7 A schematic diagram illustrating a training method for a code intent recognition model according to another embodiment of this specification is shown.

[0050] Figure 8 A flowchart illustrating a code intent recognition method according to an embodiment of this specification is shown.

[0051] Figure 9 A structural diagram of a training apparatus for a code intent recognition model according to an embodiment of this specification is shown.

[0052] Figure 10 A structural diagram of a code intent recognition device according to an embodiment of this specification is shown. Detailed Implementation

[0053] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0054] In this specification, the Large Language Model (LLM) may also be referred to simply as the Large Model. A Large Language Model is a natural language processing model based on deep learning techniques, typically with billions to hundreds of billions or even more parameters, possessing powerful language understanding and generation capabilities. Large Language Models can employ the Transformer architecture or its variants (such as GPT, BERT, etc.), which utilizes an attention mechanism to globally model sequential data, efficiently handling long-distance dependencies and thus performing exceptionally well in natural language tasks. Large Language Models learn the statistical features and semantic relationships of language through pre-training on large-scale corpora, giving them excellent generalization capabilities. The core capabilities of Large Language Models include, but are not limited to: understanding contextual semantics, generating coherent and grammatically correct text, performing logical reasoning, and handling multi-task scenarios. Its usage typically includes two modes: direct inference and fine-tuning. In direct inference mode, the user guides the Large Language Model to generate specific outputs by designing prompts. Cue words can be task descriptions or instructions in text form, used to stimulate the semantic understanding and generation capabilities of large language models. In fine-tuning mode, large language models are further trained on small-scale datasets in specific domains to optimize their performance on specific tasks. The powerful generalization ability and flexibility of large language models make them an important tool in the field of artificial intelligence, providing efficient and accurate solutions for automated text generation and understanding.

[0055] In some embodiments, large language models can also understand and generate data from other modalities (such as visual and audio data). In this case, large language models can also be called multimodal large language models (MLLMs). MLLMs provide a richer and more natural interactive experience by integrating multiple types of input and output, such as text, images, and sound. The core advantage of MLLMs lies in their ability to process and understand information from different modalities and fuse this information to complete complex tasks. For example, MLLMs can analyze an image and generate descriptive text, or generate a corresponding image based on a text description. This cross-modal understanding and generation capability makes MLLMs widely applicable across multiple fields.

[0056] As mentioned earlier, code intent recognition refers to identifying the function, purpose, or intent of code snippets through program analysis or machine learning methods. For example, in one scenario, it can be determined whether the code's function is sorting, searching, or encryption. In another scenario, it can be determined whether the code's function is login, logout, registration, sending a verification code, or verifying a verification code. In recent years, with the rapid development of Large Language Model (LLM) technology, identifying the code intent of input code through LLM has become a commonly used code intent recognition scheme. Figure 1 This diagram illustrates a scheme for identifying code intent using a large language model. For example... Figure 1 As shown, code in text form (e.g., source code) can be input into a pre-defined large language model, and the model's output will then indicate the code's intent. However, this approach also has the following problems: because it treats the code as plain text, it ignores structural information crucial for determining code intent, such as control flow and data dependencies, leading to insufficient accuracy in the large language model's output. Furthermore, the full amount of code text is typically enormous, causing the large language model to consume significant computational resources when processing it.

[0057] Figure 2 This diagram illustrates a scheme for identifying the intent of a code. For example... Figure 2 As shown, graph structure information (such as control flow graphs, data flow graphs, etc.) can be extracted from the original code text, and then the graph structure information is input into a graph neural network (GNN) to obtain the code intent recognition result. However, this approach also has the following problems: although using GNN alone can effectively use the code graph structure information for code intent recognition, it ignores the descriptive information contained in the code text, which is usually also valuable for code intent recognition, making the recognition result of this approach less accurate.

[0058] Figure 3 This illustrates a scheme based on another method for identifying the intent of the code. For example... Figure 3As shown, this approach extracts the graph structure (e.g., control flow graph, data flow graph) from the code text, then generates a natural language description of the graph structure using a graph-to-text model. Both the natural language description and the original code are then input into a large language model to obtain the intent recognition result for the original code. However, this approach also has the following problems: although it provides additional text-based graph description information to the large language model, this graph description often suffers from the loss or distortion of graph structure information, thus reducing the accuracy of the final recognition result. Another approach modifies the underlying Transformer network architecture of the large language model, allowing graph structure information to be directly input into the modified large language model to obtain the recognition result. However, this approach requires modification of the large language model's architecture itself, limiting its applicability; that is, this approach cannot identify the code intent using a conventional large language model (without modifying the underlying structure).

[0059] To address the aforementioned issues, this specification provides a training method for a code intent recognition model, as well as a corresponding code intent recognition method. Figure 4 A schematic diagram illustrates a training method for a code intent recognition model according to an embodiment of this specification. For example... Figure 4 As shown, firstly, the corresponding Code Property Graph (CPG) can be determined based on the code text. Initial features P1 of the CPG can be extracted (e.g., encoding the node and edge information in the CPG to obtain these initial features). P1 is then input into a Graph Neural Network (GNN) to obtain the output graph embedding features P2. The code text can also be input into a text embedding layer to obtain text features P3. Then, a fusion feature P4 can be obtained based on the graph embedding features P2 and the text features P3. The fusion feature P4 is input into a large language model to obtain the output result R1. Based on the intent label corresponding to the code text and the output result R1, the difference loss L1 is determined. The graph embedding feature P2 is then input into a pre-defined classifier to obtain the output result R2, yielding the difference loss L2. Based on L1 and L2, the comprehensive loss L is determined. The parameters of the graph neural network are updated with the aim of minimizing the comprehensive loss L.

[0060] After the graph neural network (GNN) is trained, it can be combined with a large language model to identify the code intent of the test code. Specifically, the initial features of the code attribute graph of the test code can be input into the trained GNN to obtain the corresponding graph embedding features. The code text of the test code can then be input into a text embedding layer to obtain the corresponding text features. Then, based on the graph embedding features and the text features P3, a fused feature can be obtained. This fused feature is then input into the large language model to obtain the identification result of the code intent of the test code.

[0061] The advantages of this method are as follows: First, compared to methods that input pure text-based code into a large language model to obtain code intent recognition results, this method extracts features from the complete code attribute graph using a graph neural network (GNN) and combines them with the features of the code text before inputting them into the large language model. This provides the large language model with complete structured information from the code, significantly improving the accuracy of the large language model's output in recognizing code intent. Second, compared to methods that simply use GNNs to recognize code intent, this method not only obtains features from the complete code attribute graph but also features from the code text, combining them before inputting them into the large language model. This allows the large language model to recognize the code intent based on the structured information in the code and valuable textual descriptions. Compared to methods that simply use networks to recognize code intent, this method also has higher recognition accuracy. Third, compared to methods that generate a natural language description of the code graph structure and then input both the natural language description and the original code into the large language model to obtain the intent recognition results for the original code, this method inputs the complete code attribute graph features into the large language model, minimizing the loss or distortion of graph structure information often present in textual descriptions. This makes the structured information of the code obtained from the large language pattern more complete and faithful, resulting in more accurate recognition of the code intent. Fourth, compared to methods that modify the structure of the large language model, this method not only does not require modification of the structure of the specific large language model itself, but the trained GNN network can also be used as an external plugin, applicable to various specific types of large language patterns. Therefore, it reduces the complexity of model training and greatly expands the applicability of this approach.

[0062] The following section further elaborates on the detailed process of this method. Figure 5 A flowchart illustrating a training method for a code intent recognition model according to an embodiment of this specification is shown. The code intent recognition model includes a first graph neural network and a first large language module, such as... Figure 5 The method includes at least the following steps:

[0063] Step S501: Obtain the first code in text form, determine the code attribute map corresponding to the first code, and extract the first vector representation corresponding to the code attribute map;

[0064] Step S503: Input the first vector representation into the first graph neural network to obtain the second vector representation;

[0065] Step S505: Extract the third vector representation corresponding to the first code; based on the second and third vector representations, obtain the fused vector representation; input the fused vector representation into the first large language module to obtain the first output result indicating the code intent of the first code.

[0066] Step S507: Determine the first difference loss based on the preset label corresponding to the first code and the first output result;

[0067] Step S509: Based on the second vector representation and a preset classifier, determine a second output result indicating the code intent of the first code; and determine a second difference loss based on the second output result and a preset label.

[0068] Step S511: Determine the target loss based on the first difference loss and the second difference loss; update the network parameters of the first graph neural network with the aim of reducing the target loss.

[0069] First, in step S501, the first code in text form can be obtained. Then, the code attribute graph corresponding to the first code is determined, and the first vector representation corresponding to the code attribute graph is extracted. In this step, the code in text form (i.e., the first code) can be obtained. In one embodiment, the obtained code may be the source code of an application or software for different specific purposes; this specification does not limit this.

[0070] After obtaining the first code, the code attribute graph corresponding to the first code can be determined. In different embodiments, the specific method for determining the code attribute graph corresponding to the first code can differ. In one embodiment, an abstract syntax tree (API) corresponding to the first code can be determined; based on the API, a control flow graph and a data flow graph corresponding to the first code can be determined; and based on the API, the control flow graph, and the data flow graph, the code attribute graph is obtained, such as... Figure 6 As shown.

[0071] After obtaining the code attribute graph, a first vector representation (also known as an initial vector representation) corresponding to the code attribute graph can be extracted. In different embodiments, the specific method for determining the code attribute graph corresponding to the first code can vary. In one embodiment, for example, the code attribute graph can be traversed to extract node information and edge information. Then, the obtained node information and edge information are encoded to obtain the first vector representation. In different embodiments, the specific algorithm for traversing the code attribute graph can vary. In one embodiment, for example, a depth-first search (DFS) algorithm can be used to traverse the code attribute graph. In different embodiments, the specific method for encoding the node information and edge information can also vary. In one embodiment, for example, multi-hot encoding can be performed on the node information and edge information to obtain the first vector representation.

[0072] After obtaining the first vector representation, in step S503, the first vector representation can be input into the first graph neural network to obtain the second vector representation.

[0073] Graph Neural Networks (GNNs) are deep learning models used to process graph-structured data. In different embodiments, the first graph neural network can be different specific types of graph neural networks. In one embodiment, the first graph neural network can be either a graph convolutional network (GCN) or a graph attention network (GAT).

[0074] In this step, the initial vector representation obtained in step S501 can be input into the graph neural network to obtain the latent semantic representation of the code attribute graph, i.e., the second vector representation. For example, the initial vector representation generated by multi-hot encoding usually represents the attributes of nodes and edges in the code attribute graph (CPG) with discrete, symbolic labels, which do not have continuity or semantic association. The second vector representation output by the graph neural network can transform the original discrete labels into coordinates in a high-dimensional continuous space that contains the contextual relationships and global intent in the code attribute graph.

[0075] In step S505, the third vector representation corresponding to the first code can be extracted. Then, based on the second and third vector representations, a fused vector representation can be obtained. Finally, the fused vector representation is input into the first large language module to obtain a first output result indicating the code intent of the first code.

[0076] In different embodiments, the specific method for extracting the third vector representation corresponding to the first code can vary. In one embodiment, the third vector representation corresponding to the first code can be extracted through a text embedding layer. In another embodiment, a truncation operation is applied to the first code to obtain a first truncated portion, and the vector representation of the first truncated portion is extracted as the third vector representation. The truncation operation involves removing and retaining a portion of the first code while discarding the rest. In different embodiments, the specific method for performing this truncation operation can vary. In one embodiment, for example, the first N lines of code can be truncated as the first truncated portion, where N is an integer. In different specific embodiments, the specific value of N can be different.

[0077] After obtaining the three vectors, a fused vector representation can be obtained based on the second and third vector representations. In different embodiments, the specific method of obtaining the fused vector representation based on the second and third vector representations can vary. Since the feature space of the GNN output vector often differs from the feature space of the input vector of a large language model in actual production scenarios, a projector is included in one embodiment to align the feature spaces of the two. Furthermore, the second vector representation can be input into the projector to obtain a fourth vector representation, the dimension of which is smaller than that of the second vector representation; based on the fourth and third vector representations, a fused vector representation is obtained, such as... Figure 7 As shown, a projector is a neural network module used to map input vector data from one feature space to another. In different embodiments, the neural network structure of the projector can differ. In one embodiment, the projector may be, for example, a multilayer perceptron (MLP).

[0078] In another embodiment, the dimension of the third vector representation is a preset dimension of the input vector of the first large language module; the dimension of the fourth vector representation is smaller than the preset dimension. Furthermore, a first prompt word can be obtained, the first prompt word containing a first code, and the first prompt word also including a first identifier of the second vector representation; the target vector representation corresponding to the first prompt word is extracted, the target vector containing a first part corresponding to the first code and a second part corresponding to the first identifier, indicating that the second part in the target vector representation is replaced with the second vector representation to obtain the fusion vector.

[0079] After obtaining the fusion vector, the fusion vector representation can be input into the first large language module to obtain the first output result. This output result can indicate the code intent of the first code. In different embodiments, the first large language module can be based on different specific types of large language models, and this specification does not limit this.

[0080] After obtaining the first output result, in step S507, a first difference loss can be determined based on the preset label corresponding to the first code and the first output result. The preset label corresponding to the first code, i.e., the preset training label, is used to indicate the code intent of the first code. In this step, the first difference loss can be determined based on the training label and the first output result. In different embodiments, the specific method for determining the first difference loss can be different. In one embodiment, for example, the cross-entropy loss between the training label and the first output result can be used as the first difference loss.

[0081] After obtaining the second vector representation, in step S509, a second output result indicating the code intent of the first code can be determined based on the second vector representation and a preset classifier. Then, a second difference loss is determined based on the second output result and a preset label.

[0082] In this step, the second vector representation can be input into the classifier to obtain a prediction result of the code intent for the first code (i.e., the second output result). A classifier is a model used to map the input vector representation to discrete categories or their probability distributions. In different embodiments, the second output result can be determined using classifiers with different network structures; this specification does not limit this.

[0083] After obtaining the second output, a second difference loss can be determined based on the training labels of the first code and the second output. Similar to determining the first difference, the specific method for determining the second difference loss can also differ in different embodiments. For example, in one embodiment, the cross-entropy loss between the training labels and the second output can be used as the second difference loss.

[0084] It should be noted that the execution order of steps S509 and S505-S507 may differ in different embodiments. For example, in one embodiment, steps S505-S507 may be executed before step S509. In another embodiment, step S509 may be executed before steps S505-S507. In yet another embodiment, steps S505-S507 may be executed in parallel with step S509.

[0085] After determining the first difference loss and the second difference loss, in step S511, a target loss can be determined based on the first difference loss and the second difference loss. The network parameters of the first graph neural network are then updated with the aim of minimizing the target loss.

[0086] In different embodiments, the specific method for determining the target loss based on the first difference loss and the second difference loss may differ. In one embodiment, for example, the target loss may be determined based on the sum of the first difference loss and the second difference loss. In another embodiment, for example, the target loss may be determined based on the weighted sum of the first difference loss and the second difference loss. In different specific embodiments, the respective weights of the first difference loss and the second difference loss may be different, and this specification does not limit this.

[0087] After obtaining the target loss, the network parameters of the first graph neural network can be updated with the aim of minimizing the target loss. In different embodiments, the specific method of updating the neural network parameters to minimize the target loss can vary. In one embodiment, for example, before determining the target loss value, the gradient information of the target loss relative to the weights and biases of each layer can be obtained by backpropagation, starting from the output layer of the graph neural network and working backwards along the network structure. This gradient vector indicates the direction and magnitude of the adjustment of each network parameter to reduce the target loss. Furthermore, the individual network parameters can be updated based on these gradient signals.

[0088] In the embodiments including the projector described above, the network parameters of the projector and the first graph neural network can also be updated with the aim of reducing the target loss. The specific method for updating the network parameters of the projector is similar to that for updating the graph neural network, and will not be described again here.

[0089] Correspondingly, a method for code intent recognition is also provided. Figure 8 A flowchart illustrating a code intent recognition method according to an embodiment of this specification is shown. Figure 8 As shown, the method includes at least the following steps:

[0090] Step S801: Obtain the second code in text form, determine the second code attribute map corresponding to the second code, and extract the fifth vector representation corresponding to the second code attribute map;

[0091] Step S803: Input the fifth vector representation into the first graph neural network included in the code intent recognition model trained by the above method to obtain the sixth vector representation;

[0092] Step S805: Extract the seventh vector representation corresponding to the second code. Based on the sixth vector representation, obtain the second fusion vector representation. Input the second fusion vector representation into the preset large language model to obtain the third output result indicating the code intent of the second code.

[0093] According to yet another embodiment, a training apparatus for a code intent recognition model is also provided. Figure 9 This diagram illustrates a structural arrangement for training a code intent recognition model according to an embodiment of this specification. Figure 9 As shown, the code intent recognition model includes a first graph neural network and a first large language module, and the device 900 includes:

[0094] The acquisition unit 901 is configured to acquire a first code in text form, determine the code attribute map corresponding to the first code, and extract a first vector representation corresponding to the code attribute map;

[0095] The representation unit 902 is configured to input the first vector representation into the first graph neural network to obtain the second vector representation;

[0096] The output unit 903 is configured to extract the third vector representation corresponding to the first code, obtain a fused vector representation based on the second vector representation and the third vector representation, input the fused vector representation into the first large language module, and obtain a first output result indicating the code intent of the first code;

[0097] The first loss unit 904 is configured to determine a first difference loss based on the preset label corresponding to the first code and the first output result;

[0098] The second loss unit 905 is configured to determine a second output result indicating the code intent of the first code based on the second vector representation and a preset classifier, and to determine a second difference loss based on the second output result and a preset label.

[0099] Training unit 906 is configured to determine the target loss based on the first difference loss and the second difference loss; and to update the network parameters of the first graph neural network with the aim of reducing the target loss.

[0100] According to yet another embodiment, a code intent recognition device is also provided. Figure 10 A structural diagram of a code intent recognition device according to an embodiment of this specification is shown. Figure 10 As shown, the device 1000 includes:

[0101] The second acquisition unit 1001 is configured to acquire the second code in text form, determine the second code attribute map corresponding to the second code, and extract the fifth vector representation corresponding to the second code attribute map.

[0102] The third processing unit 1002 is configured to input the fifth vector representation into the first graph neural network included in the code intent recognition model trained by the above method to obtain the sixth vector representation.

[0103] The recognition unit 1003 is configured to extract the seventh vector representation corresponding to the second code, obtain the second fusion vector representation based on the sixth vector representation and the sixth vector representation, input the second fusion vector representation into a preset large language model, and obtain a third output result indicating the code intent of the second code.

[0104] In another aspect, embodiments of this specification provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform any of the methods described above.

[0105] In another aspect, embodiments of this specification provide a computing device, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement any of the methods described above.

[0106] It should be understood that the descriptions such as "first" and "second" in this article are merely for the sake of simplicity in description and to distinguish similar concepts, and do not have any other limiting function.

[0107] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0108] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0109] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0110] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0111] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0112] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0113] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0114] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0115] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0116] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0117] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0118] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0120] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0121] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A method for training a code intent recognition model, the code intent recognition model comprising a first graph neural network and a first large language module, the method comprising: Obtain the first code in text form, determine the code attribute map corresponding to the first code, and extract the first vector representation corresponding to the code attribute map; The first vector representation is input into the first graph neural network to obtain the second vector representation; Extract the third vector representation corresponding to the first code, and obtain the fused vector representation based on the second and third vector representations. Input the fused vector representation into the first large language module to obtain the first output result indicating the code intent of the first code. The first difference loss is determined based on the preset label corresponding to the first code and the first output result; Based on the second vector representation and a preset classifier, a second output result indicating the code intent of the first code is determined; Based on the second output and the preset labels, determine the second difference loss; The target loss is determined based on the first difference loss and the second difference loss; With the aim of reducing the target loss, the network parameters of the first graph neural network are updated.

2. The method according to claim 1, wherein, The code intent recognition model also includes a projector; The step of obtaining the fused vector representation based on the second vector representation and the third vector representation includes: inputting the second vector representation into the projector to obtain a fourth vector representation, wherein the dimension of the fourth vector representation is smaller than that of the second vector representation; Based on the second and third vector representations, a fused vector representation is obtained, including: based on the fourth and third vector representations, a fused vector representation is obtained; With the aim of reducing the target loss, the network parameters of the first graph neural network are updated, including: With the aim of reducing the target loss, the network parameters of the projector and the first graph neural network are updated.

3. The method according to claim 2, wherein, The dimension represented by the third vector is the preset dimension of the input vector of the first large language module; the dimension represented by the fourth vector is smaller than the preset dimension. Extract the third vector representation corresponding to the first code. Based on the second and third vector representations, obtain the fused vector representation, including: Obtain a first prompt word, which contains a first code and also includes a first identifier represented by the second vector; Extract the target vector representation corresponding to the first prompt word. The target vector contains a first part corresponding to the first code and a second part corresponding to the first identifier. This means that the second part in the target vector representation is replaced with the second vector representation to obtain the fusion vector.

4. The method according to claim 1, wherein, The code attribute map corresponding to the first code is determined, including: An abstract syntax tree corresponding to the first code is determined. Based on the abstract syntax tree, a control flow graph and a data flow graph corresponding to the first code are determined. Based on the abstract syntax tree, the control flow graph, and the data flow graph, the code attribute graph is obtained.

5. The method according to claim 1, wherein, Extract the third vector representation corresponding to the first code, including: The third vector representation corresponding to the first code is extracted through the text embedding layer.

6. The method according to claim 1, wherein, Extract the third vector representation corresponding to the first code, including: A truncation operation is applied to the first code to obtain the first truncated part of the first code. The vector representation of the first truncated part is extracted and used as the third vector representation.

7. A method for code intent recognition, comprising: Obtain the second code in text form, determine the second code attribute map corresponding to the second code, and extract the fifth vector representation corresponding to the second code attribute map; The fifth vector representation is input into the first graph neural network included in the code intent recognition model trained by the method of claim 1 to obtain the sixth vector representation; Extract the seventh vector representation corresponding to the second code. Based on the sixth vector representation, obtain the second fusion vector representation. Input the second fusion vector representation into the preset large language model to obtain the third output result indicating the code intent of the second code.

8. A training apparatus for a code intent recognition model, the code intent recognition model comprising a first graph neural network and a first large language module, the apparatus comprising: The acquisition unit is configured to acquire first code in text form, determine the code attribute map corresponding to the first code, and extract the first vector representation corresponding to the code attribute map; The representation unit is configured to input the first vector representation into the first graph neural network to obtain the second vector representation; The output unit is configured to extract the third vector representation corresponding to the first code, obtain a fused vector representation based on the second and third vector representations, input the fused vector representation into the first large language module, and obtain a first output result indicating the code intent of the first code. The first loss unit is configured to determine a first difference loss based on the preset label corresponding to the first code and the first output result; The second loss unit is configured to determine a second output result indicating the code intent of the first code based on the second vector representation and a preset classifier, and to determine a second difference loss based on the second output result and a preset label. The training unit is configured to determine the target loss based on the first difference loss and the second difference loss; With the aim of reducing the target loss, the network parameters of the first graph neural network are updated.

9. A code intent recognition device, comprising: The acquisition unit is configured to acquire the second code in text form, determine the second code attribute map corresponding to the second code, and extract the fifth vector representation corresponding to the second code attribute map; The representation unit is configured to input the fifth vector representation into the first graph neural network included in the code intent recognition model trained by the method of claim 1, and obtain the sixth vector representation. The recognition unit is configured to extract the seventh vector representation corresponding to the second code, obtain the second fusion vector representation based on the sixth vector representation and the sixth vector representation, input the second fusion vector representation into a preset large language model, and obtain a third output result indicating the code intent of the second code.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-7.

11. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-7.

Images