Semantic network-based large language model collaborative patch incorporation evaluation method

Abstract

The embodiment of the invention provides a semantic network-based large language model collaborative patch merging evaluation method, which belongs to the technical field of data processing, and specifically comprises the following steps of: obtaining original patch information of a target operating system; performing semantic enrichment on the original patch information based on a large language model, and respectively generating URL analysis and code analysis by taking submitted information and code change as input; performing block processing on the code change in the patch information; submitted information, URL analysis and code analysis are spliced; inputting the semantic enhanced submission information into a BERT model to extract features, and obtaining a first semantic embedding vector corresponding to the submission information; performing embedding operation on the code block, and inputting the code block into a CodeBERT model to obtain a second semantic embedding vector corresponding to the code change; and performing classification prediction according to the first semantic embedding vector and the second semantic embedding vector to obtain a classification result corresponding to the original patch information. Through the scheme disclosed by the invention, the prediction efficiency, accuracy and adaptability are improved.

Description

Technical Field

[0001] The disclosed embodiments relate to the field of data processing technology, and in particular to a semantic network-based large language model collaborative patch integration evaluation method. Background Art

[0002] Currently, there are two main branches of Linux kernel versions, namely the mainline version and the stable version. The patches merged into the mainline version include two types: bug fixes and new functions and features, while the stable version is mainly used for commercial purposes and focuses on patches to fix bugs. The maintainers of the stable version focus on monitoring and evaluating the mainline Linux kernel patches, selecting patches suitable for merging into the stable version and merging them into the kernel source code tree of the stable version to ensure the reliability and security of the Linux stable version kernel. In actual industrial environments, this process mainly relies on manual judgment to decide whether to merge the patch into the stable version.

[0003] However, the upstream community generates a large number of patches every day, and maintainers must carefully select appropriate patches for merging based on the actual needs of the downstream community. This process consumes a lot of human resources and time. In addition, the selection of patches depends largely on the experience and subjectivity of the maintainer, which may cause some important patches to be overlooked. Therefore, it is of great significance to study how to use automated tools to complete the decision of whether a patch should be merged into the stable version. Existing methods use relatively primitive models to complete feature extraction, which is relatively complex to implement. At the same time, it is limited by the insufficient semantic expression ability of the model, and the feature extraction effect is not good.

[0004] It can be seen that there is an urgent need for a semantic network-based large language model collaborative patch incorporation evaluation method with high prediction efficiency, accuracy and adaptability. Summary of the invention

[0005] In view of this, an embodiment of the present disclosure provides a large language model collaborative patch merging evaluation method based on a semantic network, which at least partially solves the problems of poor prediction efficiency, accuracy and adaptability in the prior art.

[0006] The present disclosure provides a semantic network-based large language model collaborative patch integration evaluation method, including:

[0007] Step 1, obtaining original patch information of the target operating system, wherein the original patch information includes submission information and code changes;

[0008] Step 2: semantically enrich the original patch information based on the large language model, and generate URL parsing and code parsing respectively with the submission information and code changes as input;

[0009] Step 3, dividing the code changes in the patch information into blocks to obtain code blocks;

[0010] Step 4: Combine the submission information, URL parsing and code parsing to obtain the semantically enhanced submission information;

[0011] Step 5: Input the semantically enhanced submission information into the BERT model to extract features, and obtain the first semantic embedding vector corresponding to the submission information;

[0012] Step 6: embed the code block and input it into the CodeBERT model to obtain the second semantic embedding vector corresponding to the code change;

[0013] Step 7: Perform classification prediction based on the first semantic embedding vector and the second semantic embedding vector to obtain a classification result corresponding to the original patch information.

[0014] According to a specific implementation of the embodiment of the present disclosure, step 2 specifically includes:

[0015] Step 2.1, taking the submission information x_msg as input, using the locally deployed large language model to access the URL link of the key information of the patch contained in the submission information and return the model understanding and interpretation to form URL parsing E;

[0016] In step 2.2, the code change x_code is taken as input, and the locally deployed large language model is used to interpret the code information contained in the code change in a document form, and the code analysis C is organized and output.

[0017] According to a specific implementation of the embodiment of the present disclosure, step 3 specifically includes:

[0018] Step 3.1, split the code changes into multiple modified files:

[0019] x_code=[fil e_1, fil e_2,...fil e_o];

[0020] Step 3.2, split each modified file into multiple intermediate blocks:

[0021] file=[hunk_1,hunk_2...hunk_m];

[0022] Step 3.3: treat the continuously deleted or added codes in the middle block as code chunks, and further split them into code chunks:

[0023] hunk=[chunk_1,chunk_2...chunk_n].

[0024] According to a specific implementation of the embodiment of the present disclosure, step 5 specifically includes:

[0025] Step 5.1, the semantically enhanced submission information msg_new = [E, C, x_msg] is used as the key, the corresponding content is used as the value, and a key-value pair is formed as input;

[0026] Step 5.2, perform word embedding operation, add a special classification tag [CLS] at the beginning of the submission information to indicate the aggregate sequence representation for classification tasks, insert special tags [SEP] between sentences to distinguish different sentences, and finally generate the word embedding vector T m ;

[0027] Step 5.3, perform segment embedding operation, assign an identifier to each sentence to distinguish different sentences, use indexes from 0 to n to represent each sentence, and generate a segment embedding vector S m , where n represents the number of sentences in the commit message;

[0028] Step 5.4, position embedding, generate the position embedding vector P according to the position of each word in the submission information m , provides the position information of the word in the input sequence;

[0029] Step 5.5: embed the word into vector T m , segment embedding vector S m and position embedding vector P m Integrate into a composite input vector I m

[0030] I m =T m +S m +P m ;

[0031] Step 5.6, the composite input vector I m Input into the pre-trained BERT model to generate the first semantic embedding vector e m .

[0032] According to a specific implementation of the embodiment of the present disclosure, step 6 specifically includes:

[0033] Step 6.1, take the code chunks [chunk_1, chunk_2...chunk_n] as parallel input, for chunk_i, i∈[1,n], add a special classification tag [CLS] to represent the aggregate representation of the classification task, and insert a special tag [SEP] between code lines to distinguish different code lines;

[0034] Step 6.2: Perform word embedding and segment embedding operations to generate word embedding vector Tc , and the segment embedding vector S c , assigning each line of code an index from 0 to m, where m is the number of lines in the code block, and generating a position embedding vector P based on the position of each word c , and combine the above three embedding vectors to form a composite input vector I c

[0035] I c =T c +S c +P c ;

[0036] Step 6.3, the composite input vector I c Input CodeBERT model for feature extraction and generate embedding vector F i , select the dimension vector corresponding to the special classification tag [CLS] to represent the embedding vector F of each block i , and finally the embedding vectors F1, F2...F of all blocks k Sequential concatenation finally forms the second semantic embedding vector e corresponding to the code change c

[0037] e c =[F1,F2...F k ].

[0038] According to a specific implementation of the embodiment of the present disclosure, step 7 specifically includes:

[0039] Step 7.1: embed the first semantic vector e m and the second semantic embedding vector e c As input, the representation vector e of the patch is obtained by concatenation p

[0040]

[0041] Step 7.2, use a classifier composed of a fully connected layer and a ReLU activation function to perform the classification task, obtain a probability vector h and generate a classification result based on it, where the expression of the probability vector is

[0042] h=α(ω·e p +b)

[0043] where · represents the dot product, ω is the weight matrix associated with the concatenated vector, b is the bias value, and α is the nonlinear activation function.

[0044] The large language model collaborative patch integration evaluation scheme based on semantic network in the disclosed embodiment includes: step 1, obtaining original patch information of the target operating system, wherein the original patch information includes submission information and code changes; step 2, semantically enriching the original patch information based on the large language model, and generating URL parsing and code parsing respectively with submission information and code changes as input; step 3, processing the code changes in the patch information in blocks to obtain code blocks; step 4, splicing the submission information, URL parsing and code parsing to obtain semantically enhanced submission information; step 5, inputting the semantically enhanced submission information into the BERT model to extract features, and obtaining a first semantic embedding vector corresponding to the submission information; step 6, embedding the code block and inputting it into the CodeBERT model to obtain a second semantic embedding vector corresponding to the code change; step 7, performing classification prediction based on the first semantic embedding vector and the second semantic embedding vector to obtain a classification result corresponding to the original patch information.

[0045] The beneficial effects of the disclosed embodiment are as follows: through the disclosed solution, the patch information enhanced by LLM semantics and the original patch information are used as the input of the feature extraction module. The feature extraction module is composed of a Bert branch and a CodeBert branch in parallel, which extract features for submission information and code changes respectively. Finally, the extracted feature vector is used to determine whether the patch should be merged into the stable version of the Linux kernel. This method enhances the ability to evaluate the stability of the patch and improves the efficiency of patch management. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 A flowchart of a semantic network-based large language model collaborative patch merging evaluation method provided by an embodiment of the present disclosure;

[0048] Figure 2 A system architecture diagram of a semantic network-based large language model collaborative patch merging evaluation method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0050] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0051] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0052] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0053] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0054] Existing stable patch identification tools regard the automation of stable patch identification as a text binary classification problem, and determine whether a patch should be merged into the stable kernel version by learning historical patches. These methods usually include two steps: patch preprocessing and model prediction. In preprocessing, the patch file is processed as necessary to facilitate subsequent reading. The model prediction stage is further subdivided into feature extraction and classification prediction. Feature extraction is usually divided into extracting submission information and code change information, and decomposing the task of extracting features from code changes into four modules: submission code module, submission file module, deletion (addition) code module, and submission line module. This hierarchical approach extracts semantics from the bottom up to form an embedding vector representing the meaning of the code change. c , and finally e which indicates the meaning of the commit message m Merge to generate feature vectors and finally complete classification prediction.

[0055] Although these methods perform well in the stable patch recognition task, they still have some shortcomings. First, the existing methods are not implemented in an end-to-end form, and the generation of intermediate variables increases the storage cost. Second, the existing methods use machine learning techniques such as support vector machines (SVM) or simple neural networks composed of convolutional layers and fully connected layers for feature extraction, which are insufficient in semantic representation and poor in extraction performance. In addition, compared with pre-trained models that only require a small amount of fine-tuning training to achieve better results, existing methods usually require a large number of training iterations to optimize the model, and the training cost is high.

[0056] The disclosed embodiment provides a semantic network-based large language model collaborative patch integration evaluation method, which can be applied to the system patch management process in Internet scenarios.

[0057] See also Figure 1 , is a flow chart of a method for evaluating the collaborative patch integration of a large language model based on a semantic network provided by an embodiment of the present disclosure. Figure 1 and Figure 2 As shown, the method mainly comprises the following steps:

[0058] Step 1, obtaining original patch information of the target operating system, wherein the original patch information includes submission information and code changes;

[0059] The core idea of ​​this method is to design a serial architecture based on a large language model that integrates three modules: information preprocessing, feature extraction, and classification prediction. It uses the patch information enhanced by LLM semantics and the original patch information as the input of the feature extraction module. The feature extraction is essentially composed of a Bert branch and a CodeBert in parallel, which extract features from submission information and code changes respectively. Finally, the extracted feature vector is used to determine whether the patch should be merged into the stable version of the Linux kernel.

[0060] During specific implementation, the original patch information of the target operating system may be obtained first, which consists of two parts: submission information x_msg and code change x_code.

[0061] Step 2: semantically enrich the original patch information based on the large language model, and generate URL parsing and code parsing respectively with the submission information and code changes as input;

[0062] In the specific implementation, the original information is semantically enriched based on the large language model, and URL parsing E and code parsing C are generated respectively with x_msg and x_code as input. The specific process is as follows:

[0063] Step 2.1, taking the original submission information x_msg as input, using the locally deployed large language model to access the URL link of the key information of the patch contained in the submission information and return the model understanding and interpretation, and form and output URL parsing E.

[0064] Step,2.2, takes the code change x_code as input, uses the locally deployed large language model to,document the code information contained in the code change, organizes and outputs the,code analysis C.

[0065] Step 3, dividing the code changes in the patch information into blocks to obtain code blocks;

[0066] In specific implementation, the specific process of block processing is as follows:

[0067] Step 3.1: Since code changes have a complex structure, a patch code change involves code modifications in multiple files. Therefore, in order to deeply extract the code representation of the patch, it is first segmented according to the file. The formula is as follows:

[0068] x_code=[fil e_1,fil e_2,...fil e_o]

[0069] Step 3.2: Since each modified file involves multiple different intermediate blocks, further segmentation is performed. The formula is as follows:

[0070] file=[hunk_1,hunk_2...hunk_m]

[0071] Step 3.3: Since each intermediate block contains both deleted and newly added codes, the continuously deleted or added codes are regarded as code blocks and further split. The formula is as follows:

[0072] hunk=[chunk_1,chunk_2...chunk_n].

[0073] Step 4: Combine the submission information, URL parsing and code parsing to obtain the semantically enhanced submission information;

[0074] Step 5: Input the semantically enhanced submission information into the BERT model to extract features, and obtain the first semantic embedding vector corresponding to the submission information;

[0075] In specific implementation, the specific process of obtaining the first semantic embedding vector corresponding to the submitted information is as follows:

[0076] Step 5.1, the preprocessed submission information consists of three parts, namely msg_new = [E, C, x_msg], these parts are regarded as keys, the corresponding content is regarded as values, and a key-value pair is formed as input.

[0077] Step 5.2, word embedding, add a special classification tag [CLS] at the beginning of the submission information to indicate the aggregate sequence representation for classification tasks. Insert special tags [SEP] between sentences to distinguish different sentences, and finally generate the word embedding vector T m .

[0078] Step 5.3, segment embedding, assign an identifier to each sentence to distinguish different sentences, use an index from 0 to n to represent each sentence, and generate a segment embedding vector S m , where n is the number of sentences in the commit message.

[0079] Step 5.4, position embedding, generate the position embedding vector P according to the position of each word in the submission information m , provides the position information of the word in the input sequence.

[0080] Step 5.5: Organically integrate information from different sources to embed the word into the vector T m , segment embedding vector S m and position embedding vector P m Integrate into a composite input vector I m .Right now

[0081] I m =Tm +S m +P m

[0082] Step 5.6, finally I m Input into the pre-trained BERT model to generate the first semantic embedding vector e m .

[0083] Step 6: embed the code block and input it into the CodeBERT model to obtain the second semantic embedding vector corresponding to the code change;

[0084] In the specific implementation, the obtained code blocks [chunk_1, chunk_2...chunk_n] are used as parallel inputs, and after the embedding operation, they are input into CodeBert, and finally the second semantic embedding vector e expressing the semantics of the code change is output. c , the specific process is as follows:

[0085] In step 6.1, take the code chunks [chunk_1, chunk_2...chunk_n] as parallel inputs, for chunk_i, i∈[1,n], add a special classification tag [CLS] to represent the aggregate representation of the classification task, and insert a special tag [SEP] between code lines to distinguish different code lines.

[0086] Step 6.2, the same as the embedding operation in step 5, generates the word embedding vector T c , and the segment embedding vector S c , assigning each line of code an index from 0 to m, where m is the number of lines in the code block. At the same time, a position embedding vector P is generated based on the position of each word c , and combine the above three embedding vectors to form a composite input vector I c ,Right now

[0087] I c =T c +S c +P c ;

[0088] Step 6.3, I c Input CodeBERT model for feature extraction and generate embedding vector F i In order to simplify subsequent operations, the dimension vector corresponding to the special classification tag [CLS] is selected to represent the embedding vector F of each block i , and finally the embedding vectors F1, F2...F of all blocks k Sequential concatenation finally forms the embedding vector e of the code change c ,Right now

[0089] e c=[F1,F2...F k ].

[0090] Step 7: Perform classification prediction based on the first semantic embedding vector and the second semantic embedding vector to obtain a classification result corresponding to the original patch information.

[0091] In specific implementation, the classification prediction process is as follows:

[0092] Step 7.1, embed the submission information into vector e m and the submitted code embedding vector e c As input. The representation vector e of the patch is obtained by concatenation p , the specific calculation formula is as follows:

[0093]

[0094] In step 7.2, a classifier consisting of a fully connected (FC) layer and a ReLU activation function is used to perform the classification task. The probability vector h output by the classifier is calculated as follows:

[0095] h=α(ω·e p +b)

[0096] where · represents the dot product, ω is the weight matrix associated with the concatenated vector, b is the bias value, and α is the nonlinear activation function.

[0097] The probability vector is used as a classification prediction to generate the corresponding output result y, where y = {0, 1}. When y is equal to 1, it is considered that the patch should be merged into the stable version.

[0098] The semantic network-based large language model collaborative patch incorporation evaluation method provided in this embodiment, by combining with the large language model to assist in feature extraction, obtains URL information understanding and code-related interpretation through large language model analysis, and completes the semantic enrichment of the original patch information, which enables our method to capture complex contextual relationships. At the same time, enriching the semantic representation can improve the quality of feature extraction, and more importantly, it can ensure that our method can complete the task of processing heterogeneous and multimodal data; the end-to-end architecture design adopted, the generation results of the middle layer are directly submitted as input to the subsequent steps for processing, reducing the processing delay of the middle layer and improving the overall performance and response time. The end-to-end architecture is also convenient for iteration and reduces storage costs; advanced pre-trained models are used (BERT is used for submission information and CodeBERT is used for code changes). These models are trained on large corpora and have richer semantic understanding capabilities, so that they can generate high-quality patch representation vectors and significantly improve recognition performance. Moreover, unlike the method that regards code changes as simple word sequences, our method can recognize the grammatical complexity in code changes. By using specialized pre-trained models for different parts of a patch (commit information and code changes), it is able to more effectively capture the subtle differences in each data type, thereby improving prediction efficiency, accuracy, and usability.

[0099] It should be understood that various parts of the present disclosure may be implemented in hardware, software, firmware or a combination thereof.

[0100] The above is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A semantic network-based large language model collaborative patch integration evaluation method, characterized in that: include: Step 1, obtaining original patch information of the target operating system, wherein the original patch information includes submission information and code changes; Step 2: semantically enrich the original patch information based on the large language model, and generate URL parsing and code parsing respectively with the submission information and code changes as input; Step 3, dividing the code changes in the patch information into blocks to obtain code blocks; Step 4: Combine the submission information, URL parsing and code parsing to obtain the semantically enhanced submission information; Step 5: Input the semantically enhanced submission information into the BERT model to extract features, and obtain the first semantic embedding vector corresponding to the submission information; Step 6: embed the code block and input it into the CodeBERT model to obtain the second semantic embedding vector corresponding to the code change; Step 7: Perform classification prediction based on the first semantic embedding vector and the second semantic embedding vector to obtain a classification result corresponding to the original patch information.

2. The method according to claim 1, characterized in that , the step 2 specifically includes: Step 2.1, taking the submission information x_msg as input, using the locally deployed large language model to access the URL link of the key information of the patch contained in the submission information and return the model understanding and interpretation to form URL parsing E; In step 2.2, the code change x_code is taken as input, and the locally deployed large language model is used to interpret the code information contained in the code change in a document form, and the code analysis C is organized and output.

3. The method according to claim 2, characterized in that , the step 3 specifically includes: Step 3.1, split the code changes into multiple modified files: x_code=[file_1,file_2,...file_o]; Step 3.2, split each modified file into multiple intermediate blocks: file=[hunk_1,hunk_2...hunk_m]; Step 3.3: treat the continuously deleted or added codes in the middle block as code chunks, and further split them into code chunks: hunk=[chunk_1,chunk_2...chunk_n].

4. The method according to claim 3, characterized in that , the step 5 specifically includes: Step 5.1, the semantically enhanced submission information msg_new = [E, C, x_msg] is used as the key, the corresponding content is used as the value, and a key-value pair is formed as input; Step 5.2, perform word embedding operation, add a special classification tag [CLS] at the beginning of the submission information to indicate the aggregate sequence representation for classification tasks, insert special tags [SEP] between sentences to distinguish different sentences, and finally generate the word embedding vector T m ; Step 5.3, perform segment embedding operation, assign an identifier to each sentence to distinguish different sentences, use indexes from 0 to n to represent each sentence, and generate a segment embedding vector S m , where n represents the number of sentences in the commit message; Step 5.4, position embedding, generate the position embedding vector P according to the position of each word in the submission information m , provides the position information of the word in the input sequence; Step 5.5: embed the word into vector T m , segment embedding vector S m and position embedding vector P m Integrate into a composite input vector I m I m =T m +S m +P m ; Step 5.6, composite input vector I m Input into the pre-trained BERT model to generate the first semantic embedding vector e m .

5. The method according to claim 4, characterized in that , the step 6 specifically includes: Step 6.1, take the code chunks [chunk_1, chunk_2...chunk_n] as parallel input, for chunk_i, i∈[1,n], add a special classification tag [CLS] to represent the aggregate representation of the classification task, and insert a special tag [SEP] between code lines to distinguish different code lines; Step 6.2: Perform word embedding and segment embedding operations to generate word embedding vector T c , and the segment embedding vector S c , assigning each line of code an index from 0 to m, where m is the number of lines in the code block, and generating a position embedding vector P based on the position of each word c , and combine the above three embedding vectors to form a composite input vector I c I c =T c +S c +P c Step 6.3, the composite input vector I c Input CodeBERT model for feature extraction and generate embedding vector F i , select the dimension vector corresponding to the special classification tag [CLS] to represent the embedding vector F of each block i , and finally the embedding vectors F1, F2...F of all blocks k Sequential concatenation finally forms the second semantic embedding vector e corresponding to the code change c e c =[F1,F2...F k ]。 6. The method according to claim 5, characterized in that , the step 7 specifically includes: Step 7.1: embed the first semantic vector e m and the second semantic embedding vector e c As input, the representation vector e of the patch is obtained by concatenation p Step 7.2, use a classifier composed of a fully connected layer and a ReLU activation function to perform the classification task, obtain a probability vector h and generate a classification result based on it, where the expression of the probability vector is h=α(ω·e p +b) where · represents the dot product, ω is the weight matrix associated with the concatenated vector, b is the bias value, and α is the nonlinear activation function.

Images