A semantic network-based collaborative patch integration evaluation method for large language models

Abstract

The disclosed embodiment provides a large language model collaborative patch integration evaluation method based on a semantic network, which belongs to the field of data processing technology and specifically includes: obtaining the original patch information of the target operating system; semantically enriching the original patch information based on the large language model, and generating URL parsing and code parsing respectively with the submission information and code changes as input; processing the code changes in the patch information in blocks; splicing the submission information, URL parsing and code parsing; inputting the semantically enhanced submission information into the BERT model to extract features, and obtaining the first semantic embedding vector corresponding to the submission information; embedding the code block and inputting it into the CodeBERT model to obtain the second semantic embedding vector corresponding to the code change; performing classification prediction based on the first semantic embedding vector and the second semantic embedding vector to obtain the classification result corresponding to the original patch information. Through the scheme disclosed in the present invention, the prediction efficiency, accuracy and adaptability are improved.

Description

Technical Field

[0001] The disclosed embodiments relate to the field of data processing technology, and in particular to a semantic network-based large language model collaborative patch integration evaluation method. Background Art

[0002] Currently, the Linux kernel consists of two main branches: the mainline and the stable. Patches merged into the mainline include both bug fixes and new features. The stable version, on the other hand, is primarily for commercial use and focuses on bug fixes. Maintainers of the stable version monitor and evaluate mainline Linux kernel patches, selecting those suitable for merging into the stable version's kernel source tree to ensure the reliability and security of the stable version of the Linux kernel. In real industrial environments, this process relies heavily on manual judgment to determine whether to merge a patch into the stable version.

[0003] However, the upstream community generates a large number of patches every day, and maintainers must carefully select appropriate patches for merging based on the actual needs of the downstream community. This process consumes a lot of human resources and time. In addition, the selection of patches relies heavily on the maintainer's experience and subjectivity, which may cause some important patches to be overlooked. Therefore, it is of great significance to study how to use automated tools to determine whether patches should be merged into the stable version. Existing methods use relatively primitive models to complete feature extraction, which is relatively complex to implement. At the same time, due to the insufficient semantic expression ability of the model, the feature extraction effect is poor.

[0004] It can be seen that there is an urgent need for a semantic network-based large language model collaborative patch incorporation evaluation method with high prediction efficiency, accuracy and adaptability. Summary of the Invention

[0005] In view of this, the embodiments of the present disclosure provide a semantic network-based large language model collaborative patch integration evaluation method, which at least partially solves the problems of poor prediction efficiency, accuracy and adaptability in the existing technology.

[0006] The present disclosure provides a semantic network-based large language model collaborative patch integration evaluation method, including:

[0007] Step 1: Obtain original patch information of the target operating system, wherein the original patch information includes submission information and code changes;

[0008] Step 2: semantically enrich the original patch information based on the large language model, and generate URL parsing and code parsing respectively using the commit information and code changes as input;

[0009] Step 3: Divide the code changes in the patch information into blocks to obtain code blocks;

[0010] Step 4: Combine the submission information, URL parsing, and code parsing to obtain semantically enhanced submission information.

[0011] Step 5: Input the semantically enhanced submission information into the BERT model to extract features and obtain the first semantic embedding vector corresponding to the submission information;

[0012] Step 6: embed the code block and input it into the CodeBERT model to obtain the second semantic embedding vector corresponding to the code change;

[0013] Step 7: Perform classification prediction based on the first semantic embedding vector and the second semantic embedding vector to obtain a classification result corresponding to the original patch information.

[0014] According to a specific implementation of the embodiment of the present disclosure, step 2 specifically includes:

[0015] Step 2.1: Take the commit message x_msg as input and use the locally deployed large language model to access the URL link of the key patch information contained in the commit message and return the model understanding and interpretation to form the URL parsing E;

[0016] In step 2.2, the code change x_code is used as input, and the locally deployed large language model is used to document the code information contained in the code change, organize it, and output the code analysis C.

[0017] According to a specific implementation of the embodiment of the present disclosure, step 3 specifically includes:

[0018] Step 3.1: Split the code changes into multiple modified files:

[0019] x_code=[fil e_1, fil e_2,...fil e_o];

[0020] Step 3.2: Split each modified file into multiple intermediate blocks:

[0021] file=[hunk_1,hunk_2...hunk_m];

[0022] In step 3.3, the continuously deleted or added codes in the middle block are regarded as code chunks, and further split into code chunks:

[0023] hunk=[chunk_1,chunk_2...chunk_n].

[0024] According to a specific implementation of the embodiment of the present disclosure, step 5 specifically includes:

[0025] Step 5.1: The semantically enhanced submission message msg_new = [E, C, x_msg] is used as the key, and the corresponding content is used as the value to form a key-value pair as input;

[0026] Step 5.2: Perform word embedding operations. Add a special classification tag [CLS] at the beginning of the submission information to indicate the aggregate sequence representation for the classification task. Insert special tags [SEP] between sentences to distinguish different sentences. Finally, generate the word embedding vector T. m ;

[0027] Step 5.3, perform segment embedding operation, assign an identifier to each sentence to distinguish different sentences, use the index from 0 to n to represent each sentence, and generate a segment embedding vector S m , where n represents the number of sentences in the commit message;

[0028] Step 5.4, position embedding, generates a position embedding vector P based on the position of each word in the submission information m , provides the position information of the word in the input sequence;

[0029] Step 5.5, embed the word into vector T m , segment embedding vector S m and position embedding vector P m Integrate into a composite input vector I m

[0030] I m =T m +S m +P m ;

[0031] Step 5.6, the composite input vector I m Input into the pre-trained BERT model to generate the first semantic embedding vector e m .

[0032] According to a specific implementation of the embodiment of the present disclosure, step 6 specifically includes:

[0033] Step 6.1: Take the code chunks [chunk_1, chunk_2...chunk_n] as parallel inputs. For chunk_i, i∈[1,n], add a special classification tag [CLS] to represent the aggregated representation of the classification task. Insert a special tag [SEP] between code lines to distinguish different code lines.

[0034] Step 6.2: Perform word embedding and segment embedding operations to generate word embedding vector Tc , and segment embedding vector S c , assign an index from 0 to m to each line of code, where m is the number of lines in the code block, and generate a position embedding vector P based on the position of each word c , and combine the above three embedding vectors to form a composite input vector I c

[0035] I c =T c +S c +P c ;

[0036] Step 6.3, the composite input vector I c Input CodeBERT model for feature extraction and generate embedding vector F i , select the dimension vector corresponding to the special classification mark [CLS] to represent the embedding vector F of each block i , and finally the embedding vectors F1, F2...F of all blocks k Sequential splicing finally forms the second semantic embedding vector e corresponding to the code change c

[0037] e c =[F1,F2...F k ].

[0038] According to a specific implementation of the embodiment of the present disclosure, step 7 specifically includes:

[0039] Step 7.1, embed the first semantic vector e m and the second semantic embedding vector e c As input, the representation vector e of the patch is obtained by concatenation p

[0040]

[0041] Step 7.2: Use a classifier composed of a fully connected layer and a ReLU activation function to perform the classification task, obtain the probability vector h and generate the classification result based on it, where the expression of the probability vector is

[0042] h=α(ω·e p +b)

[0043] where · represents the dot product, ω is the weight matrix associated with the concatenated vector, b is the bias value, and α is the nonlinear activation function.

[0044] The large language model collaborative patch integration evaluation scheme based on a semantic network in the embodiment of the present disclosure includes: step 1, obtaining original patch information of a target operating system, wherein the original patch information includes submission information and code changes; step 2, semantically enriching the original patch information based on the large language model, and generating URL parsing and code parsing respectively with the submission information and code changes as input; step 3, block processing the code changes in the patch information to obtain code blocks; step 4, splicing the submission information, URL parsing and code parsing to obtain semantically enhanced submission information; step 5, inputting the semantically enhanced submission information into a BERT model to extract features and obtain a first semantic embedding vector corresponding to the submission information; step 6, embedding the code block and inputting it into a CodeBERT model to obtain a second semantic embedding vector corresponding to the code change; step 7, performing classification prediction based on the first semantic embedding vector and the second semantic embedding vector to obtain a classification result corresponding to the original patch information.

[0045] The beneficial effects of the disclosed embodiments are as follows: Through the disclosed solution, patch information enhanced with LLM semantics is used alongside the original patch information as input to the feature extraction module. The feature extraction module consists of a Bert branch and a CodeBert branch, which perform feature extraction on commit information and code changes, respectively. Ultimately, the extracted feature vectors are used to determine whether the patch should be merged into the stable version of the Linux kernel. This approach enhances the ability to assess patch stability and improves the efficiency of patch management. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0047] Figure 1 A flowchart of a semantic network-based large language model collaborative patch integration evaluation method provided by an embodiment of the present disclosure;

[0048] Figure 2 A system architecture diagram of a semantic network-based large language model collaborative patch integration evaluation method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0050] The following describes the embodiments of the present disclosure through specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0051] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement an apparatus and / or practice a method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this apparatus and / or practice this method.

[0052] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0053] Additionally, in the following description, specific details are provided to provide a thorough understanding of the examples. However, one skilled in the art will appreciate that the aspects described can be practiced without these specific details.

[0054] Existing stable patch identification tools regard the automation of stable patch identification as a text binary classification problem, and determine whether a patch should be merged into the stable kernel version by learning historical patches. These methods usually include two steps: patch preprocessing and model prediction. In preprocessing, the patch file is processed as necessary to facilitate subsequent reading. The model prediction stage is further subdivided into feature extraction and classification prediction. Feature extraction is usually divided into extracting submission information and code change information, and decomposing the task of extracting features from code changes into four modules: submission code module, submission file module, deletion (addition) code module and submission line module. This hierarchical approach extracts semantics from the bottom up to form an embedding vector e that represents the meaning of the code change. c , and finally e which means the commit message m Merge to generate feature vectors and finally complete classification prediction.

[0055] Although these methods perform well in the stable patch recognition task, they still have some shortcomings. First, because the existing methods' implementation architecture is not end-to-end, the generation of intermediate variables increases storage costs. Second, existing methods use machine learning techniques such as support vector machines (SVMs) or simple neural networks consisting of convolutional layers and fully connected layers for feature extraction, which lacks semantic representation and poor extraction performance. In addition, compared to pre-trained models that only require a small amount of fine-tuning training to achieve better results, existing methods typically require a large number of training iterations to optimize the model, resulting in high training costs.

[0056] The embodiments of the present disclosure provide a semantic network-based large language model collaborative patch integration evaluation method, which can be applied to system patch management processes in Internet scenarios.

[0057] See also Figure 1 , is a flow chart of a method for evaluating collaborative patch integration of a large language model based on a semantic network according to an embodiment of the present disclosure. Figure 1 and Figure 2 As shown, the method mainly includes the following steps:

[0058] Step 1: Obtain original patch information of the target operating system, wherein the original patch information includes submission information and code changes;

[0059] The core idea of ​​this method lies in designing a serial architecture based on a large language model, integrating information preprocessing, feature extraction, and classification prediction. The feature extraction module uses both LLM-enhanced patch information and the original patch information as input. The feature extraction module essentially consists of a Bert branch and a CodeBert branch, which extract features on commit information and code changes, respectively. Finally, the extracted feature vector is used to determine whether the patch should be merged into the stable Linux kernel version.

[0060] During specific implementation, the original patch information of the target operating system may be obtained first, which consists of two parts: submission information x_msg and code change x_code.

[0061] Step 2: semantically enrich the original patch information based on the large language model, and generate URL parsing and code parsing respectively using the commit information and code changes as input;

[0062] In specific implementation, the original information is semantically enriched based on the large language model, and URL parsing E and code parsing C are generated with x_msg and x_code as input respectively. The specific process is as follows:

[0063] In step 2.1, the original submission message x_msg is used as input, and the locally deployed large language model is used to access the URL link of the key information of the patch contained in the submission message and return the model understanding and interpretation, and form and output the URL resolution E.

[0064] Step,2.2, takes the code change x_code as input, uses the locally deployed large language model to,form a document-style explanation of the code information contained in the,code change, organizes it, and outputs the code analysis C.

[0065] Step 3: Divide the code changes in the patch information into blocks to obtain code blocks;

[0066] In specific implementation, the specific process of block processing is as follows:

[0067] Step 3.1: Since code changes have a complex structure, a patch's code change involves code modifications in multiple files. Therefore, in order to deeply extract the patch's code representation, it is first segmented by file. The formula is as follows:

[0068] x_code=[fil e_1,fil e_2,...fil e_o]

[0069] In step 3.2, since each modified file involves multiple different intermediate blocks, further segmentation is performed. The formula is as follows:

[0070] file=[hunk_1,hunk_2...hunk_m]

[0071] In step 3.3, since each intermediate block contains both deleted and added code, the continuously deleted or added code is considered as a code chunk and further split. The formula is as follows:

[0072] hunk=[chunk_1,chunk_2...chunk_n].

[0073] Step 4: Combine the submission information, URL parsing, and code parsing to obtain semantically enhanced submission information.

[0074] Step 5: Input the semantically enhanced submission information into the BERT model to extract features and obtain the first semantic embedding vector corresponding to the submission information;

[0075] In specific implementation, the specific process of obtaining the first semantic embedding vector corresponding to the submitted information is as follows:

[0076] In step 5.1, the pre-processed submission message consists of three parts, namely, msg_new = [E, C, x_msg]. These parts are regarded as keys, and the corresponding content is regarded as values, and key-value pairs are formed as input.

[0077] Step 5.2, word embedding, add a special classification tag [CLS] at the beginning of the submission information to indicate the aggregate sequence representation for classification tasks. Insert special tags [SEP] between sentences to distinguish different sentences, and finally generate the word embedding vector T m .

[0078] Step 5.3, segment embedding, assign an identifier to each sentence to distinguish different sentences, use an index from 0 to n to represent each sentence, and generate a segment embedding vector S m , where n is the number of sentences in the commit message.

[0079] Step 5.4, position embedding, generates a position embedding vector P based on the position of each word in the submission information m , provides the position information of the word in the input sequence.

[0080] Step 5.5: Organically integrate information from different sources to embed the word into the vector T m , segment embedding vector S m and position embedding vector P m Integrate into a composite input vector I m .Right now

[0081] I m =Tm +S m +P m

[0082] Step 5.6, finally I m Input into the pre-trained BERT model to generate the first semantic embedding vector e m .

[0083] Step 6: embed the code block and input it into the CodeBERT model to obtain the second semantic embedding vector corresponding to the code change;

[0084] In the specific implementation, the obtained code blocks [chunk_1, chunk_2...chunk_n] are used as parallel inputs. After the embedding operation, they are input into CodeBert, and the final output is the second semantic embedding vector e that expresses the semantics of the code change. c The specific process is as follows:

[0085] In step 6.1, take the code chunks [chunk_1, chunk_2...chunk_n] as parallel inputs. For chunk_i, i∈[1,n], add a special classification tag [CLS] to represent the aggregated representation of the classification task. Insert a special tag [SEP] between code lines to distinguish different code lines.

[0086] Step 6.2, the same as the embedding operation in step 5, generates the word embedding vector T c , and segment embedding vector S c , assign an index from 0 to m to each line of code, where m is the number of lines in the code block. At the same time, a position embedding vector P is generated based on the position of each word c , and combine the above three embedding vectors to form a composite input vector I c ,Right now

[0087] I c =T c +S c +P c ;

[0088] Step 6.3, I c Input CodeBERT model for feature extraction and generate embedding vector F i In order to simplify subsequent operations, the dimension vector corresponding to the special classification mark [CLS] is selected to represent the embedding vector F of each block i , and finally the embedding vectors F1, F2...F of all blocks k Sequential concatenation finally forms the embedding vector e of the code change c ,Right now

[0089] e c=[F1,F2...F k ].

[0090] Step 7: Perform classification prediction based on the first semantic embedding vector and the second semantic embedding vector to obtain a classification result corresponding to the original patch information.

[0091] In specific implementation, the classification prediction process is as follows:

[0092] Step 7.1, embed the submission information into vector e m and the submission code embedding vector e c As input. The patch representation vector e is obtained by concatenation p , and its specific calculation formula is as follows:

[0093]

[0094] In step 7.2, a classifier consisting of a fully connected (FC) layer and a ReLU activation function is used to perform the classification task. The probability vector h output by the classifier is calculated as follows:

[0095] h=α(ω·e p +b)

[0096] where · represents the dot product, ω is the weight matrix associated with the concatenated vector, b is the bias value, and α is the nonlinear activation function.

[0097] The probability vector is used as a classification prediction to generate the corresponding output y, where y = {0, 1}. When y is equal to 1, it is considered that the patch should be merged into the stable version.

[0098] The semantic network-based large language model collaborative patch incorporation evaluation method provided in this embodiment utilizes a large language model to assist in feature extraction. Through analysis using the large language model, URL information understanding and code-related interpretations are obtained, enabling semantic enrichment of the original patch information. This allows our method to capture complex contextual relationships. This enriched semantic representation improves feature extraction quality and, more importantly, ensures that our method can handle heterogeneous and multimodal data. The end-to-end architecture design allows intermediate layer generated results to be directly submitted as input to subsequent steps, reducing intermediate layer processing latency and improving overall performance and response time. This end-to-end architecture also facilitates iteration and reduces storage costs. Advanced pre-trained models (BERT for submission information and CodeBERT for code changes) are employed. These models, trained on large corpora, possess richer semantic understanding capabilities, enabling the generation of high-quality patch representation vectors and significantly improving recognition performance. Furthermore, unlike methods that treat code changes as simple word sequences, our method recognizes the grammatical complexity inherent in code changes. By using specialized pre-trained models for different parts of a patch (commit information and code changes), it is able to more effectively capture the nuances of each data type, thereby improving prediction efficiency, accuracy, and usability.

[0099] It should be understood that various parts of the present disclosure can be implemented in hardware, software, firmware, or a combination thereof.

[0100] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A semantic network-based large language model collaborative patch integration evaluation method, characterized by: include: Step 1: Obtain original patch information of the target operating system, wherein the original patch information includes submission information and code changes; Step 2: semantically enrich the original patch information based on the large language model, and generate URL parsing and code parsing respectively using the commit information and code changes as input; Step 3: Divide the code changes in the patch information into blocks to obtain code blocks; Step 4: Combine the submission information, URL parsing, and code parsing to obtain semantically enhanced submission information. Step 5: Input the semantically enhanced submission information into the BERT model to extract features and obtain the first semantic embedding vector corresponding to the submission information; Step 6: embed the code block and input it into the CodeBERT model to obtain the second semantic embedding vector corresponding to the code change; Step 7: Perform classification prediction based on the first semantic embedding vector and the second semantic embedding vector to obtain a classification result corresponding to the original patch information.

2. The method according to claim 1, characterized in that , the step 2 specifically includes: Step 2.1: Take the commit message x_msg as input and use the locally deployed large language model to access the URL link of the key patch information contained in the commit message and return the model understanding and interpretation to form the URL parsing E; In step 2.2, the code change x_code is used as input, and the locally deployed large language model is used to document the code information contained in the code change, organize it, and output the code analysis C.

3. The method according to claim 2, characterized in that , the step 3 specifically includes: Step 3.1: Split the code changes into multiple modified files: x_code=[file_1,file_2,...file_o]; Step 3.2: Split each modified file into multiple intermediate blocks: file=[hunk_1,hunk_2...hunk_m]; In step 3.3, the continuously deleted or added codes in the middle block are regarded as code chunks, and further split into code chunks: hunk=[chunk_1,chunk_2...chunk_n].

4. The method according to claim 3, characterized in that , the step 5 specifically includes: Step 5.1: The semantically enhanced submission message msg_new = [E, C, x_msg] is used as the key, and the corresponding content is used as the value to form a key-value pair as input; Step 5.2: Perform word embedding operations. Add a special classification tag [CLS] at the beginning of the submission information to indicate the aggregate sequence representation for the classification task. Insert special tags [SEP] between sentences to distinguish different sentences. Finally, generate the word embedding vector T. m ; Step 5.3, perform segment embedding operation, assign an identifier to each sentence to distinguish different sentences, use the index from 0 to n to represent each sentence, and generate a segment embedding vector S m , where n represents the number of sentences in the commit message; Step 5.4, position embedding, generates a position embedding vector P based on the position of each word in the submission information m , provides the position information of the word in the input sequence; Step 5.5, embed the word into vector T m , segment embedding vector S m and position embedding vector P m Integrate into a composite input vector I m I m =T m +S m +P m ; Step 5.6, the composite input vector I m Input into the pre-trained BERT model to generate the first semantic embedding vector e m .

5. The method according to claim 4, characterized in that , the step 6 specifically includes: Step 6.1: Take the code chunks [chunk_1, chunk_2...chunk_n] as parallel inputs. For chunk_i, i∈[1,n], add a special classification tag [CLS] to represent the aggregated representation of the classification task. Insert a special tag [SEP] between code lines to distinguish different code lines. Step 6.2: Perform word embedding and segment embedding operations to generate word embedding vector T c , and segment embedding vector S c , assign an index from 0 to m to each line of code, where m is the number of lines in the code block, and generate a position embedding vector P based on the position of each word c , and combine the above three embedding vectors to form a composite input vector I c I c =T c +S c +P c Step 6.3, the composite input vector I c Input CodeBERT model for feature extraction and generate embedding vector F i , select the dimension vector corresponding to the special classification mark [CLS] to represent the embedding vector F of each block i , and finally the embedding vectors F1, F2...F of all blocks k Sequential splicing finally forms the second semantic embedding vector e corresponding to the code change c e c =[F1,F2...F k ]。 6. The method according to claim 5, characterized in that , the step 7 specifically includes: Step 7.1, embed the first semantic vector e m and the second semantic embedding vector e c As input, the representation vector e of the patch is obtained by concatenation p Step 7.2: Use a classifier composed of a fully connected layer and a ReLU activation function to perform the classification task, obtain the probability vector h and generate the classification result based on it, where the expression of the probability vector is h=α(ω·e p +b) where · represents the dot product, ω is the weight matrix associated with the concatenated vector, b is the bias value, and α is the nonlinear activation function.

Images