Large model programming review system based on combination of knowledge graph and RAG

Abstract

The invention discloses a large model programming review system based on combination of a knowledge graph and an RAG, and particularly relates to the field of data collection, comprising the following steps: step 1, dynamic data collection; 2, constructing a knowledge graph; step 3, combining RAG retrieval to enhance generation; step 4, introducing a feedback mechanism; and 5, code comment generation. According to the invention, through dynamic data collection and processing, accurate extraction of key information in a programming document is realized, the information is dynamically fused into the knowledge graph, and the system can carry out high-precision information extraction and semantic association analysis on programming knowledge by combining an adaptive deep learning model and a graph attention network, so that the programming knowledge is extracted more accurately. According to the method, a queried programming knowledge graph is constructed, an RAG technology is combined, and a dynamic-dynamic weight adjustment and incremental learning mechanism is introduced, so that related information can be efficiently retrieved from the knowledge graph, and detailed code comments are generated by utilizing a generation model.

Description

Technical Field

[0001] The present invention relates to the field of data collection technology, and specifically to a large-model programming review system based on the combination of knowledge graph and RAG. Background Art

[0002] In the modern programming and software development environment, knowledge updating and information acquisition are crucial. With the continuous development of programming languages, frameworks, and tools, developers need a system that can update and integrate the latest programming knowledge in real time to conduct code review and decision support more efficiently. Traditional static data collection and processing methods can no longer meet this demand. Therefore, it is of great academic and practical significance to study how to build a dynamic data collection and processing system based on knowledge graphs and RAG (Retrieval-Augmented Generation) technology. Summary of the invention

[0003] The purpose of the present invention is to provide a large model programming review system based on the combination of knowledge graph and RAG to solve the problems raised in the above background technology.

[0004] In order to solve the above technical problems, the present invention provides the following technical solution: a large model programming review system based on the combination of knowledge graph and RAG, comprising the following steps:

[0005] Step 1: Dynamic data collection, access and parse data sources through APIs and crawler technology, and then use Apache Kafka to build a data processing pipeline. Data is transmitted from the source to the processing system through Kafka Topics;

[0006] Step 2: Knowledge graph construction: Use natural language processing technology to accurately extract key information from programming documents and dynamically integrate this information into the knowledge graph to support more efficient information retrieval and decision support.

[0007] Step 3: Combine RAG retrieval and enhanced generation. After obtaining a queryable programming knowledge graph, the model's ability to handle code review requests is enhanced by combining RAG technology, achieving more accurate code review and suggestion generation.

[0008] Step 4: Introduce a feedback mechanism and establish a user feedback interface to allow developers to submit feedback and point out problems in model predictions;

[0009] Step 5: Code review generation, combining RAG technology and introducing dynamic weight adjustment and incremental learning mechanisms to retrieve relevant information from the knowledge graph, and using the generative model to generate detailed code reviews.

[0010] In a preferred embodiment, in the step 1, for API data, a RESTful API interface is used to crawl data, and for web page data that needs to be crawled, DOM parsing and XPath selector are implemented to extract required data, and monitoring nodes are set to regularly check the update status of data sources, and Webhooks or polling mechanisms are used to ensure timely acquisition of the latest data. In the step 1, preliminary data cleaning is implemented when Apache Kafka constructs data, including filtering invalid data, standardizing formats, and correcting data errors to provide high-quality data input.

[0011] In a preferred embodiment, after the preliminary data cleaning, the data stream is smoothed using an exponentially weighted moving average model. During the smoothing process, the smoothing coefficient is dynamically adjusted according to the actual operating effect, and multi-level data processing is introduced, first performing fast and rough processing, and then performing in-depth analysis and storage to optimize processing speed and accuracy.

[0012] In a preferred embodiment, the knowledge graph construction in step 2 includes the following steps:

[0013] S1: Initialization phase, in which a pre-trained model is selected based on the task characteristics, and then pre-adjusted using a small-scale programming language dataset to initially fine-tune the model parameters;

[0014] S2: In the online adjustment phase, the actual performance of the model is evaluated by monitoring the performance indicators of the model output, and the hyperparameters of the model are dynamically adjusted according to the performance indicators. During the operation of the model, real-time fine-tuning is performed through the continuous input and feedback of incremental data to ensure that the model can adapt to the dynamic changes of the data.

[0015] In a preferred embodiment, the pre-trained models in S1 include BERT, RoBERTa and GPT-3, and the preliminary fine-tuning of the model parameters in S1 includes data cleaning, data labeling and pre-training. The data cleaning refers to removing noise data to ensure data quality; the data labeling refers to manually labeling a part of the data set to ensure that the model can identify key entities and relationships; the pre-training refers to pre-training the model using the cleaned data set and adjusting the initial configuration of the model.

[0016] In a preferred embodiment, the step three includes the following steps when combined with RAG search enhancement generation:

[0017] Step 1: Code snippet analysis: parse the received code snippet to identify the key elements, convert the code into a structured tree representation using abstract syntax tree technology, and apply static analysis tools to preliminarily detect potential vulnerabilities and problem areas in the code.

[0018] Step 2: Knowledge graph query, extract key features from the parsed code snippet, build query statements, query the programming knowledge graph based on the extracted features, use vector retrieval technology and graph database to quickly match the most relevant knowledge points, and the vector retrieval generation formula is:

[0019] Q=f query (code)

[0020] Where Q represents the query vector, f query represents the feature extraction function, code is the input code snippet,

[0021] The similarity calculation formula is:

[0022]

[0023] Among them, K i represents the vector representation of the i-th knowledge point in the knowledge graph, and sim represents the cosine similarity calculation;

[0024] Step 3: In the comment generation phase, the generative model is used to generate detailed comments and suggestions on the code snippets in combination with the retrieved knowledge points. The retrieved knowledge points are screened and filtered to remove redundant information to ensure that the knowledge points input to the generative model are concise and efficient. The knowledge points are tested for repetitiveness and duplicates are removed. The knowledge points are scored based on the relevance and importance of the information. High-scoring knowledge points are selected to input into the generative model. The screened knowledge points are integrated with the code snippets to build a complete context input to the generative model. The formula for context construction is:

[0025]

[0026] Step 4: Generate model call, input the constructed context into the generative model, ensure that the input format meets the requirements of the generative model, perform necessary preprocessing, and the generative model generates detailed comments based on the context. The model input formula is:

[0027] Input GPT =Input context

[0028] Generate output formula:

[0029] Comment = Comment = f GPT (Input GPT )

[0030] Among them, f GPT Represents the output function of the generated model, Input GPT Represents the model input;

[0031] Step 5: Dynamic weight adjustment. Calculate the relevance of each retrieved knowledge point and code snippet. Use cosine similarity or dot product to calculate the relevance score. Evaluate the importance of the knowledge point based on its usage frequency in historical data and user feedback. Construct a weight matrix to represent the relative importance of each knowledge point. Dynamically adjust the attention weight of the generative model based on the relevance and importance scores. Introduce a multi-layer attention mechanism in the generative model. The attention weights of different levels are allocated according to the dynamically adjusted weight matrix. The weight calculation formula is:

[0032] w i =α·sim(Q,K i )+β·importance(K i )

[0033] Among them, w i Represents knowledge point K i weight, α and β are weight coefficients, sim(Q, K i ) represents the relevance score, importance(K i ) represents the importance score;

[0034] The attention weight allocation formula is:

[0035]

[0036] Among them, Attention layer (H) represents the attention output of the layer, H represents the hidden state of the input, and Ki represents the vector representation of the knowledge point;

[0037] Step 6: Incremental learning and knowledge updating. Continuously collect new code snippets and user feedback data, build incremental data sets, clean and annotate incremental data, ensure data quality, and update the knowledge graph based on new entities and new relationships in the incremental data. Use the incremental update function of the graph database to ensure the dynamic update of the knowledge graph. Dynamically adjust the weights of knowledge points based on new knowledge points in the incremental data. Ensure that new high-importance knowledge points can be fully utilized in the retrieval and generation stages.

[0038] In a preferred embodiment, the incremental update formula in step 6 is:

[0039] Knowledge new =Knowledge old +ΔKnowledge

[0040] Among them, Knowledge new Represents the updated knowledge graph, Knowledge oldrepresents the original knowledge graph, and ΔKnowledge represents incremental knowledge;

[0041] The weight adjustment formula is:

[0042]

[0043] in, represents the updated weight, represents the original weight, γ is the adjustment coefficient, Δw i Indicates the incremental weight adjustment value.

[0044] In a preferred embodiment, the specific steps of introducing the feedback mechanism in step 4 are:

[0045] Feedback data processing: organize and annotate user feedback data to form a new training data set;

[0046] Model update: Use new feedback data sets to retrain and fine-tune the model to optimize the model prediction effect;

[0047] Multi-task learning: set parallel tasks to perform entity recognition, intent recognition, and semantic relationship extraction simultaneously, share the underlying representation of the model, and implement task-specific outputs at the top level.

[0048] In a preferred embodiment, the model parameter adjustment formula is:

[0049]

[0050] Among them, α represents the model parameters, η is the learning rate dynamically adjusted according to the model performance, is the loss function, D batch It is the batch data extracted from the data stream.

[0051] In a preferred embodiment, the specific steps of entity recognition, intent recognition and semantic relationship extraction are:

[0052] Entity extraction: Extract entities from text using deep learning models;

[0053] Entity encoding: Encode the extracted entities into vector form, using a pre-trained word embedding model or BERT to generate context-related vector representations;

[0054] Graph attention mechanism update: Use the graph attention network to update the entity vector and dynamically adjust its representation based on the interaction between entities;

[0055] Relationship recognition: Apply graph attention network to identify semantic relationships between entities. By constructing an entity relationship graph, nodes represent entities, edges represent relationships between entities, and edge weights represent the strength of relationships.

[0056] Relationship update: monitor the relationship patterns appearing in new data in real time, dynamically adjust the edges in the graph, and ensure that the knowledge graph can reflect the latest relationship strength and type through incremental learning methods. The formula is:

[0057]

[0058] in, Represents the entity vector at the current moment, Adj is the adjacency matrix, which represents the relationship between entities;

[0059] The dynamic update formula of the edge is:

[0060] β ij =softmax(LeakvReLU(a T [W g h i ||W g h j ]))

[0061] Among them, β ij represents the attention between nodes i and j, W g is the weight matrix, a is the parameter of the attention mechanism, and h i and h j are the feature vectors of nodes i and j respectively.

[0062] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0063] The present invention realizes accurate extraction of key information in programming documents through dynamic data collection and processing, and dynamically integrates this information into the knowledge graph. Through multi-source real-time monitoring technology, data is captured from official documents, open source project code bases and developer communities, and stream data processing technology is used to ensure the real-time and reliability of data. Combined with adaptive deep learning models and graph attention networks, the system can perform high-precision information extraction and semantic association analysis on programming knowledge, and construct a queryable programming knowledge graph;

[0064] The present invention combines RAG technology to dynamically query the knowledge graph based on the content of the code snippet when processing code review requests, generate more accurate code reviews and suggestions, and through dynamic weight adjustment and incremental learning mechanisms, the system can update the knowledge base in real time to maintain sensitivity and adaptability to the latest programming practices and technology trends. The goal of this application is to develop an efficient programming review system that can not only provide immediate and accurate programming support, but also greatly improve the quality and efficiency of code review. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0066] Figure 1 It is a flow chart of the large-model programming review system based on the combination of knowledge graph and RAG of the present invention. DETAILED DESCRIPTION

[0067] The following specific embodiments illustrate the implementation of the present invention, and the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Please refer to Figure 1 As shown, the present invention provides a large model programming review system based on the combination of knowledge graph and RAG, including the following steps:

[0069] Step 1: Dynamic data collection, access and parse data sources through APIs and crawler technology, and then use Apache Kafka to build a data processing pipeline. Data is transmitted from the source to the processing system through Kafka Topics. The high throughput characteristics of Apache Kafka ensure real-time processing of large-scale data;

[0070] Step 2: Knowledge graph construction: Use natural language processing technology to accurately extract key information from programming documents and dynamically integrate this information into the knowledge graph to support more efficient information retrieval and decision support.

[0071] Step 3: Combine RAG retrieval and enhanced generation. After obtaining a queryable programming knowledge graph, the model's ability to handle code review requests is enhanced by combining RAG technology, achieving more accurate code review and suggestion generation.

[0072] Step 4: Introduce a feedback mechanism and establish a user feedback interface to allow developers to submit feedback and point out problems in model predictions;

[0073] Step 5: Code review generation, combining RAG technology and introducing dynamic weight adjustment and incremental learning mechanisms to retrieve relevant information from the knowledge graph, and using the generative model to generate detailed code reviews.

[0074] In a preferred embodiment, in the step 1, for API data, a RESTful API interface is used to crawl data, and for web page data that needs to be crawled, DOM parsing and XPath selector are implemented to extract required data, and monitoring nodes are set to regularly check the update status of data sources, and Webhooks or polling mechanisms are used to ensure timely acquisition of the latest data. In the step 1, preliminary data cleaning is implemented when Apache Kafka constructs data, including filtering invalid data, standardizing formats, and correcting data errors to provide high-quality data input.

[0075] In a preferred embodiment, after the preliminary data cleaning, the data stream is smoothed using an exponentially weighted moving average model. During the smoothing process, the smoothing coefficient is dynamically adjusted according to the actual operating effect, and multi-level data processing is introduced. First, quick and rough processing is performed, and then in-depth analysis and storage are performed to optimize processing speed and accuracy. A larger α value increases the sensitivity to new data and is suitable for a rapidly changing data environment; a smaller α value improves data smoothness and reduces the impact of noise, such as configuring Kafka Producer and Consumer to ensure efficient input and output of data. Use KafkaConnect to integrate with external database systems (such as MongoDB or Elasticsearch) to achieve data persistence and further analysis. Error handling: Implement fault-tolerant mechanisms, including retry strategies, data backup, and exception monitoring to ensure system stability and data integrity.

[0076] In a preferred embodiment, the knowledge graph construction in step 2 includes the following steps:

[0077] S1: Initialization phase, in which a pre-trained model is selected based on the task characteristics, and then pre-adjusted using a small-scale programming language dataset to initially fine-tune the model parameters;

[0078] S2: In the online adjustment phase, the actual performance of the model is evaluated by monitoring the performance indicators of the model output, such as accuracy, recall, F1 score, etc., and the hyperparameters of the model, such as learning rate, number of layers, number of heads, etc., are dynamically adjusted according to the performance indicators. For example, when the model accuracy decreases, the learning rate may need to be reduced to avoid overfitting. During the model operation, real-time fine-tuning is performed through the continuous input and feedback of incremental data to ensure that the model can adapt to the dynamic changes of data.

[0079] In a preferred embodiment, the pre-trained models in S1 include BERT, RoBERTa and GPT-3. For example, for tasks requiring high semantic understanding, GPT-3 may be more suitable; for tasks requiring high-precision entity recognition, BERT or RoBERTa may be better. The preliminary fine-tuning of model parameters in S1 includes data cleaning, data labeling and pre-training. The data cleaning refers to removing noise data to ensure data quality; the data labeling refers to manually labeling a part of the data set to ensure that the model can identify key entities and relationships; the pre-training refers to pre-training the model using the cleaned data set and adjusting the initial configuration of the model.

[0080] In a preferred embodiment, the step three includes the following steps when combined with RAG search enhancement generation:

[0081] Step 1: Code snippet analysis: parse the received code snippets to identify key elements, convert the code into a structured tree representation using abstract syntax tree technology, and apply static analysis tools to preliminarily detect potential vulnerabilities and problem areas in the code, focusing on parts that may have security risks, such as SQL injection and cross-site scripting (XSS) attacks.

[0082] Step 2: Knowledge graph query, extract key features from the parsed code snippets, such as specific function calls, variable naming patterns, comment content, etc. These features will be used as query input to build query statements. Based on the extracted features, query the programming knowledge graph. The query content includes but is not limited to vulnerability descriptions, repair strategies, related programming rules and best practices. Use vector retrieval technology and graph databases to quickly match the most relevant knowledge points. The vector retrieval generation formula is:

[0083] Q=f query (code)

[0084] Where Q represents the query vector, f query represents the feature extraction function, code is the input code snippet,

[0085] The similarity calculation formula is:

[0086]

[0087] Among them, K i represents the vector representation of the i-th knowledge point in the knowledge graph, and sim represents the cosine similarity calculation;

[0088] Step 3: In the comment generation phase, the generative model is used to generate detailed comments and suggestions on the code snippets in combination with the retrieved knowledge points. The retrieved knowledge points are screened and filtered to remove redundant information to ensure that the knowledge points input to the generative model are concise and efficient. The knowledge points are tested for repetitiveness and duplicates are removed. The knowledge points are scored based on the relevance and importance of the information. High-scoring knowledge points are selected to input into the generative model. The screened knowledge points are integrated with the code snippets to build a complete context input to the generative model. The formula for context construction is:

[0089]

[0090] Step 4: Generate model call, input the constructed context into the generative model, ensure that the input format meets the requirements of the generative model, perform necessary preprocessing, and the generative model generates detailed comments based on the context. The model input formula is:

[0091] Input GPT =Input context

[0092] Generate output formula:

[0093] Comment = Comment = f GPT (Input GPT )

[0094] Among them, f GPT Represents the output function of the generated model, Input GPT Represents the model input;

[0095] Step 5: Dynamic weight adjustment. Calculate the relevance of each retrieved knowledge point and code snippet. Use cosine similarity or dot product to calculate the relevance score. Evaluate the importance of the knowledge point based on its usage frequency in historical data and user feedback. Construct a weight matrix to represent the relative importance of each knowledge point. Dynamically adjust the attention weight of the generative model based on the relevance and importance scores. Introduce a multi-layer attention mechanism in the generative model. The attention weights of different levels are allocated according to the dynamically adjusted weight matrix. The weight calculation formula is:

[0096] w i =α·sim(Q,K i )+β·importance(K i )

[0097] Among them, w i Represents knowledge point K i weight, α and β are weight coefficients, sim(Q, K i ) represents the relevance score, importance(Ki ) represents the importance score;

[0098] The attention weight allocation formula is:

[0099]

[0100] Among them, Attention layer (H) represents the attention output of the layer, H represents the hidden state of the input, K i Vector representation of knowledge points;

[0101] Step 6: Incremental learning and knowledge updating. Continuously collect new code snippets and user feedback data, build incremental data sets, clean and annotate incremental data, ensure data quality, and update the knowledge graph based on new entities and new relationships in the incremental data. Use the incremental update function of the graph database to ensure the dynamic update of the knowledge graph. Dynamically adjust the weights of knowledge points based on new knowledge points in the incremental data. Ensure that new high-importance knowledge points can be fully utilized in the retrieval and generation stages.

[0102] In a preferred embodiment, the incremental update formula in step 6 is:

[0103] Knowledge new =Knowledge old +ΔKnowledge

[0104] Among them, Knowledge new Represents the updated knowledge graph, Knowledge old represents the original knowledge graph, and ΔKnowledge represents incremental knowledge;

[0105] The weight adjustment formula is:

[0106]

[0107] in, represents the updated weight, represents the original weight, γ is the adjustment coefficient, Δw i Indicates the incremental weight adjustment value.

[0108] By combining RAG technology and introducing dynamic weight adjustment and incremental learning mechanisms, it is possible to efficiently retrieve relevant information from the knowledge graph and use the generative model to generate detailed code reviews.

[0109] In a preferred embodiment, the specific steps of introducing the feedback mechanism in step 4 are:

[0110] Feedback data processing: organize and annotate user feedback data to form a new training data set;

[0111] Model update: Use new feedback data sets to retrain and fine-tune the model to optimize the model prediction effect. For example, if a user points out that a certain API name is incorrectly recognized, the model can correctly recognize the API name after being updated with feedback data.

[0112] Multi-task learning: set parallel tasks to perform entity recognition, intent recognition, and semantic relationship extraction simultaneously, share the underlying representation of the model, and implement task-specific outputs at the top level.

[0113] In a preferred embodiment, the model parameter adjustment formula is:

[0114]

[0115] Among them, θ represents the model parameters, η is the learning rate dynamically adjusted according to the model performance, is the loss function, D batch It is the batch data extracted from the data stream.

[0116] In a preferred embodiment, the specific steps of entity recognition, intent recognition and semantic relationship extraction are:

[0117] Entity extraction: Extract entities from text using deep learning models;

[0118] Entity encoding: Encode the extracted entities into vector form, using a pre-trained word embedding model or BERT to generate context-related vector representations;

[0119] Graph attention mechanism update: Use the graph attention network to update the entity vector and dynamically adjust its representation based on the interaction between entities;

[0120] Relationship recognition: Apply graph attention network to identify semantic relationships between entities. By constructing an entity relationship graph, nodes represent entities, edges represent relationships between entities, and edge weights represent the strength of relationships.

[0121] Relationship update: monitor the relationship patterns appearing in new data in real time, dynamically adjust the edges in the graph, and ensure that the knowledge graph can reflect the latest relationship strength and type through incremental learning methods. The formula is:

[0122]

[0123] in, Represents the entity vector at the current moment, Adj is the adjacency matrix, which represents the relationship between entities;

[0124] The dynamic update formula of the edge is:

[0125] β ij =softmax(LeakyReLU(a T [W g h i ||W g h j ]))

[0126] Among them, β ij represents the attention between nodes i and j, W g is the weight matrix, a is the parameter of the attention mechanism, and h i and h j are the feature vectors of nodes i and j respectively.

[0127] By adopting the methods of adaptive deep learning and dynamic semantic association analysis, it is possible to extract high-precision information from programming documents and dynamically update the knowledge graph to adapt to the ever-changing programming environment and practice. This method will greatly enhance the practicality and timeliness of the knowledge graph and provide developers with instant and accurate programming support.

[0128] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A large-model programming review system based on the combination of knowledge graph and RAG, characterized by: The following steps are involved: Step 1: Dynamic data collection, access and parse data sources through APIs and crawler technology, and then use Apache Kafka to build a data processing pipeline. Data is transmitted from the source to the processing system through Kafka Topics; Step 2: Knowledge graph construction: Use natural language processing technology to accurately extract key information from programming documents and dynamically integrate this information into the knowledge graph to support more efficient information retrieval and decision support. Step 3: Combine RAG retrieval and enhanced generation. After obtaining a queryable programming knowledge graph, the model's ability to handle code review requests is enhanced by combining RAG technology, achieving more accurate code review and suggestion generation. Step 4: Introduce a feedback mechanism and establish a user feedback interface to allow developers to submit feedback and point out problems in model predictions; Step 5: Code review generation, combining RAG technology and introducing dynamic weight adjustment and incremental learning mechanisms to retrieve relevant information from the knowledge graph, and using the generative model to generate detailed code reviews.

2. According to claim 1, the large model programming review system based on the combination of knowledge graph and RAG is characterized by: In the step 1, for API data, a RESTful API interface is used to crawl data. For web page data that needs to be crawled, DOM parsing and XPath selector are implemented to extract required data, and monitoring nodes are set to regularly check the update status of data sources. Webhooks or polling mechanisms are used to ensure timely acquisition of the latest data. In the step 1, preliminary data cleaning is implemented when Apache Kafka constructs data, including filtering invalid data, standardizing formats, and correcting data errors to provide high-quality data input.

3. The large model programming review system based on the combination of knowledge graph and RAG according to claim 2 is characterized by: After the preliminary data cleaning, the data stream is smoothed using an exponentially weighted moving average model. During the smoothing process, the smoothing coefficient is dynamically adjusted according to the actual operating effect, and multi-level data processing is introduced. First, quick and rough processing is performed, and then in-depth analysis and storage are performed to optimize processing speed and accuracy.

4. The large model programming review system based on the combination of knowledge graph and RAG according to claim 1 is characterized in that: The knowledge graph construction in step 2 includes the following steps: S1: Initialization phase, in which a pre-trained model is selected based on the task characteristics, and then pre-adjusted using a small-scale programming language dataset to initially fine-tune the model parameters; S2: In the online adjustment phase, the actual performance of the model is evaluated by monitoring the performance indicators of the model output, and the hyperparameters of the model are dynamically adjusted according to the performance indicators. During the operation of the model, real-time fine-tuning is performed through the continuous input and feedback of incremental data to ensure that the model can adapt to the dynamic changes of the data.

5. The large model programming review system based on the combination of knowledge graph and RAG according to claim 4 is characterized in that: The pre-trained models in S1 include BERT, RoBERTa and GPT-3. The preliminary fine-tuning of model parameters in S1 includes data cleaning, data labeling and pre-training. The data cleaning refers to removing noise data to ensure data quality; the data labeling refers to manually labeling a part of the data set to ensure that the model can identify key entities and relationships; the pre-training refers to pre-training the model using the cleaned data set and adjusting the initial configuration of the model.

6. The large model programming review system based on the combination of knowledge graph and RAG according to claim 1 is characterized by: The step three includes the following steps when combined with RAG search enhancement generation: Step 1: Code snippet analysis: parse the received code snippet to identify the key elements, convert the code into a structured tree representation using abstract syntax tree technology, and apply static analysis tools to preliminarily detect potential vulnerabilities and problem areas in the code. Step 2: Knowledge graph query, extract key features from the parsed code snippet, build query statements, query the programming knowledge graph based on the extracted features, use vector retrieval technology and graph database to quickly match the most relevant knowledge points, and the vector retrieval generation formula is: Q=f query (code) Where Q represents the query vector, f query represents the feature extraction function, code is the input code snippet, The similarity calculation formula is: Among them, K i represents the vector representation of the i-th knowledge point in the knowledge graph, and sim represents the cosine similarity calculation; Step 3: In the comment generation phase, the generative model is used to generate detailed comments and suggestions on the code snippets in combination with the retrieved knowledge points. The retrieved knowledge points are screened and filtered to remove redundant information to ensure that the knowledge points input to the generative model are concise and efficient. The knowledge points are tested for repetitiveness and duplicates are removed. The knowledge points are scored based on the relevance and importance of the information. High-scoring knowledge points are selected to input into the generative model. The screened knowledge points are integrated with the code snippets to build a complete context input to the generative model. The formula for context construction is: Step 4: Generate model call, input the constructed context into the generative model, ensure that the input format meets the requirements of the generative model, perform necessary preprocessing, and the generative model generates detailed comments based on the context. The model input formula is: Input GPT =Input context Generate output formula: Comment=Comment=f GPT (Input GPT ) Among them, f GPT Represents the output function of the generated model, Input GPT Represents the model input; Step 5: Dynamic weight adjustment. Calculate the relevance of each retrieved knowledge point and code snippet. Use cosine similarity or dot product to calculate the relevance score. Evaluate the importance of the knowledge point based on its usage frequency in historical data and user feedback. Construct a weight matrix to represent the relative importance of each knowledge point. Dynamically adjust the attention weight of the generative model based on the relevance and importance scores. Introduce a multi-layer attention mechanism in the generative model. The attention weights of different levels are allocated according to the dynamically adjusted weight matrix. The weight calculation formula is: w i =α·sim(Q,K i )+β·importance(K i ) Among them, w i Represents knowledge point K i weight, α and β are weight coefficients, sim(Q, K i ) represents the relevance score, importance(K i ) represents the importance score; The attention weight allocation formula is: Among them, Attention layer (H) represents the attention output of the layer, H represents the hidden state of the input, K i Vector representation of knowledge points; Step 6: Incremental learning and knowledge updating. Continuously collect new code snippets and user feedback data, build incremental data sets, clean and annotate incremental data, ensure data quality, and update the knowledge graph based on new entities and new relationships in the incremental data. Use the incremental update function of the graph database to ensure the dynamic update of the knowledge graph. Dynamically adjust the weights of knowledge points based on new knowledge points in the incremental data. Ensure that new high-importance knowledge points can be fully utilized in the retrieval and generation stages.

7. The large model programming review system based on the combination of knowledge graph and RAG according to claim 6 is characterized by: The incremental update formula in step 6 is: Knowledge new =Knowledge old +ΔKnowledge Among them, Knowledge new Represents the updated knowledge graph, Knowledge old represents the original knowledge graph, and ΔKnowledge represents incremental knowledge; The weight adjustment formula is: in, represents the updated weight, represents the original weight, γ is the adjustment coefficient, Δw i Indicates the incremental weight adjustment value.

8. The large model programming review system based on the combination of knowledge graph and RAG according to claim 1 is characterized by: The specific steps of introducing the feedback mechanism in step 4 are: Feedback data processing: organize and annotate user feedback data to form a new training data set; Model update: Use new feedback data sets to retrain and fine-tune the model to optimize the model prediction effect; Multi-task learning: set parallel tasks to perform entity recognition, intent recognition, and semantic relationship extraction simultaneously, share the underlying representation of the model, and implement task-specific outputs at the top level.

9. The large model programming review system based on the combination of knowledge graph and RAG according to claim 8 is characterized by: The model parameter adjustment formula is: Among them, θ represents the model parameters, η is the learning rate dynamically adjusted according to the model performance, is the loss function, D batch It is the batch data extracted from the data stream.

10. The large model programming review system based on the combination of knowledge graph and RAG according to claim 8 is characterized by: The specific steps of entity recognition, intent recognition and semantic relationship extraction are: Entity extraction: Extract entities from text using deep learning models; Entity encoding: Encode the extracted entities into vector form, using a pre-trained word embedding model or BERT to generate context-related vector representations; Graph attention mechanism update: Use the graph attention network to update the entity vector and dynamically adjust its representation based on the interaction between entities; Relationship recognition: Apply graph attention network to identify semantic relationships between entities. By constructing an entity relationship graph, nodes represent entities, edges represent relationships between entities, and edge weights represent the strength of relationships. Relationship update: monitor the relationship patterns appearing in new data in real time, dynamically adjust the edges in the graph, and ensure that the knowledge graph can reflect the latest relationship strength and type through incremental learning methods. The formula is: in, Represents the entity vector at the current moment, Ad j is the adjacency matrix, which represents the relationship between entities; The dynamic update formula of the edge is: β ij =softmax(LeakyReLU(a T [IN g h i ||In g h j ])) Among them, β ij represents the attention between nodes i and j, W g is the weight matrix, a is the parameter of the attention mechanism, and h i and h j are the feature vectors of nodes i and j respectively.

Images