Code pre-training and generating method and system oriented to function call relation

Abstract

The invention discloses a function call relation-oriented code pre-training and generating method and system. The method comprises the following steps: 1) for each piece of code warehouse data in a code data set, extracting context information from the piece of code warehouse data; the context information is function call information, parameter dependence and a global logic structure; 2) training a code-related deep learning model by using the context information to obtain a code generation model; 3) inputting code demand description or a code to be edited into the code generation model to generate a corresponding target code; wherein for the input code demand description, the code generation model generates a target code meeting the demand according to the code demand description; and for an input to-be-edited code, the code generation model automatically repairs or optimizes the to-be-edited code to obtain a target code. According to the method, logic consistency and availability of generated codes are optimized, and code generation in a complex code scene is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of information technology, and particularly relates to a function call relationship-oriented code pre-training and generation method and system. BACKGROUND

[0002] With the rapid growth of software development needs and the continuous expansion of code size, code generation technology is becoming one of the important research directions in the field of software engineering. Code generation technology can help developers efficiently complete code completion, code repair, and code refactoring tasks, greatly improving the efficiency and quality of software development. In recent years, code generation technology based on deep learning has made significant breakthroughs, especially the emergence of large language models (LLMs), which have made code generation capabilities reach an unprecedented level. These models can generate code fragments that are grammatically correct and logically reasonable by training on large-scale code and natural language data. However, in practical applications, code generation technology still faces many challenges, especially in terms of code logic integrity and function dependency perception capabilities.

[0003] Currently, code generation technology based on large language models has become a research hotspot. Models such as GPT, CodeT5, CodeGen, etc. have shown strong capabilities in code completion, code repair, and code generation tasks through the self-attention mechanism of the Transformer architecture to model the context of the code. Model training is usually divided into two stages: pre-training and fine-tuning.

[0004] In the pre-training stage, the model usually learns the grammar rules, semantic characteristics, and relationships between contexts of large-scale code and natural language data through autoregressive language modeling (such as predicting the next Token) or fill-in-the-blank tasks (such as generating missing code fragments). This stage relies on massive training data and the expansion of model parameter size to improve generation performance. In order to adapt to specific task scenarios, the model will be fine-tuned based on pre-training. For example, for code repair tasks, the fine-tuning dataset will contain a large number of code errors and repair pairs; for specific languages (such as Python or Java), the corresponding language code data will be used for optimization. The fine-tuning stage can significantly improve the model's performance on specific tasks or languages. Although the above technologies have made great progress, their generation capabilities still have many shortcomings in complex code scenarios.

[0005] Code has natural structural properties, especially function-level call relationships, parameter dependencies, and global variable state sharing, which form an important part of the code logic. However, current large language models are mostly trained based on natural language processing techniques, mainly modeling code context through text sequences, and lack the ability to model the structural properties of code. Although this method can generate syntactically correct code, it has obvious limitations in function dependency perception and global logic consistency. For example, when generating code that calls a certain function, the model often cannot accurately understand the definition of the target function or the dependency relationship of the call parameters, resulting in incomplete or inaccurate code logic.

[0006] In addition, current large language models rely on vast amounts of code data during training, but the acquisition of high-quality code data is not unlimited. As the data size gradually expands, the marginal benefit of new data on model performance improvement will gradually decrease. In addition, many models are trained in a "generalized" manner, relying on increasing data volume and parameter size to improve performance, without fully utilizing the unique structural information (such as function call chains and dependency graphs) contained in the code. This approach not only leads to low data utilization efficiency, but also limits the model's ability to understand and generate complex code logic.

[0007] Current code generation relies on a single input sample's natural language context for inference during the reasoning phase. This limitation is particularly evident in function calls and parameter dependencies, mainly in the following aspects:

[0008] 1) Lack of global logic consistency constraints. When a target function is called in multiple context scenarios, existing methods often struggle to ensure the logical consistency of the generated results in different scenarios. For example, the same function may need to implement consistent functional logic in multiple calling scenarios, but existing generation methods only generate code based on a single context, resulting in generated results that may exhibit functional conflicts or logical inconsistencies in different scenarios.

[0009] 2) Lack of global optimization ability for multiple samples. Existing methods often fail to integrate information from multiple related input samples when generating a target function, lacking a collaborative constraint mechanism. For example, a target function may be referenced by multiple callers, but existing generation methods mainly generate functions based on natural language descriptions as context, without considering the context generated by the call relationship, and fail to fully integrate information from other calling scenarios, resulting in incomplete target function logic or failure to meet the calling needs of other scenarios. This limitation directly affects the logical accuracy and functional adaptability of the generated code, especially in complex code scenarios.

[0010] In summary, in the problem of code generation, existing methods mainly rely on general pre-training process, focusing on learning from large-scale data or generating single data based on local context. However, these methods still face obvious deficiencies in modeling the logical consistency of code generation and the collaborative use of multi-sample information. These challenges result in low efficiency of using high-quality data by existing methods, and lack of effective modeling ability for complex dependency relationships between functions, making it difficult to meet the needs of code generation in complex scenarios. Therefore, under the trend of code generation tasks gradually developing towards refinement and logic, how to deeply excavate the structured characteristics of existing code data and fully utilize global context information in the generation process to improve the logical consistency and functional integrity of code generation has become a key problem that needs to be solved in current technology development. SUMMARY

[0011] In view of the limitations of large language models in code generation tasks in the prior art, the present application provides a function call relationship-oriented code pre-training and generation method and system.

[0012] The present application obtains high-quality code data from real code repositories and extracts the function-level dependency relationships therein. These data naturally contain function call information, parameter dependencies, and global logical structures. These information collectively constitute the context environment of the code snippets, providing rich context information and more complete semantic understanding for model training. By using these context information (i.e. naturally existing function call information, parameter dependencies, and global logical structures in the data) for training, the present application designs a pre-training method based on function dependency relationships, enabling the model to effectively capture key information in the code and thus improve the accuracy and logical consistency of the generated code. Key information is the most representative and most valuable feature in the context information for the code generation task, such as function call patterns, parameter dependency relationships, etc. By extracting these key information, the model can more accurately generate code that conforms to semantic logic. This technical point can be independently applied to the generation task and is suitable for various code generation scenarios without relying on technical support from other modules.

[0013] Secondly, the present application proposes an independent reasoning strategy aimed at further optimizing the context understanding in the generation process. In this strategy, the model can generate multiple candidate code paths by integrating one / multiple relevant context information when generating the target code. In these paths, the model selects the most appropriate generation result through an internal constraint mechanism, ensuring the consistency and integrity of the generated code under multiple contexts. This strategy can be used independently with the aforementioned training method or implemented independently according to needs for optimizing specific generation scenarios, making the generated function logic more complete and accurate.

[0014] The technical scheme of the present application is:

[0015] A method for code pre-training and generation based on function call relationship, comprising the following steps:

[0016] 1) For each piece of code repository data in the code data set, extract context information from the piece of code repository data; the context information is function call information, parameter dependency and global logical structure;

[0017] 2) Use the context information to train a code-related deep learning model to obtain a code generation model;

[0018] 3) Input the code requirement description or the code to be edited into the code generation model to generate the corresponding target code; for the input code requirement description, the code generation model generates the target code according to the code requirement description; for the input code to be edited, the code generation model automatically repairs or optimizes the code to be edited to obtain the target code.

[0019] Further, the method for extracting the context information is:

[0020] 11) Fully scan the file structure in each piece of code repository data to obtain each function and its call chain in the piece of code repository data, and build the dependency relationship of each function;

[0021] 12) For each function f i in the piece of code repository data, use an abstract syntax tree to extract the parameters involved in the function f i , and establish a first key-value pair table to track and record the parameter passing relationship between each parameter involved in the function f i and other functions, and take the parameter passing relationship between the function f i and other functions as the parameter dependency relationship; and optimize the function call chain of the function f i according to the parameter passing relationship; wherein the key in the first key-value pair table is the function f i , and the value is the parameter involved in the function f i ;

[0022] 13) Use static analysis technology to record the global variables contained in the piece of code repository data, and track the state changes of each global variable by constructing a second key-value pair table; further optimize the function call chain of the function f i according to the state changes of the global variables contained in the function f i ; wherein the key in the second key-value pair table is a global variable p in the function f i , and the value is the state of the global variable p in other functions.

[0023] 14) Construct a function dependency graph G corresponding to the function call chain of each function of the code repository data as the contextual information of the code repository data according to the function call chain of each function of the code repository data obtained in step 13); each node in the function dependency graph G represents a function, and each edge represents the dependency relationship between functions; as the contextual information for subsequent model training.

[0024] Further, by tracking the definition and call of the functions in the specific code repository data through the global symbol table, the interaction and dependency relationship between each function and other functions in the specific code repository data are extracted, and the function call chain corresponding to the function is constructed.

[0025] Further, the method for training the code-related deep learning model is:

[0026] 21) Add semantic descriptions to the function dependency graph G of each code repository data, and then input it into the deep learning model; the encoder of the deep learning model extracts key contextual information from the function dependency graph G and its semantic description of each code repository data using a self-attention mechanism, and maps it to a high-dimensional vector space to generate a representation vector of the code repository data;

[0027] 22) The decoder of the deep learning model generates the target code of the code repository data according to the representation vector of the code repository data; wherein the loss function used to train the deep learning model is Where, logP θ (y t |x input ,y <t ) represents the probability of generating the tth Token of x input , y <t is the generated partial code, x input is the formal representation of the function dependency graph G, y t is the code generated according to the tth Token, and n is the total number of Tokens in x input .

[0028] Further, in step 3), the method for generating the corresponding target code by the code generation model is:

[0029] 31) The code generation model generates multiple different input data segments according to different contexts of the function calls in the user input data, and different input data segments contain different context information;

[0030] 32) Generate a candidate code path according to each input segment, and then filter each candidate code path through a set of constraints; wherein different candidate code paths have different call parameters or dependency relationships;

[0031] 33) generating a corresponding candidate code for each candidate code path reserved according to the screening; and then merging the candidate codes to generate the target code.

[0032] Further, the method for merging the candidate codes is to merge the codes corresponding to the same token in the candidate codes.

[0033] A system for function call relationship-oriented code pre-training and generation, characterized by a context information extraction module, a model training module, and a code generation module.

[0034] The context information extraction module is configured to extract context information from each piece of code repository data in a code data set; the context information includes function call information, parameter dependency, and global logical structure.

[0035] The model training module is configured to train a code-related deep learning model using the context information to obtain a code generation model.

[0036] The code generation module is configured to generate a corresponding target code according to an input code requirement description or an edited code using the code generation model; for an input code requirement description, the code generation model generates a target code that meets the requirement according to the code requirement description; for an input edited code, the code generation model automatically repairs or optimizes the edited code to obtain a target code.

[0037] A server, characterized by a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for executing the above method.

[0038] A computer-readable storage medium having a computer program stored thereon, characterized in that the computer program is executed by a processor to implement the above method.

[0039] The present application has the following beneficial effects in optimizing the ability of large language models to generate code:

[0040] 1) By training with data having function dependency relationships, the performance of the model in code generation and other tasks is improved.

[0041] 2) By deeply mining function-level dependency relationships, the performance of the model in complex code scenarios, including multi-function dependency, function call, and global logical consistency, is significantly improved.

[0042] 3) By using a multi-sample collaborative reasoning strategy, the logical consistency and usability of the generated code are further optimized, and can be independently selected for use according to actual needs. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is the basic flow of the code pre-training and generation method and system facing function call relationship. DETAILED DESCRIPTION

[0044] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail through specific implementation cases.

[0045] The present application proposes a code pre-training and generation method facing function call relationship. Through modeling the function level call relationship and parameter dependency in the code, more accurate, logically consistent and contextually required code can be generated. Specifically, based on the importance of fully utilizing data and code pre-training model, the present application designs a system to automatically solve the related problems of code data and model. The input and output of this system are described as follows:

[0046] At the input end, the user provides original code data, code requirement description or code that needs to be edited. The system processes these input data into a format suitable for training, and extracts function level dependency information by identifying the call relationship and dependency between functions. These information will help the model understand the interaction between functions when generating code, and ensure that the generated code can meet the complex function dependency and call relationship.

[0047] At the output end, the trained code generation model receives the input code requirement description or existing code at this stage. For code requirement description, the model will generate target code that meets the user's requirements according to the description; for existing code, the model will analyze and automatically repair or optimize the code. The output of the model not only meets the local context requirements, but also ensures that the generated code meets the global dependency relationship, avoiding generating logically inconsistent or functionally incomplete code.

[0048] In order to improve the code generation ability of the existing model, the present application: (1) first designs a pre-training data extraction method facing function level dependency perception; (2) then trains a deep learning model related to code; (3) adopts a multi-path generation strategy in the inference process; (4) finally evaluates the generation results of the code generation method and system facing function level dependency perception. The performance is introduced as follows:

[0049] I. Pre-training data extraction method facing function level dependency perception

[0050] 1.1 Data acquisition

[0051] The quality of the data directly determines the effectiveness of model training. In the present invention, in order to ensure that the code generation model can perform better in real-world development scenarios, we extract a large amount of high-quality code data sets from various code data sources. Each piece of data in the code data set represents a real-world code repository, and the code in the repository contains rich function call information, modular structure, and various function dependencies, but the specific content may vary depending on the data source. Overall, the code data set ensures the richness of these information to support model training.

[0052] The data sources can include but are not limited to public code repositories, project code, or other available code data. The code elements in the data set can include function definitions, function call information, parameter dependencies, function code snippets, and other information. In addition, in order to further ensure the diversity of the data, we also select code repositories containing multiple programming languages and technology stacks to ensure that the generation model can exhibit good adaptability in diversified programming languages and project architectures.

[0053] Data preprocessing includes cleaning, formatting, and standardizing the original code data. Specifically, the present invention removes irrelevant content, unifies the structure and indentation style of the code, and standardizes the code to ensure that the data can be better used for model training. The preprocessed data will retain key information that helps capture the functionality and logic of the code, while removing redundant and irrelevant parts. In this way, it ensures that the model can efficiently learn and generate code that meets the syntax rules and functional requirements.

[0054] 1.2 Extraction of context information

[0055] The present invention utilizes large-scale data sets obtained from real code repositories, which contain natural function call dependencies that do not require additional manual annotation and can be extracted and modeled through automated analysis. This data-driven approach based on natural dependencies provides comprehensive context support for the model, enabling the generated code to accurately reflect complex function interactions and dependency structures.

[0056] In the present invention, the extraction and utilization of function dependencies are key steps in achieving efficient code generation. We use multiple technical means to analyze a large amount of code data, capture the dependencies between functions, and extract function call chains from them. In order to ensure that the model can understand complex function calls and their underlying dependencies, we cover direct function call dependencies, parameter passing dependencies, and cross-file and cross-module function call dependencies, among others, forming a large and complex dependency network. Specifically, the extraction process of context information includes the following steps:

[0057] (1) Function call chain construction: In this phase, we conduct a comprehensive scan of the file structure in the entire project, and through various analysis techniques, we attempt to parse each function and its call chain in the code, constructing the dependency relationship of each function. In addition to the common extraction of function dependency within a single file, actual software projects often encounter complex situations such as cross-file calls, cross-module calls, and project-level function dependencies. Therefore, to address these complex situations, the invention uses abstract syntax tree and symbolic analysis static analysis techniques for each specific code repository data in the data set, traces the definition and call of functions in the repository through the global symbol table, and extracts the interaction and dependency between each function and other functions, constructing the function call chain. This enables the model to understand the interaction between different files when generating code, rather than just limiting to function calls within the same file, ensuring that the model can fully understand the multi-level dependency relationships in the code.

[0058] (2) Parameter dependency extraction: During function calls, the input and output parameters of functions often have important dependency relationships. To capture these dependencies, the invention analyzes each function's parameters in detail. Parameter dependency relationships are not limited to local dependencies within a function, but also include parameter passing across functions and modules. By using abstract syntax tree technology to extract the parameters involved in each function in each code repository, and establishing a key-value pair table for tracking records, the parameter passing relationship (i.e., parameter dependency) between functions is identified, and the constructed function call chain is optimized; wherein, for each function f i , the abstract syntax tree is used to extract the parameters involved in the function f i , and a key-value pair table is established for tracking records, with the key being the function f i , and the value being the parameters involved in the function f i .

[0059] (3) Capture of global state dependency: In addition to function call chains and parameter dependencies, state dependency relationships between functions are also crucial. For example, some functions may modify global variables, and other functions may rely on these global states for calculations. To capture these dependencies, the invention uses static analysis techniques to record global variables in the code contained in each code repository, and through the construction of a key-value pair table, it traces the state changes and further optimizes the constructed function call chain, thereby fully reflecting the state dependency relationships between functions in the repository; wherein the key-value pair table has a global variable p in the function f i as the key, and the state of the global variable p in other functions as the value.

[0060] Finally, to represent the complex function call relationship in a structured manner, the present application uses a graph structure to represent the function call chain of the function dependency relationship contained in each code repository data. Each node in the graph represents a function, and each edge represents the dependency relationship between functions, thereby generating a corresponding function dependency graph for each code repository data, which clearly depicts the mutual relationship between functions in the code as context information for subsequent model training.

[0061] Let the function set be F = {f1, f2, …, f n}, and the function dependency graph be G = (F, E), where E is the edge set between functions representing the dependency relationship. For any pair of functions f i ,f j ∈ F, if f i calls f j , a edge e i,j ∈ E is added to the graph G to represent the dependency of function f i on function f j .

[0062] II. Training code-related deep learning models

[0063] In the present application, training code-related deep learning models using data containing function call dependency relationships is the key to efficient code generation. By pre-training on large-scale code data, the model can learn the syntax, semantics, and complex dependency relationships between functions of the code, thereby providing strong capabilities for downstream code generation tasks. The present application designs a method that enables the model to generate code that meets the requirements based on the input context information (such as function call relationships, parameter passing, etc.).

[0064] The specific development of the entire system model can be considered as a combination of the following parts: model structure design and pre-training task design. It is worth noting that using code pre-training models is one of the implementation methods proposed by the present application for function call relationships.

[0065] 2.1 Code-related deep learning model structure

[0066] The code-related deep learning model design involved in the present application can be applied to popular Transformer-based Decoder frameworks and Encoder-Decoder frameworks. Here, taking the deep learning model with 600 million parameters in CodeT5 as an example, the present application explains how to continue pre-training on the weights of this model, which allows the model to benefit from the knowledge and representations learned by CodeT5, which is particularly advantageous in code-related tasks.

[0067] In the pre-training stage, function dependency paths (i.e., function call chains of functions) are extracted from the function dependency graph G, and the input of the model is constructed based on the paths. The input can include structured information composed only of dependency paths, or can be combined with additional semantic descriptions to enrich the model's understanding of function dependency relationships. The model learns these dependencies and their semantic associations during training to accurately generate target code in the inference stage.

[0068] In this structure, the model receives input data containing rich contextual dependencies through the encoder part. The model uses self-attention mechanisms to deeply extract key contextual information and map it to a high-dimensional vector space to generate semantically rich representation vectors. Each Token (Token is the smallest input unit formed after the input is divided, which is the basic carrier for the large model to understand and generate code, such as code snippets, function names, natural language, etc.) is converted into a vector, so that the representation vector can encode both its semantic features and its contextual relationships. In this way, the encoder can provide rich information for the subsequent decoding generation process, helping the model understand how to generate the complete implementation of a function based on the context.

[0069] The task of the decoder part is to generate the desired code for x based on the representation vectors generated by the encoder. The model will generate code step by step based on the input requirement description and key contextual information of function calls, and ensure that the generated code meets the functional requirements of x. The decoder uses an autoregressive approach during the generation process, generating one new Token at a time and using it as input for the next step until the complete function code is generated.

[0070] 2.2 Design of pre-training tasks

[0071] To enable the model to efficiently handle contextual dependencies, the invention designs a code generation task based on contextual information. The goal of the task is to help the model understand how to generate accurate code between different contexts. Through self-supervised learning, the model can learn complex contextual relationships from large-scale training data and apply them to the code generation task.

[0072] The model proposed by the invention accepts the formal description of the input as follows:

[0073] x input = x1, x2, …, x m , x m is the mth Token in the context information.

[0074] Correspondingly, the output can be formally described as follows, where y i represents the i-th output token.

[0075] y = y1, y2, …, y n

[0076] It is worth noting that according to different requirements of different tasks, the input data can contain natural language description text or function header fragments (such as code generation tasks), or existing code that needs to be modified (such as code enhancement tasks). No matter which task, the method of the present application can effectively utilize the context dependency relationship for training and generation, so as to generate high-quality code meeting the requirements.

[0077] Formally, the loss function of this task can be defined as:

[0078]

[0079] Where logP θ (y t |x input ,y <t ) represents the probability of generating the tth Token of x input , y <t is the generated partial code.

[0080] III. Employing a multi-path generation strategy in the reasoning process

[0081] In the reasoning phase of code generation, the model needs to generate the implementation of the target code according to the user's input. In order to further improve the accuracy, logical consistency and functional integrity of the generated code, the present application proposes a multi-path generation strategy. Through this strategy, the model can utilize input information from multiple different contexts to generate multiple candidate code paths, and select the best generation result among these paths, thereby improving the quality of the generated code.

[0082] 3.1 Core idea of multi-path generation strategy

[0083] Unlike traditional code generation methods that rely on a single input context, the generated code is not only influenced by the current context, but also by the upstream and downstream relationships of function calls, function dependency chains, etc. Therefore, the multi-path generation strategy proposed by the present application takes into account the information of multiple contexts to some extent, and generates multiple candidate paths to enhance the flexibility and accuracy of generation. However, it is worth noting that this strategy is not essential to the present application, it is only used to further optimize the code generation process and is not relied on as a core technology.

[0084] The core idea of the multi-path generation strategy is: for a given requirement description or existing code snippet, the model will generate multiple code candidate paths from multiple different context information. Each path may be based on different contexts and function dependencies to reason and generate different code snippets. In the final generation result, the model will filter these candidate paths according to the constraints (such as function dependency relationship, code logic consistency, etc.) to select the code that best meets the user's requirements.

[0085] 3.2 Implementation steps of the multi-path generation strategy

[0086] The implementation of the multi-path generation strategy can be divided into the following technical points:

[0087] (1) Diversification of input data: In the reasoning process, the input data (such as requirement description or existing code) may not be complete, or there may be different context information. Therefore, the system can selectively diversify the input data in different ways. For example, for the generation of a target function, the model can generate multiple different input data snippets based on the different contexts (such as the parameter types of the calling function, the function logic of the caller) contained in the requirement description or existing code. These input data snippets will contain different context information, and the model will generate a candidate code path for each input snippet. These paths may differ based on different calling parameters or dependencies, and the generated code implementations may also differ.

[0088] (2) Candidate path generation: Through the autoregressive decoder, the model will generate multiple candidate paths based on the input context. Each candidate path is generated independently and depends on its corresponding context information. The model will generate the corresponding target function code based on each input snippet (such as function requirement description, calling function context, etc.), and each path will generate a different code implementation. Each path represents a reasoning approach of the model for the target function. These candidate paths may be based on different parameter configurations, function call relationships, or function body implementations, resulting in different codes.

[0089] (3) Constraint mechanism and path filtering: Through the constraint mechanism, the model filters the multiple candidate paths generated. The constraint mechanism includes but is not limited to function dependency consistency, code logic consistency, and the degree of matching with the user's requirement description.

[0090] In the multi-path generation strategy, for each generated token, the model will perform a merge operation based on the generation results of multiple candidate paths. Let the generated candidate paths be P1, P2, …, P k , and the logits of the tth token in each path be logit t(P1), logit t (P2),…, logit t (P k ), where k is the number of candidate paths.

[0091] By constraining and screening the logits of these paths, the final synthesis result of the tth token is obtained This result represents the optimal prediction after merging all candidate paths. The merging operation can adopt various strategies, and the specific operation can be uniformly represented as:

[0092]

[0093] Where combine represents the operation on the logits of multiple paths. The final generated code is the result generated by the model after comprehensive evaluation of all candidate paths.

[0094] 3.4 Advantages of multi-path generation strategy

[0095] After introducing the multi-path generation strategy, the generation result of the present application can be further improved in diversity and accuracy. Through this strategy, the model can generate code from different angles by considering multiple context information when dealing with complex context dependencies, thereby improving the diversity and coverage of the generated result. Fully considering the complex dependency relationship between functions and the call chain, it ensures that the generated code can correctly handle multi-level dependencies, provides more choices, and increases the flexibility of the generated result.

[0096] The present application also provides a function call relationship-oriented code pre-training and generation system, characterized in that it comprises a context information extraction module, a model training module and a code generation module.

[0097] The context information extraction module is used to extract context information from each piece of code repository data in the code data set; the context information is function call information, parameter dependency and global logical structure.

[0098] The model training module is used to train a code-related deep learning model using the context information to obtain a code generation model.

[0099] The code generation module is used to generate corresponding target code according to the input code requirement description or the code to be edited using the code generation model; for the input code requirement description, the code generation model generates target code that meets the requirements according to the code requirement description; for the input code to be edited, the code generation model automatically repairs or optimizes the code to be edited to obtain the target code.

[0100] The application also provides a server, characterized by comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program comprising instructions for executing the above method.

[0101] The application also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the above method.

[0102] Fourth, technical effect evaluation of the application

[0103] Finally, to comprehensively evaluate the functionality of the application, we conducted detailed experiments on multiple datasets, mainly including testing on the pre-training task dataset of the application and task testing based on the CodeEval dataset. These tests aim to verify the performance of the application in code generation tasks, covering both pre-training tasks and practical programming task application scenarios.

[0104] In the pre-training task, the application utilizes the pre-trained model in combination with the inference strategy to achieve significant results. In this phase, we mainly use BLEU, Exact Match (EM), and CodeBLEU as evaluation metrics to comprehensively measure the quality of the generated code. Experimental results show that compared with the baseline model (CodeT5-small) that has not been pre-trained and only fine-tuned, the BLEU score of the application on this dataset is 30.25, which is significantly higher than the baseline model's 16.83. Similarly, in the Exact Match (EM) and CodeBLEU evaluations, the model of the application also shows high scores, with EM reaching 23.96 and CodeBLEU reaching 54.15, both far exceeding the baseline model's 3.54 and 3.54.

[0105] The CodeEval dataset contains 230 functions, each with multiple test cases to evaluate whether the generated code is functionally consistent with the target function. In experiments based on this dataset, the model of the application performs functional consistency testing through the generated functions, verifying the practicality and accuracy of the generated code. Test results show that the application can generate high-quality code and exhibit strong functional consistency in multiple use cases. Specifically, the pass@1 indicator of the model reaches 10.87%. Although the model of the application has not completely surpassed the baseline model in terms of generation accuracy at pass@1, especially compared with GPT-40, it still has good potential in multi-path generation strategies and handling complex dependencies.

[0106] Practical application scenarios of the application

[0107] As the software development process becomes increasingly complex, developers face more and more challenges. The proposal of code generation model effectively improves the efficiency and quality of code generation, especially in function generation, code repair and other tasks, which can reduce human intervention and improve development efficiency. However, with the continuous expansion of model parameter size, the demand for training data and computing resources also increases. Through the method of the invention, existing data can be more efficiently utilized, and the training effect of the model can be improved, thereby meeting the growing demand for software development.

[0108] The specific embodiments of the invention disclosed above are intended to help understand the content of the invention and to implement the same, and those skilled in the art can understand that various replacements, changes and modifications are possible without departing from the spirit and scope of the invention. The invention should not be limited to the content disclosed in the examples of the specification, and the protection scope of the invention is defined by the scope of the claims.

Claims

1. A method for code pre-training and generation for function call relationships, comprising the following steps: 1) For each piece of code repository data in the code dataset, extract context information from the piece of code repository data; The context information includes function call information, parameter dependencies and global logical structure; 2) using the context information to train a code-related deep learning model to obtain a code generation model; 3) Inputting the code requirement description or the code to be edited into the code generation model to generate the corresponding target code; wherein, for the input code requirement description, the code generation model generates the target code that meets the requirements according to the code requirement description; For the input code to be edited, the code generation model automatically repairs or optimizes the code to be edited to obtain the target code.

2. The method according to claim 1, characterized in that Method for extracting the context information: 11) Perform a comprehensive scan of the file structure in each code repository data, obtain each function and its call chain in the code repository data, and build the dependency relationship of each function; 12) For each function f in the code warehouse data i , use the abstract syntax tree to extract the function f i The parameters involved, and establish the first key-value pair table to track and record the function f i Each parameter involved in the parameter transfer relationship with other functions, the function f i The parameter passing relationship with other functions is regarded as parameter dependency; And optimize the function f according to the parameter transfer relationship i Function call chain; The key in the first key-value pair table is the function f i , the value is the function f i Parameters involved; 13) Use static analysis technology to record the global variables contained in the code warehouse data, and track the state changes of each global variable by constructing a second key-value pair table; according to function f i The state changes of the global variables included further optimize the function f i Function call chain; the key in the second key-value pair table is function f i A global variable p in the function whose value is the state of the global variable p in other functions; 14) Based on the function call chains of each function obtained in step 13), a function dependency graph G corresponding to the code repository data is constructed as context information for the code repository data; each node in the function dependency graph G represents a function, and each edge represents a dependency relationship between functions; As context information for subsequent model training.

3. The method according to claim 2, characterized in that Through the global symbol table, the definition and call of the function in the specific code warehouse data are tracked, the interaction and dependency relationship between each function and other functions in the specific code warehouse data are extracted, and the function call chain of the corresponding function is constructed.

4. The method according to claim 2, characterized in that The method for training code-related deep learning models is: 21) Adding semantic descriptions to the functional dependency graph G of each piece of code repository data, and then inputting it into the deep learning model; The encoder of the deep learning model uses a self-attention mechanism to extract key contextual information from the functional dependency graph G and semantic description of each code repository data, and maps it to a high-dimensional vector space to generate a representation vector for the code repository data; 22) The decoder of the deep learning model generates the target code of the code warehouse data according to the representation vector of the code warehouse data; wherein the loss function used in training the deep learning model is Among them, logP θ (y t |x input ,y <t ) represents the generation of x input The probability of the tth Token, y <t is part of the generated code, x input is the formal representation of the functional dependency graph G, y t is the code generated based on the tth Token, n is x input The total number of tokens in.

5. The method according to claim 1, wherein In step 3), the method for the code generation model to generate the corresponding target code is: 31) The code generation model generates a plurality of different input data segments according to different contexts of function calls in the user input data, wherein different input data segments contain different context information; 32) generating a candidate code path based on each input fragment, and then screening each candidate code path using a set constraint mechanism; wherein different candidate code paths have different call parameters or dependencies; 33) Generate corresponding candidate codes based on each candidate code path retained by the screening; then merge the candidate codes to generate the target code.

6. The method according to claim 5, characterized in that The method for merging the candidate codes is: merging the codes corresponding to the same token in the candidate codes.

7. A system for code pre-training and generation for function call relationships, characterized in that: Context information extraction module, model training module and code generation module; The context information extraction module is configured to extract context information from each piece of code repository data in the code dataset; The context information includes function call information, parameter dependencies and global logical structure; The model training module is used to train a code-related deep learning model using the context information to obtain a code generation model; The code generation module is configured to generate corresponding target code based on the input code requirement description or the code to be edited using the code generation model; wherein, for the input code requirement description, the code generation model generates target code that meets the requirements based on the code requirement description; For the input code to be edited, the code generation model automatically repairs or optimizes the code to be edited to obtain the target code.

8. A server, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, the computer program is configured to be executed by the processor, and the computer program comprises instructions for executing the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images