A method and system for function call relationship-oriented code pre-training and generation

Abstract

The application discloses a function call relationship-oriented code pre-training and generation method and system. The method is as follows: 1) for each code repository data in the code data set, context information is extracted from the code repository data; the context information is function call information, parameter dependency and global logical structure; 2) a code-related deep learning model is trained by using the context information to obtain a code generation model; 3) a code requirement description or to-be-edited code is input into the code generation model to generate corresponding target code; wherein for the input code requirement description, the code generation model generates target code meeting the requirement according to the code requirement description; and for the input to-be-edited code, the code generation model automatically repairs or optimizes the to-be-edited code to obtain the target code. The application optimizes the logical consistency and availability of generated code, and significantly improves the code generation in a complex code scenario.

Description

Technical Field

[0001] This invention belongs to the field of information technology, specifically relating to a method and system for code pre-training and generation oriented towards function call relationships. Background Technology

[0002] With the rapid growth of software development demands and the continuous expansion of code size, code generation technology is becoming one of the important research directions in the field of software engineering. Code generation technology can help developers efficiently complete tasks such as code completion, code repair, and code refactoring, greatly improving the efficiency and quality of software development. In recent years, deep learning-based code generation technology has achieved significant breakthroughs, especially the emergence of large-scale pre-trained language models (LLMs), which has brought code generation capabilities to an unprecedented level. These models, trained on large-scale code and natural language data, can generate syntactically correct and logically sound code snippets. However, in practical applications, code generation technology still faces many challenges, particularly in terms of logical integrity and functional dependency awareness.

[0003] Currently, code generation technology based on large language models has become a research hotspot. Models such as GPT, CodeT5, and CodeGen, which model the context of code through the self-attention mechanism of the Transformer architecture, have demonstrated powerful capabilities in code completion, code repair, and code generation tasks. Model training typically consists of two phases: a pre-training phase and a fine-tuning phase.

[0004] In the pre-training phase, models typically learn the grammatical rules, semantic features, and contextual relationships of large-scale code and natural language data through autoregressive language modeling (e.g., predicting the next token) or fill-in-the-blank tasks (e.g., generating missing code snippets). This phase relies on massive amounts of training data and the expansion of model parameter size to improve generation performance. To adapt to specific task scenarios, the model is fine-tuned based on pre-training. For example, for code repair tasks, the fine-tuning dataset contains a large number of code errors and fix pairs; for specific languages ​​(e.g., Python or Java), code data in the corresponding language is used for optimization. The fine-tuning phase can significantly improve the model's performance on specific tasks or languages. Despite the significant progress made by the above techniques, their generation capabilities still have many shortcomings in complex code scenarios.

[0005] Code possesses inherent structure, particularly in its function-level call relationships, parameter dependencies, and shared state of global variables, which constitute a crucial part of code logic. However, most current large language models are trained using natural language processing techniques, primarily modeling code context through text sequences, and lack sufficient ability to model the structural characteristics of code. While this approach can generate syntactically correct code, it exhibits significant limitations in function dependency awareness and global logical consistency. For example, when generating code that calls a function, the model often fails to accurately understand the definition of the target function or the dependencies of the call parameters, resulting in incomplete or inaccurate generated code logic.

[0006] Furthermore, current large language models rely on massive amounts of code data for training, but obtaining high-quality code data is not unlimited. As the scale of data gradually increases, the marginal benefit of adding new data to improve model performance gradually diminishes. In addition, many models are trained in a rather "generalized" way, improving performance simply by increasing the amount of data and the size of parameters, without fully utilizing the unique structural information contained in the code (such as function call chains and dependency graphs). This approach leads to low data utilization efficiency and limits the model's ability to understand and generate complex code logic.

[0007] Current code generation relies on the natural language context of a single input sample for inference during the reasoning phase. This limitation is particularly pronounced in tasks involving function calls and parameter dependencies, mainly in the following aspects:

[0008] 1) Lack of global logical consistency constraints. When the target function is called in multiple contexts, existing methods often struggle to guarantee the logical consistency of the generated results across different scenarios. For example, the same function may require consistent functional logic in multiple calling scenarios, but existing generation methods only generate code based on a single context, leading to potential functional conflicts or logical inconsistencies in the generated results across different scenarios.

[0009] 2) Lack of global optimization capabilities through multi-sample collaboration. Existing methods often fail to comprehensively consider multiple related input samples when generating the objective function, lacking collaborative constraint mechanisms. For example, the objective function may be referenced by multiple callers, but existing generation methods mainly use natural language descriptions as context for generating the function, without considering the context generated by the call relationships. This fails to fully integrate information from other calling scenarios, resulting in incomplete logic in the generated objective function or failure to meet the calling requirements of other scenarios. This limitation directly affects the logical accuracy and functional adaptability of the generated code, especially in complex code scenarios.

[0010] In summary, existing methods for code generation primarily rely on generalized pre-training processes, focusing on learning from large-scale data or inferring and generating code based on single data points within a local context. However, these methods still face significant shortcomings in modeling logical consistency in code generation and the collaborative utilization of multi-sample information. These challenges result in low efficiency in utilizing high-quality data and a lack of effective modeling capabilities for complex dependencies between functions, making it difficult to meet the needs of code generation in complex scenarios. Therefore, as code generation tasks gradually evolve towards refinement and logicalization, how to deeply mine the structured characteristics of existing code data and fully utilize global context information during the generation process to improve the logical consistency and functional integrity of code generation has become a critical issue that urgently needs to be addressed in current technological development. Summary of the Invention

[0011] To address the limitations of existing large language models in code generation tasks, this invention provides a method and system for code pre-training and generation oriented towards function call relationships.

[0012] This invention acquires high-quality code data from real-world code repositories and extracts the function-level dependencies within it. This data naturally contains function call information, parameter dependencies, and global logical structures. These elements collectively constitute the context of the code snippets, providing rich contextual information and a more complete semantic understanding for model training. By using this contextual information (i.e., the naturally occurring function call information, parameter dependencies, and global logical structures in the data) for training, this invention designs a pre-training method based on function dependencies. This enables the model to effectively capture key information in the code, thereby improving the accuracy and logical consistency of the generated code. Key information refers to the most representative and valuable features of the contextual information for code generation tasks, such as function call patterns and parameter dependencies. By extracting this key information, the model can more accurately generate code that conforms to semantic logic. This technique can be applied independently to generation tasks, is suitable for various code generation scenarios, and does not require technical support from other modules.

[0013] Secondly, this invention proposes an independent inference strategy aimed at further optimizing contextual understanding during the generation process. In this strategy, the model can integrate one or more relevant contextual information to generate multiple candidate code paths when generating target code. Among these paths, the model selects the most suitable generation result through a built-in constraint mechanism, ensuring the consistency and integrity of the generated code across various contexts. This strategy can be used alone in conjunction with the aforementioned training method, or it can be implemented independently as needed to optimize specific generation scenarios, making the generated function logic more complete and accurate.

[0014] The technical solution of this invention is as follows:

[0015] A method for code pre-training and generation oriented towards function call relationships, comprising the following steps:

[0016] 1) For each piece of code repository data in the code dataset, extract context information from that piece of code repository data; the context information includes function call information, parameter dependencies, and global logical structure;

[0017] 2) Use the context information to train a code-related deep learning model to obtain a code generation model;

[0018] 3) Input the code requirement description or the code to be edited into the code generation model to generate the corresponding target code; wherein, for the input code requirement description, the code generation model generates target code that meets the requirements according to the code requirement description; for the input code to be edited, the code generation model automatically repairs or optimizes the code to be edited to obtain the target code.

[0019] Furthermore, a method for extracting the context information:

[0020] 11) Perform a full scan of the file structure in each code repository data, obtain each function and its call chain in the code repository data, and construct the dependency relationship of each function;

[0021] 12) For each function f in the code repository data... i The function f is extracted using an abstract syntax tree. i The parameters involved are recorded, and a first key-value pair table is created to track and record the function f. i The parameter passing relationships between each parameter and other functions are involved in the function f. i The parameter passing relationships between functions are considered as parameter dependencies; and the function f is optimized based on these parameter passing relationships. i The function call chain; where the key in the first key-value pair table is the function f. i The value is the function f i The parameters involved;

[0022] 13) Use static analysis techniques to record the global variables contained in the code repository data, and track the state changes of each global variable by constructing a second key-value table; according to function f i The state changes of the included global variables further optimize the function f. i The function call chain; where the key in the second key-value pair table is the function f. i A global variable p in one function, whose value is the state of the global variable p in other functions;

[0023] 14) Based on the function call chains of each function obtained in step 13), construct the function dependency graph G corresponding to the code repository data as the context information of the code repository data; each node in the function dependency graph G represents a function and each edge represents the dependency relationship between functions; as the context information for subsequent model training.

[0024] Furthermore, by tracing the definition and call of functions in the specific code repository data through the global symbol table, the interaction and dependency relationship between each function and other functions in the specific code repository data are extracted, and the function call chain of the corresponding function is constructed.

[0025] Furthermore, the method for training code-related deep learning models is as follows:

[0026] 21) Add a semantic description to the functional dependency graph G of each code repository data, and then input it into the deep learning model; the encoder of the deep learning model uses a self-attention mechanism to extract key context information from the functional dependency graph G and its semantic description of each code repository data, and maps it to a high-dimensional vector space to generate the representation vector of the code repository data.

[0027] 22) The decoder of the deep learning model generates the target code for the code repository data based on the representation vector of that code repository data; wherein the loss function used to train the deep learning model is... Wherein, logP θ (y t |x input ,y <t ) indicates generating x input The probability of the t-th token, y <t This is the generated portion of the code, x input For a formal representation of the functional dependency graph G, y t Let x be the code generated based on the t-th token, where n is x. input The total number of tokens in the system.

[0028] Furthermore, in step 3), the method by which the code generation model generates the corresponding target code is as follows:

[0029] 31) The code generation model generates multiple different input data fragments based on different contexts of function calls in the user input data, and the different input data fragments contain different context information;

[0030] 32) Generate a candidate code path for each input fragment, and then filter each candidate code path through a set constraint mechanism; where different candidate code paths have different calling parameters or dependencies;

[0031] 33) Generate corresponding candidate code based on each candidate code path that is selected and retained; then merge the candidate codes to generate the target code.

[0032] Furthermore, the method for merging the candidate codes is as follows: merge the codes corresponding to the same token in each candidate code.

[0033] A system for code pre-training and generation oriented towards function call relationships, characterized by a context information extraction module, a model training module, and a code generation module;

[0034] The context information extraction module is used to extract context information from each piece of code repository data in the code dataset; the context information includes function call information, parameter dependencies, and global logical structure.

[0035] The model training module is used to train a code-related deep learning model using the context information to obtain a code generation model;

[0036] The code generation module is used to generate corresponding target code based on the input code requirement description or the code to be edited using the code generation model; wherein, for the input code requirement description, the code generation model generates target code that meets the requirements; for the input code to be edited, the code generation model automatically repairs or optimizes the code to be edited to obtain the target code.

[0037] A server is characterized by comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the methods described above.

[0038] A computer-readable storage medium having a computer program stored thereon, characterized in that the computer program implements the above-described method when executed by a processor.

[0039] This invention has the following beneficial effects in optimizing the ability to generate code from large language models:

[0040] 1) By training with data that has functional dependencies, the model's performance on tasks such as code generation has been improved.

[0041] 2) By deeply mining function-level dependencies, the model's performance in complex code scenarios, including multiple function dependencies, function calls, and global logic consistency, has been significantly improved.

[0042] 3) Through a multi-sample collaborative reasoning strategy, the logical consistency and usability of the generated code are further optimized, and it can be used independently according to actual needs. Attached Figure Description

[0043] Figure 1 It is the basic process of code pre-training and generation methods and systems oriented towards function call relationships. Detailed Implementation

[0044] To make the above-mentioned objectives, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below through specific implementation examples.

[0045] This invention proposes a method for code pre-training and generation oriented towards function call relationships. By modeling function-level call relationships and parameter dependencies in the code, it can generate more accurate, logically consistent code that conforms to context requirements. Specifically, based on the importance of fully utilizing data and code pre-training models, this invention designs a system to automatically solve related problems of code data and models. The input and output of this system are described as follows:

[0046] On the input side, users provide raw code data, a description of code requirements, or code to be edited. The system processes this input data into a format suitable for training and extracts function-level dependency information by identifying call relationships and dependencies between functions. This information helps the model understand the interactions between functions when generating code, ensuring that the generated code can satisfy complex function dependencies and call relationships.

[0047] At the output end, the trained code generation model receives either a code requirement description or existing code as input. For code requirement descriptions, the model generates target code that meets the user's requirements; for existing code, the model analyzes it and automatically repairs or optimizes it. The model's output must not only meet the requirements of the local context but also ensure that the generated code conforms to global dependencies, avoiding the generation of inconsistent logic or incomplete functionality.

[0048] To improve the code generation capabilities of existing models, this invention: (1) firstly designs a pre-training data extraction method oriented towards function-level dependency awareness; (2) then trains a code-related deep learning model; (3) employs a multi-path generation strategy during the inference process; and (4) finally evaluates the generation results of the function-level dependency-aware code generation method and system. The specific performance details are as follows:

[0049] I. A Function-Level Dependency-Aware Pre-Training Data Extraction Method

[0050] 1.1 Data Acquisition

[0051] The quality of the data directly determines the effectiveness of model training. In this invention, to ensure that the code generation model performs better in real-world development scenarios, we extract a large amount of high-quality code datasets from multiple code data sources. Each data point in the code dataset represents a real-world code repository. The code in the repository contains rich function call information, modular structure, and various function dependencies, although the specific content may vary depending on the data source. Overall, the code dataset ensures the richness of this information to support model training.

[0052] Data sources may include, but are not limited to, publicly available code repositories, project code, or other accessible code data. Code elements in the dataset may include function definitions, function call information, parameter dependencies, function code snippets, and other information. Furthermore, to further ensure data diversity, we selected code repositories containing multiple programming languages ​​and technology stacks to ensure that the generated model exhibits good adaptability across diverse programming languages ​​and project architectures.

[0053] Data preprocessing includes cleaning, formatting, and standardizing the raw code data. Specifically, this invention ensures that the data can be better used for model training by removing irrelevant content, standardizing code structure and indentation style, and normalizing the code. The preprocessed data retains key information that helps capture the code's functionality and logic, while removing redundant and irrelevant parts. In this way, it ensures that the model can learn efficiently and generate code that conforms to syntax rules and functional requirements.

[0054] 1.2 Extraction of Contextual Information

[0055] This invention utilizes a large-scale dataset obtained from real-world code repositories, which contains natural function call dependencies. These dependencies require no additional manual annotation and can be extracted and modeled through automated analysis. This data-driven approach based on natural dependencies provides comprehensive contextual support for the model, enabling the generated code to accurately reflect complex function interactions and dependency structures.

[0056] In this invention, the extraction and utilization of function dependencies are crucial for achieving efficient code generation. We employ various techniques to analyze large amounts of code data, capturing the dependencies between functions and extracting function call chains. To ensure the model can understand complex function calls and their underlying dependencies, we cover direct function call dependencies, parameter passing dependencies, and cross-file and cross-module function call dependencies, forming a vast and complex dependency network. Specifically, the context information extraction process includes the following steps:

[0057] (1) Construction of Function Call Chains: In this stage, we conduct a comprehensive scan of the file structure of the entire project. Through various analysis techniques, we attempt to parse each function and its call chain in the code, and construct the dependency relationships of each function. In addition to the common extraction of function dependencies within a single file, real-world software projects often encounter complex situations, such as cross-file calls, cross-module calls, and project-level function dependencies. Therefore, to address these complex situations, this invention employs static analysis techniques of abstract syntax trees and symbol resolution for each specific code repository data in the dataset. By tracking the definition and calls of functions in the repository through a global symbol table, we can extract the interactions and dependencies between each function and other functions, and construct the function call chain. This enables the model to understand the interactions between different files when generating code, rather than being limited to function calls within the same file, ensuring that the model can comprehensively understand the multi-level dependencies in the code.

[0058] (2) Extraction of parameter dependencies: During function calls, there are often important dependencies between the input and output parameters of a function. To capture these dependencies, this invention performs a detailed analysis of the parameters of each function. Parameter dependencies are not limited to local dependencies within a function, but also include parameter passing across functions and modules. By extracting the parameters involved in each function contained in each code repository using abstract syntax tree technology and establishing a key-value pair table for tracking and recording, the parameter passing relationships (i.e., parameter dependencies) between functions are identified, thereby optimizing the constructed function call chain; whereby, for each function f i The function f is extracted using an abstract syntax tree. i The parameters involved are recorded, and a key-value pair table is created for tracking. The key in the key-value pair table is the function f. i The value is the function f i The parameters involved.

[0059] (3) Capturing Global State Dependencies: Besides function call chains and parameter dependencies, the state dependencies between functions are equally crucial. For example, some functions may modify global variables, while other functions depend on these global states for computation. To capture these dependencies, this invention uses static analysis techniques to record global variables in the code contained in each code repository, and tracks their state changes by constructing a key-value pair table. This further optimizes the constructed function call chain, thereby comprehensively reflecting the state dependencies between functions in the repository; where the key in the key-value pair table is the function f. i A global variable p is defined as the state of that global variable p in other functions.

[0060] Finally, to structurally represent complex function call relationships, this invention uses a graph structure to represent the function call chain of function dependencies contained in each code repository data. Each node in the graph represents a function, and each edge represents the dependency relationship between functions, thereby generating a corresponding function dependency graph for each code repository data, which clearly depicts the interrelationships between various functions in the code, serving as contextual information for subsequent model training.

[0061] Let the set of functions be F = {f1, f2, ..., f...} n The functional dependency graph is G = (F, E), where E is the set of edges between functions representing dependencies. For any pair of functions f... i ,f j ∈F, if f i f was called j Then add an edge e to graph G. i,j ∈E represents the function f i For function f j Dependence.

[0062] II. Training deep learning models related to code

[0063] In this invention, training a deep learning model with data containing function call dependencies is key to achieving efficient code generation. By pre-training on large-scale code data, the model can learn the syntax, semantics, and complex dependencies between functions, thus providing powerful capabilities for downstream code generation tasks. This invention designs a method that enables the model to generate code that meets requirements based on input contextual information (such as function call relationships, parameter passing, etc.).

[0064] The development of the entire system model can be viewed as a combination of the following parts: model structure design and pre-training task design. It is worth noting that using code to pre-train the model is one implementation method corresponding to the function call relationship proposed in this invention.

[0065] 2.1 Deep learning model structure related to code

[0066] The code-related deep learning model design aspects involved in this invention are applicable to popular Transformer-based Decoder and Encoder-Decoder frameworks. Here, we take a deep learning model with 600 million parameters in CodeT5 as an example to illustrate how this invention can continue pre-training on the model's weights. This allows the model to benefit from the knowledge and representations learned by CodeT5, which is particularly advantageous in code-related tasks.

[0067] During the pre-training phase, function dependency paths (i.e., function call chains) are extracted from the function dependency graph G, and the model's input is constructed based on these paths. The input can include structured information consisting solely of dependency paths, or it can incorporate additional semantic descriptions to enrich the model's understanding of function dependencies. During training, the model learns these dependencies and their semantic relationships to accurately generate target code during the inference phase.

[0068] In this structure, the model receives input data rich in contextual dependencies through the encoder. The model utilizes a self-attention mechanism to deeply extract key contextual information and maps it to a high-dimensional vector space to generate semantically rich representation vectors. Each token (the smallest input unit formed after input segmentation, serving as the basic carrier for the larger model to understand and generate code, such as code snippets, function names, and natural language) is converted into a vector, allowing this representation vector to simultaneously encode its semantic features and their relationships within the context. In this way, the encoder provides rich information for the subsequent decoding and generation process, helping the model understand how to generate the complete implementation of the function based on the context.

[0069] The decoder's task is to generate the desired code for x based on the representation vector generated by the encoder. The model progressively generates code based on the input requirement description and key contextual information of the function call, ensuring that the generated code meets the functional requirements of x. The decoder uses an autoregressive approach during the generation process, generating a new token each time and using it as input for the next step until the complete function code is generated.

[0070] 2.2 Design of Pre-training Tasks

[0071] To enable the model to efficiently handle context dependencies, this invention designs a context-based code generation task. The goal of the task is to help the model understand how to generate accurate code across different contexts. Through self-supervised learning, the model can learn complex contextual relationships from large-scale training data and apply them to the code generation task.

[0072] The formal description of the input received by the model proposed in this invention is as follows:

[0073] x input =x1,x2,…,x m x m This refers to the m-th token in the context information.

[0074] Accordingly, the output can be formally described as follows, where y i This represents the token of the i-th output.

[0075] y = y1, y2, ..., y n

[0076] It is worth noting that, depending on the specific requirements of different tasks, the input data can include natural language description text or function header fragments (such as code generation tasks), or it can be existing code that needs to be modified (such as code enhancement tasks). Regardless of the task, the method of this invention can effectively utilize contextual dependencies for training and generation, thereby generating high-quality code that meets the requirements.

[0077] Formally, the loss function for this task can be defined as:

[0078]

[0079] Where logP θ (y t |x input ,y <t ) indicates generating x input The probability of the t-th token, y <t This is a portion of the code that has already been generated.

[0080] III. Employing a multi-path generation strategy in the reasoning process

[0081] During the inference phase of code generation, the model needs to generate the implementation of the target code based on user input. To further improve the accuracy, logical consistency, and functional completeness of the generated code, this invention proposes a multi-path generation strategy. This strategy allows the model to utilize input information from multiple different contexts to generate multiple candidate code paths and select the best generation result from these paths, thereby improving the quality of the generated code.

[0082] 3.1 The core idea of ​​multi-path generation strategy

[0083] Unlike traditional code generation methods that rely on a single input context, generated code is influenced not only by the current context but also by the upstream and downstream relationships of function calls and function dependency chains. Therefore, the multi-path generation strategy proposed in this invention considers information from multiple contexts to a certain extent, enhancing the flexibility and accuracy of generation by generating multiple candidate paths. However, it is worth noting that this strategy is not a necessary part of this invention; it is only used to further optimize the code generation process and is not a core technology.

[0084] The core idea of ​​the multi-path generation strategy is that, given a requirement description or existing code snippet, the model will generate multiple candidate code paths from various contextual information. Each path may reason based on different contexts and functional dependencies, thus generating different code snippets. In the final generated result, the model will filter these candidate paths according to constraints (such as functional dependencies, code logic consistency, etc.) and select the code that best meets the user's requirements.

[0085] 3.2 Implementation steps of the multi-path generation strategy

[0086] The implementation of multi-path generation strategies can be divided into the following technical aspects:

[0087] (1) Diversified processing of input data: During the inference process, the input data (such as requirement descriptions or existing code) may be incomplete or contain different contextual information. Therefore, the system can selectively diversify the input data in different ways. For example, for the generation of a target function, the model can generate multiple different input data fragments based on the different contexts of function calls contained in the requirement description or existing code (such as the parameter types of the calling function, the function logic of the caller, etc.). These input data fragments will contain different contextual information, and the model will generate a candidate code path based on each input fragment. These paths may differ, and the generated code implementation may also differ based on different call parameters or dependencies.

[0088] (2) Candidate Path Generation: Through an autoregressive decoder, the model generates multiple candidate paths step by step based on the input context. Each candidate path is generated independently, depending on its corresponding context information. The model generates the corresponding target function code based on each input fragment (such as function requirement description, function call context, etc.), and each path generates a different code implementation. Each generated path represents a reasoning method of the model for the target function. These candidate paths may generate different code based on different parameter configurations, function call relationships, or function body implementation methods.

[0089] (3) Constraint Mechanisms and Path Selection: The model selects from multiple candidate paths through constraint mechanisms. Constraint mechanisms include, but are not limited to, consistency of functional dependencies, consistency of code logic, and degree of matching with user requirements.

[0090] In the multi-path generation strategy, for each generated token, the model merges the results of multiple candidate paths. Let the generated candidate paths be P1, P2, ..., P... k The logits of each path when generating the t-th token are represented as logit. t(P1),logit t (P2),…,logit t (P k ), where k is the number of candidate paths.

[0091] By constraining and filtering the logits of these paths, the final result of synthesizing the t-th token is obtained. This result represents the optimal prediction after merging all candidate paths. The merging operation can employ various strategies, but the specific operation can be uniformly represented as follows:

[0092]

[0093] Here, `combine` represents the operation performed on the logits of multiple paths. The final generated code is the result of the model comprehensively evaluating all candidate paths.

[0094] 3.4 Advantages of Multi-Path Generation Strategy

[0095] By introducing a multi-path generation strategy, the generation results of this invention can be further improved in terms of diversity and accuracy. Through this strategy, the model can generate code from different perspectives when dealing with complex contextual dependencies by considering multiple contextual information, thereby improving the diversity and coverage of the generated results. By fully considering the complex dependencies and call chains between functions, the generated code can correctly handle multi-level dependencies, providing more choices and increasing the flexibility of the generated results.

[0096] The present invention also provides a system for code pre-training and generation oriented towards function call relationships, characterized by a context information extraction module, a model training module, and a code generation module;

[0097] The context information extraction module is used to extract context information from each piece of code repository data in the code dataset; the context information includes function call information, parameter dependencies, and global logical structure.

[0098] The model training module is used to train a code-related deep learning model using the context information to obtain a code generation model;

[0099] The code generation module is used to generate corresponding target code based on the input code requirement description or the code to be edited using the code generation model; wherein, for the input code requirement description, the code generation model generates target code that meets the requirements; for the input code to be edited, the code generation model automatically repairs or optimizes the code to be edited to obtain the target code.

[0100] The present invention also provides a server, characterized in that it includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the above-described methods.

[0101] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program implements the above-described method when executed by a processor.

[0102] IV. Evaluation of the Technical Effects of the Invention

[0103] Finally, to comprehensively evaluate the functionality of this invention, we conducted detailed experiments on multiple datasets, primarily including tests on the pre-training task dataset of this invention and task tests based on the CodeEval dataset. These tests aimed to verify the performance of this invention in code generation tasks, covering application scenarios of both pre-training tasks and real-world programming tasks.

[0104] In the pre-training task, this invention utilizes the pre-trained model and combines it with an inference strategy for testing, achieving significant results. In this stage, we primarily used BLEU, Exact Match (EM), and CodeBLEU as evaluation metrics to comprehensively measure the quality of the code generated by the model. Experimental results show that compared to the baseline model (CodeT5-small) which was not pre-trained and only fine-tuned, this invention achieves a BLEU score of 30.25 on this dataset, a significant improvement over the baseline model's 16.83. Similarly, in the Exact Match (EM) and CodeBLEU evaluations, this invention's model also demonstrates high scores, reaching 23.96 for EM and 54.15 for CodeBLEU, both far exceeding the baseline model's 3.54 and 3.54 respectively.

[0105] The CodeEval dataset contains 230 functions, each with multiple test cases to evaluate whether the generated code is functionally consistent with the target function. In experiments based on this dataset, the model of this invention was tested for functional consistency using the generated functions, verifying the practicality and accuracy of the generated code. Test results show that the present invention can generate high-quality code and exhibits strong functional consistency across multiple test cases. Specifically, the model's pass@1 metric reaches 10.87%. Although the model of this invention has not yet completely surpassed the benchmark model in terms of generation accuracy on pass@1, especially compared with GPT-4O, it still shows good potential in handling multi-path generation strategies and complex dependencies.

[0106] Practical application scenarios of this invention:

[0107] As software development becomes increasingly complex, developers face more and more challenges. Code generation models have effectively improved the efficiency and quality of code generation, especially in tasks such as function generation and code fixing, reducing human intervention and increasing development efficiency. However, as the scale of model parameters continues to expand, the demand for training data and computing resources also increases. The method of this invention can utilize existing data more efficiently, improving model training performance and thus meeting the ever-growing needs of software development.

[0108] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.

Claims

1. A method for code pre-training and generation oriented towards function call relationships, comprising the following steps: 1) For each piece of code repository data in the code dataset, extract context information from that piece of code repository data; The context information includes function call information, parameter dependencies, and global logical structure. 2) Use the context information to train a code-related deep learning model to obtain a code generation model; 3) Input the code requirement description or the code to be edited into the code generation model to generate the corresponding target code; wherein, for the input code requirement description, the code generation model generates target code that meets the requirements according to the code requirement description; for the input code to be edited, the code generation model automatically repairs or optimizes the code to be edited to obtain the target code; Method for extracting the context information: 11) Perform a full scan of the file structure in each code repository data, obtain every function and its call chain in that code repository data, and construct the dependency relationship of each function; 12) For each function in the code repository data... Extract the function using an abstract syntax tree. The parameters involved are recorded, and a first key-value table is created to track and record the function. The parameter passing relationships between each parameter and other functions are involved, thus defining the function. The parameter passing relationships between functions are considered parameter dependencies; and the function is optimized based on these parameter passing relationships. The function call chain; where the key in the first key-value pair table is the function. The value is the function. The parameters involved; 13) Use static analysis techniques to record the global variables contained in the code repository data, and track the state changes of each global variable by constructing a second key-value table; based on the function... The state changes of the included global variables further optimize the function. The function call chain; where the key in the second key-value pair table is the function. The value of the global variable p is the state of the global variable p in other functions; 14) Based on the function call chains of each function obtained in step 13), construct the function dependency graph G corresponding to the code repository data as the context information of the code repository data; each node in the function dependency graph G represents a function and each edge represents the dependency relationship between functions; as the context information for subsequent model training.

2. The method according to claim 1, characterized in that, By tracing the definition and call of functions in this code repository data through the global symbol table, the interaction and dependency relationship between each function and other functions in this code repository data are extracted, and the function call chain of the corresponding function is constructed.

3. The method according to claim 1, characterized in that, The method for training deep learning models related to code is as follows: 21) Add a semantic description to the functional dependency graph G of each code repository data, and then input it into the deep learning model; the encoder of the deep learning model uses a self-attention mechanism to extract key context information from the functional dependency graph G and its semantic description of each code repository data, and maps it to a high-dimensional vector space to generate the representation vector of the code repository data. 22) The decoder of the deep learning model generates the target code for the code repository data based on the representation vector of that code repository data; wherein the loss function used to train the deep learning model is... ;in, Indicates generation The The probability of each token. This is the generated portion of the code. Formal representation of the functional dependency graph G According to the first The code that generates each Token, where n is... The total number of tokens in the system.

4. The method according to claim 1, characterized in that, In step 3), the method by which the code generation model generates the corresponding target code is as follows: 31) The code generation model generates multiple different input data fragments based on different contexts of function calls in the user input data, and the different input data fragments contain different context information; 32) Generate a candidate code path for each input fragment, and then filter each candidate code path through a set constraint mechanism; where different candidate code paths have different calling parameters or dependencies; 33) Generate corresponding candidate code based on each candidate code path that is selected and retained; then merge the candidate codes to generate the target code.

5. The method according to claim 4, characterized in that, The method for merging candidate codes is to merge the code corresponding to the same token in each candidate code.

6. A system for code pre-training and generation oriented towards function call relationships, characterized in that, Context information extraction module, model training module, and code generation module; The context information extraction module is used to extract context information from each piece of code repository data in the code dataset; the context information includes function call information, parameter dependencies, and global logical structure; wherein, the method for extracting the context information is as follows: 11) Perform a full scan of the file structure in each code repository data, obtain every function and its call chain in that code repository data, and construct the dependency relationship of each function; 12) For each function in the code repository data... Extract the function using an abstract syntax tree. The parameters involved are recorded, and a first key-value table is created to track and record the function. The parameter passing relationships between each parameter and other functions are involved, thus defining the function. The parameter passing relationships between functions are considered parameter dependencies; and the function is optimized based on these parameter passing relationships. The function call chain; where the key in the first key-value pair table is the function. The value is the function. The parameters involved; 13) Use static analysis techniques to record the global variables contained in the code repository data, and track the state changes of each global variable by constructing a second key-value table; based on the function... The state changes of the included global variables further optimize the function. The function call chain; where the key in the second key-value pair table is the function. The value of the global variable p is the state of the global variable p in other functions; 14) Based on the function call chains of each function obtained in step 13), construct the function dependency graph G corresponding to the code repository data as the context information of the code repository data; each node in the function dependency graph G represents a function, and each edge represents the dependency relationship between functions; as the context information for subsequent model training; The model training module is used to train a code-related deep learning model using the context information to obtain a code generation model; The code generation module is used to generate corresponding target code based on the input code requirement description or the code to be edited using the code generation model; wherein, for the input code requirement description, the code generation model generates target code that meets the requirements; for the input code to be edited, the code generation model automatically repairs or optimizes the code to be edited to obtain the target code.

7. A server, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images