A code processing method, apparatus, device, storage medium, and program product

Abstract

Embodiments of the present application disclose a code processing method, device and equipment, a storage medium and a program product. The code processing method comprises: in the case of receiving a code splitting instruction, obtaining original code from the code splitting instruction; splitting the original code based on an abstract syntax tree to obtain a code syntax tree; determining a plurality of code description information corresponding to a plurality of codes in the code syntax tree; the plurality of code description information comprises a plurality of code structure information and a plurality of code function information; and performing adaptive code splitting processing on the plurality of codes based on the plurality of code description information to obtain a plurality of split codes. Through the present application, the accuracy of code generation can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a code processing method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] In the process of code retrieval generation, retrieval-augmented generation (RAG) technology can be used to retrieve information highly relevant to an input query from a relevant code library or knowledge base. The information usually relates to specific code snippets, function definitions, document descriptions and other code-related knowledge. Then, the retrieved code-related information and the input query are used as conditions to guide a large model to generate more accurate and highly relevant code retrieval task outputs.

[0003] In related technologies, code editors and integrated development environments (IDEs) provide basic code structure recognition functions. However, they usually recall too many code snippets or inaccurate code snippets in the subsequent process of generating code using a large model. This amplifies the hallucinations of the model, resulting in the generation of code that does not meet user needs, i.e., reducing the accuracy of code generation. SUMMARY

[0004] To solve the above technical problems, the embodiments of the present application aim to provide a code processing method, device, equipment, storage medium and program product, which can improve the accuracy of code generation.

[0005] The technical solution of the present application is implemented as follows:

[0006] The embodiments of the present application provide a code processing method, which comprises the following steps:

[0007] In the case of receiving a code segmentation instruction, the original code is obtained from the code segmentation instruction;

[0008] The original code is segmented based on an abstract syntax tree to obtain a code syntax tree;

[0009] A plurality of code description information corresponding to a plurality of code segments in the code syntax tree is determined; the plurality of code description information includes a plurality of code structure information and a plurality of code function information;

[0010] The plurality of code segments are adaptively segmented based on the plurality of code description information to obtain a plurality of segmented codes.

[0011] The embodiments of the present application provide a code processing device, which comprises:

[0012] The acquisition unit is used to acquire the original code from the code splitting instruction upon receiving the code splitting instruction;

[0013] A segmentation unit is used to segment the original code based on an abstract syntax tree to obtain a code syntax tree;

[0014] A determining unit is used to determine multiple code description information corresponding to multiple code segments in the code syntax tree; the multiple code description information includes multiple code structure information and multiple code function information.

[0015] The processing unit is used to perform adaptive code segmentation processing on the multiple code segments based on the multiple code description information to obtain multiple segmented codes.

[0016] This application provides a code processing device, the code processing device comprising:

[0017] The system includes a memory, a processor, and a communication bus. The memory communicates with the processor via the communication bus. The memory stores a program that the processor can execute to process code. When the program is executed, the processor performs the code processing method described above.

[0018] This application provides a storage medium storing a computer program for use in a code processing device, characterized in that the computer program, when executed by a processor, implements the code processing method described above.

[0019] This application also provides a computer program product, including a computer program that can be executed by a processor in a code processing device to complete the steps described in the aforementioned code processing method.

[0020] This application provides a code processing method, apparatus, device, storage medium, and program product. The code processing method includes: upon receiving a code segmentation instruction, obtaining the original code from the code segmentation instruction; segmenting the original code based on an abstract syntax tree to obtain a code syntax tree; determining multiple code description information corresponding to multiple code segments in the code syntax tree; the multiple code description information includes multiple code structure information and multiple code function information; and performing adaptive code segmentation processing on the multiple code segments based on the multiple code description information to obtain multiple segmented codes. The above-described solution utilizes an abstract syntax tree to coarsely segment the original code, obtaining a code syntax tree. Based on this code syntax tree, multiple code descriptions are determined for multiple code segments. These descriptions include multiple code result information and multiple code function information corresponding to the multiple code segments. This allows for detailed segmentation of the multiple code segments based on the code structure and function information, resulting in multiple segmented codes that correspond to the code structure information and possess code functions. Furthermore, when retrieving code from these segmented codes for generation, the system can retrieve and process code that matches the code generation requirements based on the code structure and function information, thus improving the accuracy of code generation. Attached Figure Description

[0021] Figure 1 A schematic diagram of an existing RAG code generation technology method provided in this application embodiment;

[0022] Figure 2 A flowchart of a code processing method provided in an embodiment of this application;

[0023] Figure 3 This application provides an exemplary adaptive code splitting process diagram;

[0024] Figure 4 This application provides an exemplary adaptive code splitting process diagram.

[0025] Figure 5 This application provides an exemplary adaptive code retrieval process diagram;

[0026] Figure 6 This application provides an exemplary adaptive requirement rewriting process diagram.

[0027] Figure 7 This application provides an exemplary retrieval process diagram;

[0028] Figure 8 This application provides an exemplary adaptive code judgment and generation process diagram;

[0029] Figure 9 This application provides an exemplary adaptive code generation process diagram;

[0030] Figure 10 This is a schematic diagram of the composition structure of a code processing device provided in an embodiment of this application;

[0031] Figure 11 This is a schematic diagram of the composition structure of a code processing device provided in an embodiment of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0033] RAG (Retrieval-Augmented Generation) is a natural language processing technique that combines retrieval and generation. It enhances the model's ability to understand context and the richness of generated content by introducing external knowledge sources into traditional sequence generation models.

[0034] In code retrieval and generation, such as Figure 1 As shown, RAG technology first retrieves information highly relevant to the input query from relevant codebases or knowledge bases. This information typically involves specific code snippets, function definitions, documentation, and other code-related knowledge. Then, this retrieved code-related information is used together with the input query as conditions to guide the large model in generating more accurate output that is highly relevant to the code retrieval task. In this way, RAG effectively solves some problems existing in traditional code retrieval models, such as insufficient understanding of fuzzy queries and inaccurate retrieval due to lack of contextual information, greatly improving the efficiency and quality of code retrieval generation.

[0035] Currently, the main technical solutions for combining code scenarios with RAG technology are derived and developed following the technical path of "data-slicing-vectorized storage-retrieval-query concatenation-large model generation".

[0036] For example, patent CN117743559B employs a unique two-stage splitting method to handle users' original questions in data slicing scenarios. The first stage involves retrieval, and the second stage involves concatenation and enhancement. This approach effectively improves the multi-turn dialogue capabilities of the language model. Applied to programming scenarios, it can better understand and respond to developers' complex questions and needs. Patent CN118113806A proposes a novel knowledge candidate retrieval strategy. This strategy helps the language model mitigate the illusion problem and can clarify the context of complex events. In large-scale automated programming, this means that the necessary knowledge and information can be provided to developers more accurately, avoiding the generation of erroneous or irrelevant code. Patent CN117891838B innovatively updates the retrieval method. In addition to traditional vector retrieval methods, it optimizes the retrieval process by constructing two indexes and training a cross-encoder, significantly improving retrieval accuracy. When combined with programming, it ensures that the retrieved code snippets or knowledge are more accurate and relevant, improving the quality of code generation.

[0037] Patent CN118095435A innovates from the perspective of large language model reasoning. Through step-by-step reasoning, expert example feedback, and the implantation of external memory, it enhances the language model's decision-making and rational reasoning capabilities in the RAG reasoning domain. Based on the programming domain, this will result in more rational and efficient code, better adaptable to different programming tasks and scenarios.

[0038] However, the above general solutions cannot perfectly solve the problem of personalized code generation. This is because the mapping from natural language input to code is often distorted, resulting in the retrieved code snippets not matching the user's actual needs. Furthermore, due to the timeliness of the pre-training corpus and issues such as generation illusion, general code models will further accumulate generation errors, leading to a significant difference between the final code generated by the large model and the user's actual needs, thus reducing the accuracy of code generation.

[0039] The following implementation scheme can be used to address the problems existing in the prior art.

[0040] This application provides a code processing method, which is applied to a code processing device. Figure 2 A flowchart of a code processing method provided in this application embodiment is shown below. Figure 2 As shown, code processing methods may include:

[0041] S101. Upon receiving a code splitting instruction, obtain the original code from the code splitting instruction.

[0042] The code processing method provided in this application embodiment is applicable to scenarios where original code needs to be segmented.

[0043] In the embodiments of this application, the code processing apparatus can be implemented in various forms. For example, the code processing apparatus described in this application may include a core network, a server, a terminal, or other devices, and the specific embodiments of this application do not limit this.

[0044] In this embodiment, the code splitting instruction can be an instruction input by the user into the code processing device, or an instruction transmitted to the code processing device by other devices. The specific way in which the code processing device receives the code splitting instruction can be determined according to the actual situation, and this embodiment does not limit this.

[0045] In this embodiment, the original code is the code carried in the code splitting instruction, or it can be the code stored in a certain area indicated in the code splitting instruction. The specific code can be determined according to the actual situation, and this embodiment does not limit it.

[0046] It should be noted that the original code can be historically written code or code stored in the code processing device. The specific original code can be determined according to the actual situation, and this application embodiment does not limit it.

[0047] It should also be noted that the original code can be JAVA code, Pathon code, or C language code, or other programming languages. The specific language of the original code can be determined according to the actual situation, and this application embodiment does not limit it in this regard.

[0048] S102. The original code is segmented based on the abstract syntax tree to obtain the code syntax tree.

[0049] In this embodiment of the application, after the code processing device obtains the original code from the code splitting instruction, it splits the original code based on the abstract syntax tree to obtain the code syntax tree.

[0050] In this embodiment, the Abstract Syntax Tree (AST) can be a segmenter configured in the code processing device, a segmenter transmitted from other devices to the code processing device, or a segmenter obtained by the code processing device through other means. The specific way the code processing device obtains the AST can be determined according to the actual situation, and this embodiment does not limit it.

[0051] It should be noted that once the code processing device obtains the original code, it inputs the original code into the AST code splitter (Abstract Syntax Tree). The AST code splitter performs the syntax tree parsing step, and through in-depth analysis of the original code, it extracts rich code information from the original code, providing a basic understanding of the code structure for subsequent processing.

[0052] For example, the AST code splitter can use AST representation, the original code can be represented by the character D, and the code syntax tree can be represented by the character C. The process of splitting the original code based on the abstract syntax tree to obtain the code syntax tree is shown in formula (1):

[0053] (1)

[0054] It should be noted that C in formula (1) can also be understood as the structured code corpus data (i.e., code syntax tree) parsed by AST.

[0055] S103. Determine multiple code description information corresponding to multiple code segments in the code syntax tree; the multiple code description information includes multiple code structure information and multiple code function information.

[0056] In this embodiment of the application, the code processing device segments the original code based on the abstract syntax tree. After obtaining the code syntax tree, it determines the multiple code description information corresponding to the multiple code segments in the code syntax tree.

[0057] In this embodiment, the code processing device further includes an adaptive code segmenter, which can determine multiple code descriptions corresponding to multiple code segments in the code syntax tree. Specifically, the adaptive code segmenter learns the parsed code syntax tree, and through this learning process, understands the overall abstract structure of the code data to be segmented and the functions of each module. Then, the adaptive segmentation agent determines the structure (i.e., code structure information) and semantic category (i.e., code function information) of each code segment through semantic analysis, and generates a corresponding natural language description for that code segment.

[0058] It should be noted that multiple code segments correspond one-to-one with multiple code description information, that is, one code segment corresponds to one code description information.

[0059] It should be noted that the code processing device is equipped with a large model, which can be used to implement the function of an adaptive code splitter.

[0060] In this embodiment of the application, the multiple code description information includes multiple code structure information and multiple code function information for multiple code segments. The multiple code structure information includes the granularity of the code segmentation, and the multiple code function information includes the functions that the multiple code segments can perform.

[0061] It should be noted that the granularity of segmentation includes granularity such as row, class or function, as well as personalized granularity such as block, multi-function, etc. The specific granularity of segmentation can be determined according to the actual situation, and the embodiments of this application do not limit it.

[0062] In this embodiment, the adaptive code segmenter can learn a code syntax tree and understand multiple code description information (the segmentation granularity of multiple code segments and the functions that multiple code segments can achieve) corresponding to multiple code segments through the process of learning the code syntax tree. Through semantic analysis, it determines the code description information of each code segment in the multiple code segments and generates corresponding natural language description information for that code segment. Specifically, it can generate multiple natural language description information corresponding to multiple code segments based on multiple code description information.

[0063] For example, multiple code structure information and multiple code function information can be summarized to obtain multiple natural language description information.

[0064] S104. Based on multiple code description information, perform adaptive code segmentation on multiple code segments to obtain multiple segmented code.

[0065] In this embodiment of the application, after the code processing device determines the multiple code description information corresponding to multiple code segments in the code syntax tree, it performs adaptive code segmentation processing on the multiple code segments based on the multiple code description information to obtain multiple segmented codes.

[0066] In this embodiment of the application, the code processing device is equipped with a large model, which can be used to perform adaptive code segmentation processing on multiple code segments based on multiple code description information to obtain multiple segmented codes.

[0067] For example, the functionality of an adaptive code splitter implemented using a large model can be utilized to adaptively split multiple code segments based on multiple code description information, resulting in multiple split code segments.

[0068] In this embodiment of the application, the number of multiple code segments is greater than or equal to the number of multiple code segments.

[0069] In this embodiment of the application, the process of the code processing device performing adaptive code segmentation on multiple code segments based on multiple code description information to obtain multiple segmented codes includes: obtaining code segmentation requirements from code segmentation instructions; determining whether multiple code segments need to be further segmented based on the target segmentation granularity in the code segmentation requirements and the segmentation granularity in multiple code description information; and performing adaptive code segmentation on multiple code segments to obtain multiple segmented codes if it is determined that multiple code segments need to be further segmented.

[0070] In this embodiment of the application, the code splitting instruction also includes a code splitting requirement, which includes information requesting the code processing device to split the code, specifically including the target splitting granularity of the code.

[0071] In this embodiment, the target segmentation granularity in the code segmentation requirement can be compared with the segmentation granularity in multiple code description information. If the target segmentation granularity in the code segmentation requirement is the same as the segmentation granularity in multiple code description information, it is determined that multiple code segments do not need to be further segmented; if the target segmentation granularity in the code segmentation requirement is different from the segmentation granularity in multiple code description information, it is determined that multiple code segments need to be further segmented.

[0072] In this embodiment of the application, if the segmentation granularity of the first code description information in the multiple code description information is different from the target segmentation granularity in the code segmentation requirement, then the first code segment corresponding to the first code description information is determined in the multiple code segments, and then the first code segment is subjected to adaptive code segmentation processing to obtain the segmented first code segment. The remaining code in the multiple code segments excluding the first code segment and the segmented first code segment are taken as the multiple segmented codes.

[0073] It should be noted that the first code description information can be any one of multiple code description information.

[0074] It should be noted that the first code segment is the code that corresponds to the first code description information among multiple code segments.

[0075] In this embodiment, if it is determined that multiple code segments need to be segmented based on the target segmentation granularity in the code segmentation requirements and the segmentation granularity in multiple code description information, then adaptive code segmentation processing is performed on the multiple code segments to obtain multiple segmented codes. If it is determined that a portion of the code in the multiple code segments needs to be segmented based on the target segmentation granularity in the code segmentation requirements and the segmentation granularity in multiple code description information (such as segmenting the first code segment in the multiple code segments), then adaptive code segmentation processing is performed on the first code segment to obtain the segmented first code segment. The remaining code in the multiple code segments excluding the first code segment and the segmented first code segment are taken as the multiple segmented codes.

[0076] In this embodiment of the application, the process of adaptive code segmentation for multiple code segments to obtain multiple segmented code segments can be determined according to the actual situation, and this embodiment of the application does not limit this.

[0077] It should be noted that if multiple segmentation codes are obtained, all segmentation codes will be stored in the retrieval database.

[0078] In this embodiment, the adaptive code segmenter splits the code that needs to be segmented (which can be determined based on the target segmentation granularity in the code segmentation requirements and the segmentation granularity of multiple code segments or multiple segmentation data) and stores it in the retrieval database. It performs the segmentation operation according to the target segmentation granularity in the code segmentation requirements (i.e., preset categories for line, class, or function granularity, or block, multi-function, and other personalized granularities). Simultaneously, the adaptive segmentation intelligence performs semantic-level decomposition of the code data based on semantic integrity, giving the same description to code with the same semantics (i.e., code with the same function has the same description), and stores the code data in the corresponding retrieval database according to its segmentation category and semantics.

[0079] In this embodiment of the application, the process of the code processing device performing adaptive code segmentation on multiple code segments to obtain multiple segmented codes includes: continuing to perform adaptive code segmentation on the multiple code segments to obtain multiple segmented data; if the segmentation granularity corresponding to the multiple segmented data is the same as the target segmentation granularity in the code segmentation requirement, using the multiple segmented data as multiple segmented codes; if the segmentation granularity corresponding to the multiple segmented data is different from the target segmentation granularity in the code segmentation requirement, continuing to perform adaptive segmentation on the multiple segmented data until the segmentation granularity of the segmented data is the same as the target segmentation granularity in the code segmentation requirement, thus obtaining multiple segmented codes.

[0080] In this embodiment, multiple code segments can first undergo adaptive code segmentation. The granularity of the code after this adaptive code segmentation is then compared with the target segmentation granularity in the code segmentation requirements. If the granularities are the same, the adaptive code segmentation process is discontinued, and the code after this adaptive code segmentation is used as multiple segmented codes. If the granularities are different, adaptive code segmentation continues until the segmentation granularity of the resulting data matches the target segmentation granularity in the code segmentation requirements, resulting in multiple segmented codes.

[0081] In this embodiment, when the granularity of multiple data segments differs from the target granularity in the code segmentation requirement, the adaptive code segmenter determines that the multiple data segments are incompletely segmented. Then, it continues to segment the incompletely segmented data, recursively iterating and segmenting according to the hierarchy and semantic structure, continuing this process until the smallest granularity is reached (which is the target granularity in the code segmentation requirement).

[0082] Understandably, this application can automatically and interactively decompose the original code at multiple fine-grained and semantic levels, preserving the functional and structural information of the original code to the greatest extent, and more accurately locating and retrieving code fragments related to user needs, thereby improving the efficiency and accuracy of code processing.

[0083] For example, such as Figure 3 As shown: Upon receiving a code splitting instruction, the code processing device obtains the original code from the instruction and then splits it based on an Abstract Syntax Tree (AST) code splitter to obtain a code syntax tree. Next, an adaptive code splitter determines multiple code descriptions corresponding to multiple code segments in the code syntax tree and obtains the code splitting requirements from the instruction. Based on the target splitting granularity in the code splitting requirements and the splitting granularity in the multiple code descriptions, it determines whether the multiple code segments need further splitting. If further splitting is required, adaptive code splitting is performed on the multiple code segments to obtain multiple split data. If the splitting granularity of the multiple split data segments is the same as the target splitting granularity in the code splitting requirements, these multiple split data segments are used as multiple split codes. If the splitting granularity of the multiple split data segments is different from the target splitting granularity in the code splitting requirements, adaptive splitting is performed on the multiple split data segments until the splitting granularity of the resulting data segments is the same as the target splitting granularity in the code splitting requirements, resulting in multiple split codes. The code for multiple data segmentation includes code for class-level granular data, code for function-level granular data, code for row-level granular data, etc.

[0084] In the field of programming, code organization and maintenance are crucial. Existing code editors and IDEs offer basic code structure recognition capabilities, but they typically cannot intelligently segment code based on complex semantics and context. Furthermore, during subsequent Large Language Model (LLM) responses, they often recall excessive or inaccurate code snippets, failing to improve LLM's code generation capabilities and instead amplifying the illusion of the model (i.e., the large language model), resulting in code that does not meet user needs. This problem can be addressed using the adaptive code segmenter described in this application. The adaptive code segmenter is a tool that utilizes LLM to understand and process code, enabling adaptive, multi-level, and multi-granular precise segmentation of code data based on input code data. To achieve intelligent judgment, this application uses manually constructed segmentation corpus data and professional programming feedback information to fine-tune the general code large model and train it using Reinforcement Learning from Human Feedback (RLHF). This imbues the general code large model with the ability to judge code granularity and extract information, thereby completing adaptive code structure analysis and code extraction. Based on this, an adaptive segmentation agent based on the large language model is constructed.

[0085] Understandably, adaptive code splitters aim to simplify code organization and processing during programming, avoid the fragmentation and unreasonable organization of code data caused by current common document segmentation algorithms, facilitate storage in vector databases and subsequent retrieval, and improve development efficiency.

[0086] In this embodiment of the application, the code processing device performs adaptive code segmentation processing on multiple code segments based on multiple code description information. After obtaining multiple segmented codes, it also obtains the first segmentation path and the syntax tree structure of the code syntax tree corresponding to the multiple segmented codes. Based on the first segmentation path, the syntax tree structure and multiple code description information, a code knowledge graph is established.

[0087] In this embodiment, the syntax tree structure can be determined from the code syntax tree. The first segmentation path corresponding to multiple segmented codes can be determined based on the method of obtaining multiple segmented codes from the original code.

[0088] In this embodiment of the application, an initial knowledge graph can be established first using the first segmentation path and syntax tree structure, and then multiple code description information can be used to supplement and explain the information in the initial knowledge graph to obtain a code knowledge graph.

[0089] It should be noted that after the initial graph is established using the first segmentation path and syntax tree structure, multiple code descriptions and multiple natural language descriptions can be used to supplement and interpret the information in the initial graph, thereby obtaining a code knowledge graph. That is, the code knowledge graph also includes multiple natural language descriptions.

[0090] In this embodiment of the application, after the code knowledge graph is established, it is stored in the graph database.

[0091] In this embodiment, the code processing device determines whether multiple code segments need to be further segmented based on the target segmentation granularity in the code segmentation requirement and the segmentation granularity in multiple code description information. If it is determined that multiple code segments do not need to be segmented, the device obtains the syntax tree structure of the code syntax tree; obtains the second segmentation path of the multiple code segments; and establishes a code knowledge graph based on the second segmentation path, the syntax tree structure, and multiple code description information.

[0092] It should be noted that if it is determined that multiple code segments do not need to be split, the code processing device will treat the multiple code segments as multiple split code segments.

[0093] For example, such as Figure 4As shown: Upon receiving a code segmentation instruction, the code processing device obtains the original code from the instruction and then segments it based on the Abstract Syntax Tree (AST) code segmenter to obtain a code syntax tree. Next, an adaptive code segmenter determines multiple code descriptions corresponding to various code segments within the code syntax tree (learning the parsed syntax tree and determining code structure and category). It then obtains the code segmentation requirements from the instruction. Based on the target segmentation granularity in the requirements and the segmentation granularity in the multiple code descriptions, it determines whether further segmentation is needed. If further segmentation is needed... In the case of code segmentation, adaptive code segmentation is performed on multiple code segments to obtain multiple segmented data. If the segmentation granularity of multiple segmented data is the same as the target segmentation granularity in the code segmentation requirement (complete code segmentation), the multiple segmented data are used as multiple segmented codes. If the segmentation granularity of multiple segmented data is different from the target segmentation granularity in the code segmentation requirement (incompletely segmented data), adaptive segmentation is performed on multiple segmented data until the segmentation granularity of the segmented data is the same as the target segmentation granularity in the code segmentation requirement, resulting in multiple segmented codes, which are then stored in the retrieval database. The first segmentation path and the syntax tree structure of the code syntax tree corresponding to the multiple segmented codes are obtained. Based on the first segmentation path, the syntax tree structure, and multiple code description information, a code knowledge graph is established (i.e., constructed) and stored in the graph database. If it is determined that multiple code segments do not need to be segmented, obtain the syntax tree structure of the code syntax tree; obtain the second segmentation path (segmentation path) of the multiple code segments; based on the second segmentation path, the syntax tree structure (syntax tree) and multiple code description information, establish (i.e. construct) the code knowledge graph and store it in the graph database.

[0094] Understandably, this application provides an intelligent code segmentation scheme through techniques such as recursive iteration, calling the AST code splitter, and constructing a code knowledge graph. It offers users an efficient, accurate, and easy-to-use code processing tool, thereby improving programming efficiency and code quality. Furthermore, through more precise code segmentation, it plays a crucial role in the construction of code vector databases, significantly improving the quality and accuracy of code retrieval in subsequent RAG scenarios.

[0095] In this embodiment, the code processing device performs adaptive code segmentation on multiple code segments to obtain multiple segmented codes. Upon receiving a code generation instruction, it obtains code generation requirements from the code generation instruction; determines the complexity of the code generation requirements; if the complexity determines that the code generation requirements need to be split, it splits the code generation requirements to obtain target requirements; it retrieves the requirement code from the multiple segmented codes based on the target requirements; and if the requirement code meets the code generation requirements, it generates the target code corresponding to the code generation instruction based on the requirement code.

[0096] In this embodiment, the code generation instruction can be an instruction input by the user, an instruction transmitted from another device to the code processing device, or an instruction obtained by the code processing device through other means. The specific method of receiving the code generation instruction can be determined according to the actual situation, and this embodiment does not limit it.

[0097] It should be noted that the code generation instruction carries the code generation requirement. This requirement specifically refers to the need to generate code.

[0098] In this embodiment of the application, the process by which the code processing device determines the complexity of the code generation requirement includes: determining the requirement length, semantic complexity, and nesting degree of the code generation requirement; and determining the complexity of the code generation requirement based on the requirement length, semantic complexity, and nesting degree.

[0099] Specifically, the process of determining the complexity of code generation requirements based on requirement length, semantic complexity, and nesting degree includes obtaining requirement length coefficient, semantic complexity coefficient, and nesting degree coefficient, and determining the complexity of code generation requirements based on requirement length, semantic complexity, nesting degree, requirement length coefficient, semantic complexity coefficient, and nesting degree coefficient.

[0100] For example, the complexity of code generation requirements can be determined based on requirement length, semantic complexity, nesting degree, requirement length coefficient, semantic complexity coefficient, and nesting degree coefficient, as shown in formula (2):

[0101] (2)

[0102] It should be noted that score represents the complexity of the code generation requirement, L represents the requirement length, 0.5 is the requirement length coefficient, C represents the semantic complexity, 0.3 is the semantic complexity coefficient, D represents the nesting degree, and 0.2 is the nesting degree coefficient.

[0103] It should be noted that the requirement length coefficient, semantic complexity coefficient, and nesting degree coefficient are configured values ​​that can be adjusted according to specific application scenarios.

[0104] In this embodiment of the application, an exemplary parameter table for determining the complexity of code generation requirements is shown in Table 1:

[0105] Table 1

[0106]

[0107] In this embodiment, if the code generation requirement is determined to be complex based on complexity, then the code generation requirement needs to be broken down for processing. If the code generation requirement is determined to be simple based on complexity, then the code generation requirement needs to be supplemented for processing. In both cases of complex and simple code generation requirements, the code generation requirement is considered abnormal.

[0108] In this embodiment of the application, a complexity threshold can be set, and the code generation requirement can be determined as complex, simple, or without anomalies based on the determined complexity and the complexity threshold.

[0109] In this embodiment, once the target requirement is obtained, the requirement code can be retrieved from multiple code segments based on the target requirement, or the requirement code can be retrieved from multiple code segments based on the target requirement and a code knowledge graph. Then, the requirement code is checked to determine whether it meets the code generation requirements. If the requirement code meets the code generation requirements, target code corresponding to the code generation instructions is generated based on the requirement code.

[0110] It should be noted that the similarity between the requirement code and the code generation requirement can be determined, and whether the requirement code meets the code generation requirement can be determined based on the similarity. Other methods can also be used to determine whether the requirement code meets the code generation requirement. The specific implementation method can be determined according to the actual situation, and this application does not limit it.

[0111] In this embodiment, the process of the code processing device breaking down code generation requirements to obtain target requirements includes: breaking down the code generation requirements into multiple sub-requirements; supplementing any sub-requirement with first information if the content of any sub-requirement is insufficient or the difference between any sub-requirement and the code generation requirements is greater than a difference threshold, obtaining a supplemented sub-requirement; continuing to supplement any sub-requirement if the content of any supplemented sub-requirement is insufficient or the difference between it and the code generation requirements is greater than a difference threshold, until the content of any target sub-requirement is sufficient and the difference between it and the code generation requirements is less than or equal to the difference threshold; and replacing any sub-requirement among the multiple sub-requirements with any target sub-requirement to obtain the target requirement.

[0112] It should be noted that the first information includes the current programming environment or supplementary information obtained through interaction with the user.

[0113] It should also be noted that if the first information is supplementary information obtained through interaction with the user, then the first information can be supplementary information obtained through one or more interactions with the user. The specific number of interactions with the user can be determined according to the actual situation, and this application embodiment does not limit this.

[0114] In this embodiment, if the complexity indicator of the code generation requirement is complex, the code generation requirement needs to be broken down into multiple sub-requirements, and each of these sub-requirements is an independent sub-requirement. Then, it is determined whether the content of each sub-requirement is sufficient or the difference between each sub-requirement and the code generation requirement is significant. If the content of any sub-requirement is insufficient or the difference between any sub-requirement and the code generation requirement is greater than a difference threshold, then the first information is used to supplement that sub-requirement until the content is sufficient and the difference between it and the code generation requirement is less than or equal to the difference threshold.

[0115] It should be noted that determining whether the requirements in each sub-requirement are sufficient includes determining whether the requirements in each sub-requirement are appropriate and whether there is sufficient contextual information.

[0116] It should also be noted that appropriate requirements can be for writing code that does not contain errors.

[0117] In this embodiment, the difference threshold can be a threshold configured in the code processing device, a threshold transmitted to the code processing device from other devices, or a threshold obtained by the code processing device through other means. The specific way the code processing device obtains the difference threshold can be determined according to the actual situation, and this embodiment does not limit it.

[0118] It should be noted that if any target sub-requirement is used to replace any one of the multiple sub-requirements to obtain the target requirement, then the target requirement includes the target sub-requirement and the remaining sub-requirements of the multiple sub-requirements excluding the target sub-requirement.

[0119] For example, such as Figure 5As shown: Upon receiving a code generation instruction, the code generation requirements are obtained from the instruction. An adaptive code retrieval tool is used to process these requirements, determining their complexity (i.e., whether the requirement needs refactoring). If the complexity indicates a complex requirement, refactoring is deemed necessary, and the requirement is broken down into multiple sub-requirements. If the content of any sub-requirement is insufficient, or if the difference between any sub-requirement and the code generation requirement exceeds a threshold, the context is deemed insufficient. The first piece of information is used to supplement the sub-requirement, resulting in a supplemented sub-requirement. If the content of any supplemented sub-requirement is still insufficient, or if the difference between it and the code generation requirement exceeds a threshold, the context is deemed insufficient. This process continues until the content of any target sub-requirement is sufficient and the difference between it and the code generation requirement is less than or equal to the threshold. At this point, the context is deemed sufficient, and the retrieval tool is directly invoked to search the user context code database (a graph database storing code knowledge graphs) and the code database (a retrieval database storing multiple segmented codes) for the requirement code corresponding to that target sub-requirement. If the content of any one of the multiple sub-requirements is sufficient and the difference between it and the code generation requirement is less than or equal to the difference threshold, then the context is considered sufficient. These multiple sub-requirements are then used as the target requirement, and the retrieval tool is directly invoked to search for the corresponding requirement code in the user context code database (a graph database storing code knowledge graphs) and the code database (a retrieval database storing multiple segmented codes). If the complexity indicator indicates that the code generation requirement needs supplementary processing, then the code generation requirement is used to search within multiple segmented codes to obtain retrieval information. This retrieval information is then used to supplement the code generation requirement. If the content of the supplemented code generation requirement is sufficient and the difference between it and the code generation requirement is less than or equal to the difference threshold, then the context is considered sufficient, thus obtaining the target requirement. The retrieval tool is then directly invoked to search for the corresponding requirement code in the user context code database (a graph database storing code knowledge graphs) and the code database (a retrieval database storing multiple segmented codes). If the complexity indicator for code generation shows no anomalies, and a refactoring is deemed unnecessary, the next step is to determine if a knowledge base is required. If so, the code generation requirement is designated as the target requirement, and the retrieval tool is directly invoked to search for the corresponding requirement code in the user context code database (a graph database storing code knowledge graphs) and the code database (a retrieval database storing multiple code segments). If a knowledge base is not required, the code generation requirement is processed directly (requirement processing), without needing to invoke the user context code database (a graph database storing code knowledge graphs) and the code database (a retrieval database storing multiple code segments).

[0120] The code generation requirement is determined to be abnormal; a requirement refactoring (supplementing the code generation requirement) is necessary. First, supplement any sub-requirement using the first piece of information, including supplementary information obtained from the current programming environment or user interaction. If the content of any supplemented sub-requirement is insufficient or the difference between it and the code generation requirement exceeds a threshold, continue supplementing any sub-requirement until the content of any target sub-requirement (with sufficient context) is sufficient and the difference between it and the code generation requirement is less than or equal to the threshold. Replace any sub-requirement among the multiple sub-requirements with the target sub-requirement to obtain the target requirement. Based on the target requirement, retrieve the requirement code from the user context code database (which stores a code knowledge graph) and the code database (which stores multiple segmented codes). If the content of each sub-requirement among the multiple sub-requirements is sufficient and the difference between it and the code generation requirement is less than or equal to the threshold, it is determined that the code generation requirement is not abnormal; no requirement refactoring is needed, and the multiple sub-requirements are used as target requirements to retrieve the requirement code. If the complexity identifier code generation requirement needs to be supplemented, a knowledge base is required. In this case, the code generation requirement is used to search through multiple code segments to obtain search information. The search information is then used to supplement the code generation requirement to obtain the target requirement. If the complexity identifier code generation requirement is normal, a knowledge base is not required, and the code generation requirement is determined as the target requirement.

[0121] In this embodiment of the application, after the code processing device breaks down the code generation requirement into multiple sub-requirements, if the requirement content of any one of the multiple sub-requirements is sufficient and the difference between it and the code generation requirement is less than or equal to the difference threshold, the multiple sub-requirements are taken as the target requirement.

[0122] It should be noted that if multiple sub-requirements are taken as target requirements, then the target requirements include multiple sub-requirements.

[0123] In this embodiment of the application, after the code processing device determines the complexity of the code generation requirement, before retrieving the requirement code from multiple segmented codes based on the target requirement, if the complexity indicates that the code generation requirement needs to be supplemented, the code generation requirement is used to search in multiple segmented codes to obtain search information; the search information is used to supplement the code generation requirement to obtain the target requirement; if the complexity indicates that the code generation requirement is not abnormal, the code generation requirement is determined as the target requirement.

[0124] In this embodiment, the code generation requirement can be used to perform a single search across multiple code segments to obtain one search result, which can then be used to supplement the code generation requirement once. Alternatively, the code generation requirement can be used to perform multiple searches across multiple code segments to obtain multiple search results, which can then be used to supplement the code generation requirement multiple times.

[0125] In this embodiment, the code generation requirements can be supplemented using search information, or supplementary information can be determined based on the current programming environment. Alternatively, supplementary information can be obtained from user interaction to supplement the code generation requirements, or other methods can be used to obtain supplementary information. The specific method can be determined according to the actual situation, and this embodiment does not limit it.

[0126] It should be noted that if the code generation requirements are supplemented and no exceptions are found, then the supplemented requirements should be taken as the target requirements.

[0127] For example, such as Figure 6As shown: The code processing device first determines the complexity of the code generation requirement (user requirement). If the complexity determines that the code generation requirement needs to be broken down (i.e., it is judged to be a complex requirement), it breaks down the code generation requirement into several independent sub-requirements. Then, it checks whether the content of any sub-requirement is sufficient or whether the difference between any sub-requirement and the code generation requirement exceeds a threshold (i.e., whether the content of any sub-requirement is appropriate, consistent with the user's original requirement, and whether there is sufficient contextual information). If a sub-requirement is insufficient or significantly different from the original requirement (i.e., the content of any sub-requirement is insufficient or the difference between any sub-requirement and the code generation requirement exceeds a threshold), it performs a context search based on the current user programming environment (first requirement), prompting the user to interact and supplement information and contextual information to modify the sub-requirement (modifying any sub-requirement). Multi-turn dialogue combined with enhanced retrieval is used to satisfy the conditions, thus obtaining any target sub-requirement whose content is sufficient and whose difference from the code generation requirement is less than or equal to the threshold. When the complexity indicator of a code generation requirement requires further processing (i.e., it is judged to be a simple requirement), such as a one-line requirement, the system will automatically perform requirement retrieval and integration (i.e., expansion). Specifically, if the knowledge model itself does not possess the knowledge involved in the current requirement or it is highly personalized, the model will call the retrieval tool to search the code knowledge base data based on the judgment result, supplementing the missing domain or personalized knowledge, and finally obtaining the target requirement (i.e., using the code generation requirement to search in multiple code segments to obtain retrieval information; using the retrieval information to supplement the code generation requirement to obtain the target requirement). If the complexity indicator of a code generation requirement is normal, the code generation requirement is determined as the target requirement and directly output (directly output the requirement).

[0128] In this embodiment of the application, the process by which the code processing device retrieves the requirement code from multiple segmented codes based on the target requirement includes: determining multiple first initial codes corresponding to the target requirement from the multiple segmented codes; determining multiple second initial codes corresponding to the target requirement from the multiple segmented codes using a retrieval model; searching for upstream and downstream codes of the multiple first initial codes and the multiple second initial codes in a code knowledge graph; determining the similarity between the upstream and downstream codes, the multiple first initial codes, and the multiple second initial codes and the target requirement; and selecting the code with the highest similarity among the upstream and downstream codes, the multiple first initial codes, and the multiple second initial codes as the requirement code.

[0129] In this embodiment of the application, after the target requirement is determined, a semantic similarity model can be invoked to retrieve the code fragment (i.e., the requirement code) that is most similar to the current requirement step.

[0130] In this embodiment of the application, after determining multiple first initial codes corresponding to the target requirement among multiple segmented codes, the matching degree between the multiple first initial codes and the target requirement can also be determined to obtain multiple matching degrees; the multiple first initial codes are sorted based on the multiple matching degrees to obtain sorted codes, and the sorted codes are displayed.

[0131] In this embodiment, the Embedding model can be used to transform multiple segmented codes into multiple first vectors, and the Embedding model can be used to transform the target requirement into a second vector. The similarity between the multiple first vectors and the second vectors can be determined, thereby identifying multiple first initial codes corresponding to the target requirement.

[0132] It should be noted that multiple segmentation codes can also be converted into multiple first vectors in other ways, or the target requirement can be converted into a second vector. The specific implementation method can be determined according to the actual situation, and this application embodiment does not limit it.

[0133] In this embodiment of the application, the retrieval model can also be implemented using a large model. For example, the retrieval model can be an LLM2Vec model.

[0134] In this embodiment of the application, the target requirement can be used as input, and a retrieval model can be used to retrieve multiple code fragments that are most similar to the target requirement from multiple segmented codes to obtain multiple second initial codes.

[0135] In this embodiment, an Embedding model can be used to transform code generation requirements (i.e., target requirements) and corresponding code snippets (multiple segmented codes) into vector representations. By calculating the cosine similarity between vectors, this application can evaluate the degree of matching between code snippets and requirements. Secondly, based on the preliminary search results, a Rerank model will further rank the code snippets. This process ensures that the most relevant code snippets are presented to the user first. The search tool also utilizes an LLM2Vec model, taking sub-requirements as instruction inputs to assist in retrieving the code snippets most similar to the current requirement (i.e., using the search model to determine multiple second initial codes corresponding to the target requirement from multiple segmented codes). First, the recalled code snippets (i.e., multiple first initial codes and multiple second initial codes) will be sent to a graph database to find their upstream and downstream code structures, thereby expanding the search content. Secondly, the Rerank model is used to re-rank all the above search results to select the code snippets most similar to the current requirements (i.e., determine the similarity between the upstream and downstream code, multiple first initial codes, and multiple second initial codes and the target requirements respectively; among the upstream and downstream code, multiple first initial codes, and multiple second initial codes, the code with the highest similarity is taken as the requirement code).

[0136] For example, such as Figure 7 As shown: Among multiple code segments, identify multiple first initial codes corresponding to the target requirement (i.e., use an embedding model to transform the requirement and code fragments into vectors); use a retrieval model to identify multiple second initial codes corresponding to the target requirement from the multiple code segments (use an LLM2Vec model to assist in retrieving similar codes); in the code knowledge graph, find the upstream and downstream codes of the multiple first initial codes and multiple second initial codes respectively (structure retrieval in the graph database to find the upstream and downstream code structure); determine the similarity between the upstream and downstream codes, the multiple first initial codes, and the multiple second initial codes and the target requirement respectively (finally re-rank and select the most relevant code fragments); among the upstream and downstream codes, the multiple first initial codes, and the multiple second initial codes, select the code with the highest similarity as the requirement code (final retrieval information).

[0137] It should be noted that, after identifying multiple initial codes corresponding to the target requirement among multiple code segments, the matching degree between the multiple initial codes and the target requirement can be determined to obtain multiple matching degrees (cosine similarity is calculated to evaluate the degree of matching); the multiple initial codes are sorted based on the multiple matching degrees to obtain sorted codes (the rerank model is applied to sort the code snippets), and the sorted codes are displayed.

[0138] Understandably, this application provides a more accurate and efficient code retrieval service by intelligently analyzing user needs, combining an advanced semantic similarity model (used to retrieve code snippets most similar to code generation needs), and an adaptive requirement adjustment mechanism (adjusting code generation requirements).

[0139] In this embodiment, after the code processing device retrieves the requirement code from multiple segmented codes based on the target requirement, if the requirement code does not meet the code generation requirement, it supplements the code generation requirement to obtain a supplemented code generation requirement; it then retrieves a new requirement code based on the supplemented code generation requirement; and if the number of times the new requirement code is determined is greater than a preset number or the new requirement code meets the code generation requirement, it generates the target code corresponding to the code generation instruction based on the requirement code.

[0140] It should be noted that the similarity between the requirement code and the code generation requirement can be determined, and the similarity can be used to determine whether the requirement code meets the code generation requirement; other methods can also be used to determine whether the requirement code meets the code generation requirement; the specific implementation method can be determined according to the actual situation, and the embodiments of this application do not limit it.

[0141] In this embodiment, if the required code does not meet the code generation requirements, the code generation requirements can be supplemented to obtain supplemented code generation requirements. Alternatively, the target requirements can be further supplemented to obtain supplemented code generation requirements. Specific implementation methods can be determined based on actual circumstances, and this embodiment does not limit this approach.

[0142] Specifically, the code generation requirements are supplemented again, and the method for obtaining the supplemented code generation requirements is the same as the method for obtaining the target requirements by supplementing the code generation requirements. The target requirements are then further supplemented, and the method for obtaining the supplemented code generation requirements is the same as the method for obtaining the target requirements by supplementing the code generation requirements.

[0143] In this embodiment of the application, the method of retrieving new requirement code based on the supplemented code generation requirements is the same as the method of retrieving requirement code from multiple segmented codes based on target requirements. For details, please refer to the method of retrieving requirement code from multiple segmented codes based on target requirements.

[0144] In this embodiment, the preset number of times can be information configured in the code processing device, information transmitted to the code processing device from other devices, or information obtained by the code processing device through other means. The specific way in which the code processing device obtains the preset number of times can be determined according to the actual situation, and this embodiment does not limit it.

[0145] For example, the preset number of times can be 5 times, or it can be 10 times, or it can be other values. The specific value of the preset number of times can be determined according to the actual situation, and this application embodiment does not limit it.

[0146] In this embodiment, the code processing device includes an adaptive code retrieval unit, which can be implemented using a large model. The adaptive code retrieval unit first rigorously judges the currently retrieved code fragments to determine whether they meet the current requirements (i.e., whether the required code meets the code generation requirements). If the retrieved code fragment meets the current requirements, it is stored in the retrieval memory for subsequent use. If the code fragment does not meet the requirements (i.e., the required code does not meet the code generation requirements), the adaptive code retrieval unit further splits and expands the programming requirements based on context information (i.e., it supplements the code generation requirements to obtain supplemented code generation requirements), and selects a code database of appropriate granularity for re-retrieval (i.e., it re-retrieves new required code based on the supplemented code generation requirements). Simultaneously, it autonomously judges whether the retrieved code fragments are valid, continuously adjusts the retrieval strategy, and continues retrieval until the requirements are met or the maximum number of retrievals is reached (i.e., when the number of times new required code is determined is greater than a preset number or the new required code meets the code generation requirements, target code corresponding to the code generation instruction is generated based on the required code).

[0147] In this embodiment, a requirement rewriting judgment agent and a code retrieval judgment agent are constructed through post-training reinforcement learning. During training, the agents continuously learn to construct optimal selection and implementation paths. Therefore, in practical applications, the agents can construct the best implementation path for the current scenario based on the current scene and user interaction using a Monte Carlo search tree. This enables the adaptive code retrieval engine to intelligently analyze user needs, invoke semantic similarity models, LLM2Vec models, and graph databases, and employ adaptive requirement and retrieval adjustment mechanisms.

[0148] Understandably, this application implements an adaptive retrieval framework that applies reinforcement learning and path search to requirement decomposition, summarization, and intelligent retrieval. It can autonomously analyze user needs, decompose complex needs, and expand more detailed code generation requirements based on context, transforming vague needs into specific and searchable programming requirements, thereby improving the overall accuracy of retrieval.

[0149] In this embodiment of the application, the process of the code processing device generating target code corresponding to the code generation instruction based on the requirement code includes: generating initial code corresponding to the code generation requirement according to the requirement code; optimizing the initial code based on multiple segmented codes and code knowledge graphs when running the initial code and determining that there is erroneous code, to obtain optimized code; and adjusting the format of the optimized code to obtain target code when running the optimized code and determining that there is no erroneous code.

[0150] In this embodiment, the code processing device includes an adaptive code judge. Specifically, the adaptive code judge can be implemented using a large model. Given the required code and code generation requirements, it can first analyze the user's programming requirements (i.e., code generation requirements) to understand their logic and goals, while simultaneously learning the required code and extracting key information from it. Based on this comprehensive understanding, initial code is generated using internal intelligent algorithms and a programming knowledge system.

[0151] In modern programming, efficiently and accurately meeting user needs and generating executable code is a crucial task. As software development becomes increasingly complex, existing code generation methods often struggle to meet diverse requirements. To address this issue, adaptive code generators can be integrated with intelligent decision-making mechanisms to provide users with high-quality code generation services.

[0152] In this embodiment of the application, the method of generating initial code corresponding to the code generation requirements can be determined according to the actual situation, and this embodiment of the application does not limit it.

[0153] In this embodiment of the application, the code processing device also includes a code executor, which can be used to run the initial code to obtain the running result. If the running result indicates that there is erroneous code in the initial code, the initial code is optimized based on multiple code segments and a code knowledge graph to obtain optimized code.

[0154] In this embodiment of the application, the process of optimizing the initial code based on multiple code segments and a code knowledge graph to obtain optimized code includes: searching for associated code with the erroneous code in multiple code segments based on the code knowledge graph; and optimizing the initial code based on the associated code to obtain optimized code.

[0155] The code executor can be a sandbox, or any other executor capable of running code. The specific code executor can be determined based on the actual situation, and this application does not limit this.

[0156] For example, code can be executed by invoking a programming sandbox. Based on the sandbox feedback, it is determined whether environment operations are needed for the sandbox and whether user context codebase information should be retrieved to optimize the code. If the sandbox feedback indicates problems, such as memory leaks or execution failures (i.e., the presence of erroneous code), appropriate measures will be taken to adjust the strategy or optimize the algorithm. If it is determined that context information needs to be retrieved, more resources and knowledge will be sought based on the programming environment to optimize the code.

[0157] In this embodiment of the application, after obtaining the optimized code, the optimized code is run. If the running result of the optimized code determines that there is no erroneous code in the optimized code, the format of the optimized code is adjusted to obtain the target code.

[0158] It should be noted that the format of the adjusted and optimized code includes the adjusted code style and naming format.

[0159] Specifically, the user-written code can be used as a preset format, and then the optimized code can be adjusted to the preset format to obtain the target code.

[0160] For example, while retrieving contextual information, code style and naming conventions are continuously modified to conform to the user's programming context. It can detect and adjust for differences in style, such as converting underscore naming to camelCase. After a series of operations, it outputs executable code that meets requirements and has undergone rigorous testing and optimization, exhibiting high quality and reliability.

[0161] For example, such as Figure 8 As shown, given the code generation requirements (code requirements) and the requirement code (retrieval information), the adaptive code judge generates initial code corresponding to the code generation requirements based on the requirement code, calls the sandbox to run the initial code, and obtains the running result; if the running result indicates that there is erroneous code in the initial code, it optimizes the initial code based on multiple code segments and code knowledge graph (user context code database) to obtain optimized code; if the optimized code is confirmed to be free of erroneous code by running it in the sandbox, the format of the optimized code is adjusted to obtain the target code (executable code).

[0162] It should be noted that the adaptive code generator, as an innovative programming tool, provides users with efficient and accurate code generation services through its intelligent learning and understanding, adaptive decision-making mechanism, and continuous adjustment of code style and naming conventions. It has demonstrated powerful capabilities in handling complex programming needs and diverse programming environments.

[0163] Understandably, this application comprehensively utilizes code generation requirements and retrieved information, adaptively invokes a programming sandbox for code execution, optimizes code programming based on sandbox feedback, and ensures that the final generated code conforms to the user's current programming context and coding style.

[0164] For example, such as Figure 9As shown, the code processing device includes an adaptive code segmenter, an adaptive code retrieval unit, and an adaptive code judge, where both the adaptive code segmenter, adaptive code retrieval unit, and adaptive code judge can be implemented using a large model. The code processing device first uses the adaptive code segmenter to segment the original code, resulting in multiple segmented codes, which are stored in a code database. Upon receiving a code generation instruction, the device retrieves the code generation requirements from the instruction and uses the adaptive code retrieval unit to retrieve the required code from the code database. Finally, the adaptive code judge generates initial code corresponding to the required code based on the required code, calls a sandbox to run the initial code, and obtains the execution result. If the execution result indicates the presence of erroneous code in the initial code, the device optimizes the initial code based on the multiple segmented codes and a code knowledge graph (user context code database), obtaining optimized code. If the optimized code is confirmed to be error-free by running it in the sandbox, the format of the optimized code is adjusted to obtain the target code (executable code).

[0165] This application provides a complete set of more intelligent and personalized programming assistance devices by setting up an adaptive code segmenter, an adaptive code retrieval device, and an adaptive code judge. Specifically, this application achieves efficient code data processing and generation through three core components. First, the adaptive code segmenter utilizes an innovative recursive iterative large language model segmenter device. It first segments the original code structure through an abstract syntax tree, and then recursively calls the adaptive segmentation device to achieve multi-level, multi-granularity, and multi-semantic code data segmentation. Second, the adaptive code retrieval device first intelligently analyzes user requirements and expands and summarizes them interactively. Then, it retrieves code fragments based on a traditional semantic similarity model, judges whether they meet the requirements, and if not, further expands the requirements and adaptively judges them until the requirements are met, and then stores them in the retrieval memory. Third, the adaptive code judge first learns and understands the programming requirements and the retrieved code fragments to generate an initial version of code. Then, it adaptively calls the programming sandbox to execute the code, judges whether to perform environment operations and retrieve user context code library information based on feedback to optimize the generated code, and continuously modifies the code style and naming conventions, finally outputting executable code results. This provides the programming field with an efficient, accurate, and robust mechanism for code segmentation, retrieval, and generation. By simplifying user operations, optimizing model structure, and improving model performance, the efficiency and quality of programming work can be significantly improved.

[0166] This application employs a large-scale model fine-tuning technique to achieve unified model application across multiple domains (i.e., the adaptive code judge, adaptive code retrieval unit, and adaptive code splitter in this application can all be implemented based on a large model, utilizing a single large model to execute the code processing methods in this application), which has a profound and positive impact on the adaptability and robustness of the entire solution. First, since all models are fine-tuned based on the same large code model, this method simplifies the model selection process. In traditional machine learning and deep learning projects, selecting a suitable model often requires extensive experimentation and adjustment, which is not only time-consuming and labor-intensive but also requires specialized knowledge. This application, through a unified base model, reduces the complexity of this process, making model selection and adjustment more direct and efficient. Second, the use of a unified base model means that models in different domains have consistency in their underlying structure. This consistency allows models to adapt to new environments more quickly when applied across domains, improving model adaptability. Whether in different programming tasks such as code generation, code defect detection, or code recommendation, the model demonstrates good generalization ability. Finally, this application also promotes model interpretability and transparency. When all models are based on the same foundation, it becomes easier to understand and interpret the behavior of the models.

[0167] This application employs large model fine-tuning and reinforcement learning techniques, using a unified code-based large model as the base model. This not only simplifies the model selection and maintenance process but also significantly improves the model's adaptability, robustness, maintainability, and interpretability.

[0168] Understandably, the code processing device first uses an abstract syntax tree to roughly segment the original code, obtaining a code syntax tree. Based on the code syntax tree, it determines multiple code description information for multiple code segments. This multiple code description information includes multiple code result information and multiple code function information corresponding to multiple code segments. This allows for detailed segmentation of multiple code segments based on multiple code structure information and multiple code function information, resulting in multiple segmented codes that correspond to the code structure information and have code functions. When recalling code from multiple segmented codes for code generation, the device can retrieve segmented codes that match the code generation requirements based on the code structure information and code functions, thus improving the accuracy of code generation.

[0169] Based on the same inventive concept as the above-described code processing method, this application provides a code processing device 1, corresponding to a code processing method; Figure 10 A schematic diagram of the composition structure of a code processing device provided in this application embodiment. Figure 1 The code processing device 1 may include:

[0170] The acquisition unit 11 is used to acquire the original code from the code splitting instruction when a code splitting instruction is received;

[0171] The segmentation unit 12 is used to segment the original code based on the abstract syntax tree to obtain a code syntax tree;

[0172] The determining unit 13 is used to determine multiple code description information corresponding to multiple code segments in the code syntax tree; the multiple code description information includes multiple code structure information and multiple code function information.

[0173] The processing unit 14 is used to perform adaptive code segmentation processing on the multiple code segments based on the multiple code description information to obtain multiple segmented codes.

[0174] In some embodiments of this application, the acquisition unit 11 is used to acquire code splitting requirements from the code splitting instructions;

[0175] The determining unit 13 is used to determine whether the multiple code segments need to be further segmented based on the target segmentation granularity in the code segmentation requirement and the segmentation granularity in the multiple code description information.

[0176] The processing unit 14 is used to perform adaptive code segmentation processing on the multiple code segments when it is determined that the multiple code segments need to be further segmented, so as to obtain the multiple segmented code segments.

[0177] In some embodiments of this application, the processing unit 14 is configured to continue adaptive code segmentation processing on the multiple code segments to obtain multiple segmented data; if the segmentation granularity corresponding to the multiple segmented data is different from the target segmentation granularity in the code segmentation requirement, the processing unit 14 continues adaptive segmentation processing on the multiple segmented data until the segmentation granularity of the segmented data is the same as the target segmentation granularity in the code segmentation requirement, thereby obtaining the multiple segmented code.

[0178] The determining unit 13 is used to take the multiple segmented data as the multiple segmented codes when the segmentation granularity corresponding to the multiple segmented data is the same as the target segmentation granularity in the code segmentation requirement.

[0179] In some embodiments of this application, the apparatus further includes an establishment unit;

[0180] The acquisition unit 11 is used to acquire the first segmentation path corresponding to the plurality of segmented codes and the syntax tree structure of the code syntax tree;

[0181] The establishment unit is used to establish a code knowledge graph based on the first segmentation path, the syntax tree structure, and the multiple code description information.

[0182] In some embodiments of this application, the apparatus further includes a splitting unit, a retrieval unit, and a generation unit;

[0183] The acquisition unit 11 is used to acquire code generation requirements from the code generation instructions when a code generation instruction is received.

[0184] The determining unit 13 is used to determine the complexity of the code generation requirement;

[0185] The splitting unit is used to split the code generation requirement to obtain the target requirement when it is determined that the code generation requirement needs to be split based on the complexity.

[0186] The retrieval unit is used to retrieve the requirement code from the plurality of segmented codes based on the target requirement;

[0187] The generation unit is used to generate target code corresponding to the code generation instruction based on the requirement code, provided that the requirement code meets the code generation requirements.

[0188] In some embodiments of this application, the apparatus further includes a supplementary unit and a replacement unit;

[0189] The splitting unit is used to split the code generation requirement into multiple sub-requirements;

[0190] The supplementary unit is configured to supplement any sub-requirement with first information when the content of any sub-requirement among the plurality of sub-requirements is insufficient or the difference between any sub-requirement and the code generation requirement is greater than a difference threshold, thereby obtaining a supplemented sub-requirement; the first information includes supplementary information obtained from the current programming environment or through interaction with the user; if the content of any supplemented sub-requirement is insufficient or the difference between it and the code generation requirement is greater than a difference threshold, the unit continues to supplement the sub-requirement until the content of any target sub-requirement obtained is sufficient and the difference between it and the code generation requirement is less than or equal to the difference threshold;

[0191] The replacement unit is used to replace any one of the plurality of sub-requirements with any one of the target sub-requirements to obtain the target requirement.

[0192] In some embodiments of this application, the determining unit 13 is used to take the plurality of sub-requirements as the target requirement when the requirement content of any one of the plurality of sub-requirements is sufficient and the difference between the sub-requirement and the code generation requirement is less than or equal to a difference threshold.

[0193] In some embodiments of this application, the retrieval unit is used to retrieve retrieval information by using the code generation requirement in the multiple segmented code when the complexity indicates that the code generation requirement needs to be supplemented.

[0194] The supplementary unit is used to supplement the code generation requirements with the retrieved information to obtain the target requirements;

[0195] The determining unit 13 is used to determine the code generation requirement as the target requirement when the complexity indicator shows no abnormality in the code generation requirement.

[0196] In some embodiments of this application, the apparatus further includes a search unit;

[0197] The determining unit 13 is configured to: determine, from the plurality of segmented codes, a plurality of first initial codes corresponding to the target requirement; determine, from the plurality of segmented codes, a plurality of second initial codes corresponding to the target requirement using a retrieval model; determine, respectively, the similarity between the upstream and downstream codes, the plurality of first initial codes, and the plurality of second initial codes and the target requirement; and select the code with the highest similarity among the upstream and downstream codes, the plurality of first initial codes, and the plurality of second initial codes as the requirement code.

[0198] The search unit is used to search for upstream and downstream code of the plurality of first initial codes and the plurality of second initial codes in the code knowledge graph.

[0199] In some embodiments of this application, the supplementing unit is used to supplement the code generation requirements based on the code generation requirements when the requirement code does not meet the code generation requirements, so as to obtain the supplemented code generation requirements.

[0200] The retrieval unit is used to retrieve new requirement code based on the supplemented code generation requirements.

[0201] The generation unit is used to generate target code corresponding to the code generation instruction based on the requirement code when the number of times the new requirement code is determined is greater than a preset number or the new requirement code meets the code generation requirements.

[0202] In some embodiments of this application, the apparatus further includes an optimization unit and an adjustment unit;

[0203] The generation unit is used to generate initial code corresponding to the code generation requirement based on the requirement code;

[0204] The optimization unit is configured to optimize the initial code based on multiple code segments and the code knowledge graph, after running the initial code and determining that there is erroneous code, to obtain optimized code;

[0205] The adjustment unit is used to adjust the format of the optimized code to obtain the target code after running the optimized code and determining that there are no errors.

[0206] It should be noted that, in practical applications, the aforementioned acquisition unit 11, segmentation unit 12, determination unit 13, and processing unit 14 can be implemented by the processor 15 on the code processing device, specifically by a CPU (Central Processing Unit), MPU (Microprocessor Unit), DSP (Digital Signal Processor), or FPGA (Field Programmable Gate Array), etc.; the aforementioned data storage can be implemented by the memory 16 on the code processing device.

[0207] This application also provides a code processing device, such as... Figure 11 As shown, the code processing device includes a processor 15, a memory 16, and a communication bus 17. The memory 16 communicates with the processor 15 through the communication bus 17. The memory 16 stores programs executable by the processor 15. When the program is executed, the processor 15 executes the code processing method as described above.

[0208] In practical applications, the aforementioned memory 16 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 15.

[0209] This application provides a computer-readable storage medium having a computer program thereon, which, when executed by a processor 15, implements the code processing method described above.

[0210] For example, this application also provides a computer program product, including a computer program that can be executed by a processor 15 in a code processing device to complete the steps described in the aforementioned code processing method.

[0211] Understandably, the code processing device first uses an abstract syntax tree to roughly segment the original code, obtaining a code syntax tree. Based on the code syntax tree, it determines multiple code description information for multiple code segments. This multiple code description information includes multiple code result information and multiple code function information corresponding to multiple code segments. This allows for detailed segmentation of multiple code segments based on multiple code structure information and multiple code function information, resulting in multiple segmented codes that correspond to the code structure information and have code functions. When recalling code from multiple segmented codes for code generation, the device can retrieve segmented codes that match the code generation requirements based on the code structure information and code functions, thus improving the accuracy of code generation.

[0212] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0213] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0214] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0215] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0216] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A code processing method, characterized in that, The method includes: Upon receiving a code splitting instruction, the original code is obtained from the code splitting instruction; The original code is segmented based on the abstract syntax tree to obtain a code syntax tree; Determine multiple code description information corresponding to multiple code segments in the code syntax tree; the multiple code description information includes multiple code structure information and multiple code function information. Based on the multiple code description information, adaptive code segmentation is performed on the multiple code segments to obtain multiple segmented codes; The adaptive code segmentation process based on the multiple code description information to obtain multiple segmented codes includes: Obtain the code splitting requirements from the code splitting instructions; Based on the target segmentation granularity in the code segmentation requirement and the segmentation granularity in the multiple code description information, determine whether the multiple code segments need to be further segmented; If it is determined that the multiple code segments need to be further segmented, adaptive code segmentation is performed on the multiple code segments to obtain the multiple segmented code segments.

2. The method according to claim 1, characterized in that, The adaptive code segmentation process, which involves processing the multiple code segments to obtain the multiple segmented code segments, includes: Continue to perform adaptive code segmentation on the multiple code segments to obtain multiple segmentation data; If the granularity of the multiple data segments is the same as the target granularity in the code segmentation requirement, the multiple data segments will be used as the multiple code segments. If the granularity of the multiple segmented data is different from the target granularity in the code segmentation requirement, the multiple segmented data will continue to be adaptively segmented until the granularity of the segmented data is the same as the target granularity in the code segmentation requirement, thus obtaining the multiple segmented codes.

3. The method according to claim 1, characterized in that, After performing adaptive code segmentation on the multiple code segments based on the multiple code description information to obtain multiple segmented code segments, the method further includes: Obtain the first segmentation path corresponding to the multiple segmented codes and the syntax tree structure of the code syntax tree; A code knowledge graph is established based on the first segmentation path, the syntax tree structure, and the multiple code description information.

4. The method according to claim 1, characterized in that, After performing adaptive code segmentation on the multiple code segments to obtain multiple segmented code segments, the method further includes: Upon receiving a code generation instruction, the code generation requirements are obtained from the code generation instruction; Determine the complexity of the code generation requirements; If the code generation requirement needs to be split based on the complexity, the code generation requirement is split to obtain the target requirement. Based on the target requirements, the requirement code is retrieved from the multiple segmentation codes; If the requirement code meets the code generation requirements, target code corresponding to the code generation instruction is generated based on the requirement code.

5. The method according to claim 4, characterized in that, The process of breaking down the code generation requirements to obtain target requirements includes: The code generation requirement is broken down into multiple sub-requirements; If the content of any sub-requirement among the plurality of sub-requirements is insufficient or the difference between any sub-requirement and the code generation requirement is greater than the difference threshold, the sub-requirement is supplemented using first information to obtain a supplemented sub-requirement; the first information includes the current programming environment or supplementary information obtained through interaction with the user. If the content of any supplemented sub-requirement is insufficient or the difference between it and the code generation requirement is greater than the difference threshold, continue to supplement any sub-requirement until the content of any target sub-requirement is sufficient and the difference between it and the code generation requirement is less than or equal to the difference threshold. The target requirement is obtained by replacing any one of the plurality of sub-requirements with any one of the target sub-requirements.

6. The method according to claim 5, characterized in that, After breaking down the code generation requirement into multiple sub-requirements, the method further includes: If the content of any one of the multiple sub-requirements is sufficient and the difference between it and the code generation requirement is less than or equal to a difference threshold, then the multiple sub-requirements shall be taken as the target requirement.

7. The method according to claim 5, characterized in that, After determining the complexity of the code generation requirement, and before retrieving the requirement code from the multiple code segments based on the target requirement, the method further includes: When the complexity indicates that the code generation requirement needs to be supplemented, the code generation requirement is used to search among the multiple code segments to obtain search information; The search information is used to supplement the code generation requirements to obtain the target requirements; If the complexity indicator shows no abnormalities in the code generation requirement, then the code generation requirement is identified as the target requirement.

8. The method according to claim 4, characterized in that, The step of retrieving the requirement code from the multiple segmented codes based on the target requirement includes: Among the multiple segmented codes, a plurality of first initial codes corresponding to the target requirement are determined; A retrieval model is used to determine multiple second initial codes corresponding to the target requirement from the multiple segmented codes; In the code knowledge graph, the upstream and downstream codes of the plurality of first initial codes and the plurality of second initial codes are searched respectively; The similarity between the upstream and downstream code, the plurality of first initial codes and the plurality of second initial codes and the target requirement is determined respectively; Among the upstream and downstream codes, the plurality of first initial codes, and the plurality of second initial codes, the code with the highest similarity is selected as the requirement code.

9. The method according to claim 4, characterized in that, After retrieving the requirement code from the multiple segmented codes based on the target requirement, the method further includes: If the required code does not meet the code generation requirements, the code generation requirements are supplemented to obtain the supplemented code generation requirements. Based on the supplemented code generation requirements, new requirement codes were retrieved again. If the number of times a new requirement code is identified is greater than a preset number, or if the new requirement code meets the code generation requirements, target code corresponding to the code generation instruction is generated based on the requirement code.

10. The method according to claim 4, characterized in that, The step of generating target code corresponding to the code generation instruction based on the requirement code includes: Generate initial code corresponding to the code generation requirements based on the stated requirements; If the initial code is run and errors are found, the initial code is optimized based on multiple code segments and a code knowledge graph to obtain optimized code. If no errors are found when running the optimized code, the format of the optimized code is adjusted to obtain the target code.

11. A code processing device, characterized in that, The device includes: The acquisition unit is used to acquire the original code from the code splitting instruction upon receiving the code splitting instruction; A segmentation unit is used to segment the original code based on an abstract syntax tree to obtain a code syntax tree; A determining unit is used to determine multiple code description information corresponding to multiple code segments in the code syntax tree; the multiple code description information includes multiple code structure information and multiple code function information. The processing unit is configured to perform adaptive code segmentation processing on the multiple code segments based on the multiple code description information to obtain multiple segmented codes; The acquisition unit is used to acquire code splitting requirements from the code splitting instructions; The determining unit is used to determine whether the multiple code segments need to be further segmented based on the target segmentation granularity in the code segmentation requirement and the segmentation granularity in the multiple code description information; The processing unit is used to perform adaptive code segmentation processing on the multiple code segments when it is determined that the multiple code segments need to be further segmented, so as to obtain the multiple segmented code segments.

12. An electronic device, characterized in that, The electronic device includes: Memory and processor; The memory stores a computer program or instructions, which, when executed by a processor, implement the method described in any one of claims 1 to 10.

13. A storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method described in any one of claims 1 to 10.

14. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images