Method for generating training data and electronic device

Abstract

The application discloses a training data generation method and an electronic device, relates to the technical field of model data training, and through accurate symbol tracking and cross-file reference analysis of a parsed tree model after training, structured data generated more accurately reflects the use and semantic relationship of symbols in a code library, thereby providing a solid foundation for subsequent generated training data. By associating the structured data with technical specification documents, the generated knowledge base contains official definitions and functional descriptions of code symbols, which can greatly enhance the understanding ability of a large model for underlying system code. The construction of a global symbol table takes into account conditional compilation and multi-architecture support, which enables the generated training data to cover different compilation configurations and architectures, thereby enhancing the generalization ability and practicality of the model. Therefore, the problem that existing schemes are difficult to generate effective training data in complex scenarios can be solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of model data training, and in particular, relates to a training data generation method and an electronic device. BACKGROUND

[0002] In the process of training data generation in the prior art, simple code preprocessing and syntax tree parsing are generally used, and there is a lack of processing of cross-file dependencies, and it is even more impossible to process complex situations such as conditional compilation and macro expansion. The training data generated by directly processing UEFI (Unified Extensible Firmware Interface, a modern firmware interface standard, aiming to replace the traditional BIOS) / BIOS (Basic Input Output System, a firmware system responsible for completing hardware initialization and startup booting process) code using the existing code parsing method will have problems such as missing hardware semantics and insufficient coverage of compilation configuration, which will cause the code generated by the large model trained by the code training data to be unusable in the real compilation environment.

[0003] That is, the prior art is difficult to generate effective training data in a complex scenario. SUMMARY

[0004] The present application provides a training data generation method and an electronic device to at least solve the problem that the prior art is difficult to generate effective training data in a complex scenario.

[0005] The present application provides a training data generation method, which comprises: processing symbols in a source code library and a compiled code library using a trained parse tree model to obtain a global symbol table, wherein the global symbol table represents the mapping relationship of symbol types, identifiers, definition positions and reference file lists, and the trained parse tree model is a parse tree model that has injected code query rules, and the code query rules at least represent the mapping relationship of rules, type matching patterns and example fragments; based on the global symbol table, the trained parse tree model is used to parse the source code library to obtain structured data; the structured data and the corresponding relevant technical specification documents are associated to generate a knowledge base, and the training data is generated according to the structured data and the knowledge base, and the relevant technical specification documents are use specification documents related to the source code library and the compiled code library.

[0006] The present application also provides an electronic device comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs comprise a method for executing any one of the methods.

[0007] Through the precise symbol tracking and cross-file reference resolution of the post-training parse tree model generated by the present application, the structured data more accurately reflects the usage and semantic relationships of symbols in the codebase, providing a solid foundation for subsequent training data generation. Since the model has injected code query rules for specific domains such as UEFI / BIOS, it can more effectively identify and process domain-specific symbols and syntax structures, generating higher-quality training data. By associating structured data with technical specification documents, the generated knowledge base contains official definitions and functional descriptions of code symbols, which can greatly enhance the understanding of underlying system code by large models. During training, the model not only learns code structure and syntax but also understands hardware operation semantics and protocol specifications behind the code, enabling it to generate more reasonable code completion suggestions. The construction of the global symbol table takes into account conditional compilation and multi-architecture support, enabling the generated training data to cover different compilation configurations and architectures. When the large model is trained, it will be able to better understand and generate code snippets that meet conditional compilation requirements and adapt to multiple hardware platforms, enhancing the model's generalization ability and practicality. Therefore, the problem of existing solutions being difficult to generate effective training data in complex scenarios can be solved. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0009] Figure 1 A flowchart of a training data generation method provided by an embodiment of the present application;

[0010] Figure 2 A schematic diagram of the construction process of the global symbol table provided by an embodiment of the present application;

[0011] Figure 3 A schematic diagram of the preprocessor flow of the compiler provided by an embodiment of the present application;

[0012] Figure 4 A schematic diagram of the cross-file code parsing flow provided by an embodiment of the present application;

[0013] Figure 5 A schematic diagram of the code completion training data generation flow provided by an embodiment of the present application;

[0014] Figure 6 A structural block diagram of a training data generation device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0016] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0017] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0018] The embodiments of this application provide a method for generating training data, and the method is described in detail in conjunction with the execution flow of the training data generation method.

[0019] like Figure 1 As shown, the method includes the following steps:

[0020] Step S201: The symbols in the source code library and the compiled code library are processed using the trained parse tree model to obtain a global symbol table. The global symbol table represents the mapping relationship between symbol type, identifier, definition location and reference file list. The trained parse tree model is a parse tree model with injected code query rules. The code query rules at least represent the mapping relationship between rules, type matching patterns and example fragments.

[0021] Symbols can include macros, functions, global variables, structures, and file inclusions.

[0022] Step S202: Based on the global symbol table, the source code library is parsed using a trained parse tree model to obtain structured data;

[0023] Step S203: Associate structured data with corresponding technical specification documents to generate a knowledge base, and generate training data based on the structured data and the knowledge base. The relevant technical specification documents are usage specification documents related to the source code library and the compiled code library.

[0024] In the above steps, through the accurate symbol tracking and cross-file reference resolution of the trained parse tree model, the generated structured data more accurately reflects the usage and semantic relationships of symbols in the codebase, providing a solid foundation for the subsequent generated training data. Since the model has injected code query rules specific to certain domains (such as UEFI / BIOS), it can more effectively identify and handle domain-specific symbols and syntax structures, thereby generating higher-quality training data. By associating the structured data with technical specification documents, the generated knowledge base contains official definitions and functional descriptions of code symbols, which can greatly enhance the large model's understanding of underlying system code. During training, the model not only learns code structure and syntax but also understands hardware operation semantics and protocol specifications behind the code, enabling it to generate more reasonable code completion suggestions. The construction of the global symbol table takes into account conditional compilation and multi-architecture support, allowing the generated training data to cover different compilation configurations and architectures. When the large model is trained, it will be able to better understand and generate code snippets that meet conditional compilation requirements and adapt to multiple hardware platforms, enhancing the model's generalization ability and practicality. Therefore, the problem of existing solutions being difficult to generate effective training data in complex scenarios can be solved.

[0025] In an embodiment of the present application, a trained parse tree model is used to process symbols in the source code library and the compiled code library to obtain a global symbol table, including: using the trained parse tree model to process the symbols of the source code library to obtain source code multi-dimensional association relationships, the source code multi-dimensional association relationships at least representing the mapping relationship of macro definition information and the logical structure of conditional compilation, the macro definition information including macro name, parameter list and expansion rule; storing the symbols of the compiled code library with different compilation configurations in separate directories respectively; associating the symbols of each independent directory with the corresponding line numbers to obtain compiled code association relationships; and generating a global symbol table based on the compiled code association relationships and the source code multi-dimensional association relationships.

[0026] The present application also provides a specific use scenario for generating a global symbol table: in a large firmware project (such as UEFI / BIOS), the development team faces serious code understanding and maintenance difficulties. The project is filled with a large number of conditional compilation instructions, macro definitions, and complex dynamic symbol generation, making the code readability very low. New developers often need to spend weeks or even months to understand and adapt to the codebase. In addition, due to the special nature of firmware code, involving hardware operations and protocol calls, traditional code generation and completion tools often cannot provide effective assistance, resulting in low development efficiency.

[0027] The specific use scenario of generating a global symbol table is as follows: the system uses the trained parse tree model to parse the source code library, automatically extracts macro definition information, including the name of the macro, the list of accepted parameters, and the expansion rule of the macro. The model can capture information such as preprocessor directives and build a logical tree for conditional compilation, recording the information "valid under what compilation conditions" for each macro. The system iterates through different compilation configurations, calls the compiler to generate preprocessed code, and stores these codes in separate directories. During preprocessing, the #line directive information is preserved, and the original line number of each symbol in the source code is recorded to ensure traceability between the preprocessed code and the original code location. By establishing a correspondence between the multi-dimensional association relationship of the source code and the association relationship of the compiled code, the system can accurately track the dynamic symbols after macro expansion under different compilation configurations. The global symbol table not only contains the definition and reference of the symbol, but also adds the state information of conditional compilation, ensuring that the meaning and usage of the same symbol in different compilation environments are clear and unambiguous.

[0028] The beneficial effects of generating a global symbol table in a specific use scenario are as follows: developers can quickly query macro definition and conditional compilation information, reducing the time spent on searching and testing in the code, greatly improving the efficiency and accuracy of code understanding; the existence of the global symbol table supports real-time collaboration, new members can quickly get started and understand the scope and calling logic of each symbol in the code library, reducing communication costs and promoting effective collaboration among teams. Integrating the global symbol table with conditional compilation awareness into the code generation tool enables the completed and generated code snippets to automatically adapt to different compilation configurations, improving the applicability and correctness of the code. Through accurate macro definition and conditional compilation information, symbol conflicts and code errors caused by improper compilation configurations can be effectively avoided, reducing the potential error risk in the project; the overall scheme can reduce the time spent by developers on code understanding and debugging, thereby accelerating the development progress of the firmware project and shortening the product launch time; systematic parsing and association help to find potential problems in the code, such as undefined symbol references and mismatched parameter types, thereby improving the quality and stability of the code.

[0029] In an embodiment of the present application, a trained parse tree model is used to process the symbols of the source code library to obtain the multi-dimensional association relationship of the source code, including: using the trained parse tree model to process each symbol of the source code library to construct a plurality of first syntax trees; traversing the first syntax tree according to the code query rule in the trained parse tree model to extract relevant information, including macro definition information and the logical structure of conditional compilation; based on the relevant information, constructing the multi-dimensional association relationship of the source code.

[0030] Specific use cases of code optimization and refactoring in UEFI firmware development: In UEFI firmware development, the codebase is often very large and complex, filled with macro definitions, conditional compilation directives, and hardware-specific code logic. When the development team needs to optimize or refactor the firmware, understanding the runtime effects of these macro definitions and conditional compilations becomes crucial, as it directly impacts the portability, maintainability, and performance of the code. For example, different hardware platforms may require different configuration options, and macro definitions and conditional compilations can ensure that the firmware runs correctly on different platforms, but this also increases the difficulty of code understanding. By using a trained parse tree model to process the symbols in the source codebase, multiple first syntax trees are constructed, and then the syntax trees are traversed according to the code query rules contained in the model, and the developers can automatically extract the macro definition information and the logical structure of the conditional compilation. Based on this information, the multi-dimensional association relationship of the source code is further constructed, including the association between macro definitions and their effective conditions, the logical relationship between conditional compilation branches, and the dependency relationship between code elements.

[0031] The beneficial effects of the specific use cases of code optimization and refactoring in UEFI firmware development: The automated syntax tree construction and information extraction greatly reduce the workload of developers manually parsing and understanding conditional compilations and macro definitions, allowing them to quickly grasp the structure of the codebase and improve development efficiency. When performing code refactoring, developers can rely on accurate macro definition and conditional compilation structure information to ensure that the compilation configuration on different hardware platforms is not broken, avoiding the introduction of difficult-to-debug errors. By constructing multi-dimensional code association relationships, the development team can more easily identify redundancies or potential performance bottlenecks in the codebase, perform targeted code optimization, and maintain code consistency and readability. When developing new features or adapting to new hardware platforms, relevant macro definitions and conditional compilation instructions can be quickly located and updated, thereby accelerating the firmware development and iteration process.

[0032] In an embodiment of the present application, the symbols of each independent directory are associated with the corresponding line numbers to obtain a compiled code association relationship, including: using a trained parse tree model to process the symbols of each independent directory to construct multiple second syntax trees; traversing each second syntax tree to extract actual symbol information and record the corresponding line numbers, the actual symbol information including function definitions and calls, variable declarations and uses, type instantiation and reference relationships; based on the actual symbol information and the corresponding line numbers, a compiled code association relationship is constructed.

[0033] In particular, the trained parse tree model can more accurately identify and process complex structures in the firmware code, such as nested conditional pre-compiler directives, dynamic symbol generation, and multi-version definitions. This enables the second syntax tree (i.e., the syntax tree of the compiled code) to more accurately reflect the actual syntax structure of the code, thereby improving the accuracy of symbol information extraction. By constructing the second syntax tree, the system can more effectively track symbol reference and definition relationships across files. In firmware development, the definition of a function or variable can be scattered across multiple files, and conditional compilation makes such reference relationships more complex. Based on the processing of the second syntax tree, these dependencies can be automatically identified and processed to ensure that the generated compiled code association relationship is complete and accurate. The actual symbol information extracted from the second syntax tree, including function calls, variable declarations, and type instantiations, is crucial for generating high-quality training data. By recording the actual symbol information and corresponding line numbers, the system can construct more accurate code completion training samples that contain complete context information and accurate completion targets, which helps to improve the performance of large models on the firmware code completion task.

[0034] In an embodiment of the present application, based on the compiled code association relationship and the source code multi-dimensional association relationship, a global symbol table is generated, including: in the process of generating the global symbol table according to the compiled code association relationship and the source code multi-dimensional association relationship, constructing a line number association relationship, wherein the line number association relationship at least represents the mapping relationship between the symbols in the source code library, the symbols in the compiled code library, and the corresponding line numbers; generating a macro association relationship according to the compiled code association relationship and the source code multi-dimensional association relationship, wherein the macro association relationship represents the mapping relationship between the macros in the source code library, the macros in the compiled code library, and the corresponding trigger parameters, and the trigger parameters are used to generate the macros in the compiled code library from the macros in the source code library; performing conditional fusion processing based on the source code multi-dimensional association relationship, the macro association relationship, and the line number association relationship to generate the global symbol table.

[0035] In particular, the line number association ensures the correspondence between the line numbers of the preprocessed code and the original source code, which is crucial for accurate symbol tracking. When the parser locates a certain symbol in the preprocessed code, it can backtrack to the exact location in the source code through the line number mapping, thereby obtaining richer contextual information, including the original context of macro definitions and the state of conditional compilation, which helps the parser more accurately understand the code semantics. The establishment of macro association helps to handle the expansion results of macro definitions under different conditions. By recording the specific mapping relationship between macros and preprocessed symbols, including the triggering parameters of macros, it can be determined which macro expanded a particular symbol under what conditions, which is crucial for correctly managing macro definitions in the global symbol table, avoiding confusion and conflicts of macro definitions in multiple compilation configuration environments. Conditional fusion processing allows the system to handle different definitions of the same symbol under multiple compilation configurations. Through the fusion of conditional expressions and compilation flags, the system can automatically identify the correct definition of the symbol under different environments and record it reasonably in the global symbol table, which is a key step in building a conditional compilation-aware global symbol table. This processing ensures that the large model can understand and adapt to complex and variable firmware development environments, improving its code generation capabilities on different hardware platforms. When generating training data, the use of accurate line number association and macro association ensures that the use of symbols and macros in the training samples is based on the actual compilation results, thereby avoiding inefficient or erroneous training data caused by parsing errors or symbol ambiguity. Conditional fusion processing ensures that the training data covers all possible compilation configurations, enhancing the model's generalization ability.

[0036] In an embodiment of the present application, in the process of generating a global symbol table based on source code multi-dimensional association, macro association and line number association, the method further comprises: in the case where the same symbol has multiple different definitions under the same condition, marking the corresponding symbol as a conflict symbol, and the conflict symbol represents that the symbol does not participate in the process of generating training data.

[0037] Specifically, the conflicting symbols can originate from the same symbol being assigned different meanings in different files, or semantic ambiguity caused by macro definition polymorphism in the preprocessing phase. If not handled, the generated training data will contain erroneous or confusing symbol information, which will mislead the model's learning and reduce the accuracy of the model's predictions. Excluding conflicting symbols can reduce the noise in the training data, making the model focus more on learning symbols that have consistent behavior across all compilation configurations. In this way, the model can better understand and generate code that works on different hardware platforms when faced with new code snippets, enhancing the model's versatility and adaptability. The handling of conflicting symbols is usually complex and requires additional computational resources to resolve their specific meanings under different conditions. By pre-marking and excluding these conflicting symbols, the preparation process of the training data can be simplified, unnecessary computational overhead can be reduced, and the speed and efficiency of model training can be improved.

[0038] In an embodiment of the present application, based on the global symbol table, the source code library is parsed using the trained parse tree model to obtain structured data, including: using the trained parse tree model to process each symbol of the source code library to generate a third syntax tree to identify key code nodes, including function call nodes, macro call nodes, variable use nodes, and structure member access nodes; performing local definition checking on the identified key code nodes, and marking the symbol as an external reference if the corresponding symbol of the key code node is not defined locally; extracting definition location information from the global symbol table according to the symbol marked as an external reference, the definition location information including a target file name and a specific line number; and extracting the context of the key code nodes according to the definition location information, the global symbol table, and the third syntax tree to obtain the structured data.

[0039] Specifically, the trained model can more accurately identify key functional units in the code, such as function calls and macro calls, which is particularly important for complex code in the firmware field, as these fields often use specific macros and pre-compiled instructions. Through context extraction, not only can the nodes themselves be identified, but also their scope of action and dependencies in the code can be understood, which is essential for understanding and generating code fragments that span files. Marking symbols that are not defined locally helps to explicitly know when more information needs to be introduced from other files or externally in subsequent parsing processes. Using a global symbol table can track the definition and reference of symbols across multiple files and compilation configurations, solving the problem of symbol ambiguity and dynamics that is difficult to handle in traditional methods. When processing external references, the global symbol table determines the definition location and conditional compilation state of the symbol, ensuring that the generated training data is consistent with the code behavior in the actual compilation environment. Combined with the results of document parsing, relevant hardware knowledge and protocol specifications can be injected into the code, enhancing the semantic understanding ability of the training data and making the generated code more consistent with the professional needs of the firmware field.

[0040] In an embodiment of the present application, the training data is generated according to the structured data and the knowledge base, comprising: extracting a plurality of logical blocks from the structured data, the logical blocks including function definitions, structure definitions, and loop definitions; cutting each logical block into code fragments suitable for training according to logical boundaries, the logical boundaries including function entrances and loop starting points; associating target text information in the relevant technical specification document with the code fragments to construct a target association relationship; and deleting part of the code in the target association relationship to generate the training data.

[0041] Specifically, by associating semantic information in the technical specification document with the code fragments, the training data not only includes the code itself, but also contains the context knowledge that the code depends on, such as the function of the function, the purpose of the structure, and the logic of the loop, which helps the model to more accurately understand the semantics and functions of the code, thereby improving the accuracy of code completion and generation. The cutting of logical blocks ensures that each training sample is a self-contained code unit with clear boundaries and complete context. This avoids the problem of incomplete code fragments or excessive irrelevant information in traditional methods, making the training data more refined and targeted. Firmware development often involves multiple compilation configurations and platform-specific code. Through conditional compilation awareness and dynamic loading of dependencies for code fragments, training data can cover a wider range of compilation scenarios and hardware differences, making the model more stable and reliable when encountering complex conditions in actual applications. The division and deletion of logical blocks generate training data that can guide the model to quickly locate the demand points for code completion, such as incomplete function parameters and missing structure members, thereby improving the speed and efficiency of the model in completing the code.

[0042] In an embodiment of the present application, the method further comprises: in the process of associating the target text information in the related technical specification document with the code snippet to construct the target association relationship, in the case where the target string is determined to exist, calculating the name similarity by using the edit distance algorithm to determine the target name, the target string being a string lacking of identification, and the target name being at least part of all names; and matching the parameter type of the target string based on the target name to match the target name with the corresponding type.

[0043] Specifically, the edit distance algorithm (such as Levenshtein distance) can effectively handle the spelling differences or incomplete matches of the target string and the identifiers in the code snippet, so as to more accurately identify the target name. For example, “MmioRead32” described in the document may be abbreviated as “MmioRd32” in the code, and by calculating the edit distance, the similarity of the two names can be identified to help match the correct target name. The parameter type matching ensures the semantic consistency between the code snippet and the document description. By comparing the parameter types of the target name in the document and the code snippet, the correctness of the name can be further confirmed, and the false association caused by similar names but mismatched parameter types can be avoided. The combination of edit distance and parameter type matching can improve the robustness of the training data in handling non-standard naming, abbreviations or spelling variants, so that the model can still maintain stable performance when encountering various naming habits in actual development. By calculating the name similarity, more information related to the code snippet can be extracted from the document, even if the name described in the document is not completely consistent with the name actually used in the code. This increases the coverage of the document knowledge base, so that more technical details can be learned and utilized by the model, improving the completion quality and semantic understanding ability of the model. The edit distance algorithm supports fuzzy matching, and even if there is some difference between the target string and the accurate name, the closest matching item can be found. This provides flexibility for handling new naming rules or code styles that may appear in the future, enhancing the scalability and adaptability of the system. The parameter type matching provides another layer of verification to ensure that even in similar names, the most suitable associated object can be filtered out, improving the precision of the system.

[0044] UEFI / BIOS code-specific query rule definition: Tree-sitter (i.e., parse tree model) is a general parser generation tool and incremental parsing library for efficiently parsing source code and generating abstract syntax trees, which can be used in a python (a widely used high-level programming language) programming environment. However, in addition to using a large amount of C language syntax, UEFI / BIOS code also contains custom data structures, protocols, and complex preprocessing macros. Directly parsing the code using Tree-sitter will have the problem of incomplete or incorrect parsing trees, so additional query rules need to be added to the Tree-sitter syntax tree query rules to cover the syntax rules of UEFI / BIOS code as much as possible. By adding new scm files under the. / tree-sitter-c / queries path, the definition of UEFI / BIOS code-specific query rules can be achieved, and part of the query rule key content is shown in Table 1.

[0045] Table 1

[0046]

[0047] Establish a global symbol table: The global symbol table is the basis for cross-file code parsing, and through the global symbol table, the reference and dependency relationship between different files can be queried. The symbol types include Macro, Function, Variable, Struct, Include, etc. The identifier is the specific implementation of the symbol type, such as macro name, global variable name, function name, etc. The definition location contains the definition file path of the identifier and the specific line number in the file. The reference file list contains information about where the identifier is referenced in the code file (including the line number of the referenced code).

[0048] As shown in Figure 2 , the construction process of the global symbol table includes the following steps:

[0049] Step 1: Extracting the raw source code metadata without preprocessing: The raw source code without preprocessing may contain preprocessor directives such as #if, #ifdef, #define, etc. This step is to extract relevant semantic meta-information (macro definitions, conditional compilation structures, type declaration existence conditions, etc.) to provide traceable source context and existence conditions for subsequent symbols. Input: Raw code file without preprocessing; Tree-sitter query rules (macros, conditional compilation, type declarations, etc.). Processing flow: 1) Use Tree-sitter to build an abstract syntax tree (AST) for the raw code, clearly showing the hierarchical relationship and structural position of each code element, laying the foundation for subsequent precise information extraction. 2) Traverse the AST according to the predefined query rules (Tree-sitter code query rules) to systematically extract key information. This includes the complete information of macro definitions (macro name, parameter list, expansion rule, type declaration, and parent type relationship), and the logical structure of conditional compilation (if / elif / else nesting chain). 3) Establish multi-dimensional associations: maintain a stack of #if / #ifdef / #elif / #else (#if, #ifdef, #elif, #else, and #endif are preprocessor directives used for conditional compilation. These directives allow the compiler to include or exclude code fragments based on specific conditions, enabling more flexible code management) while traversing, record the "effective under what compilation conditions" condition (condition) tag (such as A&&!B) for each macro, bind the macro definition with its effective condition context; record the typedef pointer, struct / union field type, and enum member (typedef, struct / union, and enum are commonly used keywords for defining new types); at the same time, record the effective conditions for declarations / fields to facilitate the identification of differences under different configurations, treat if-elif-else as a chain to save the conditions and positions of each branch, and construct the complete logical chain of conditional branches. Identify parameterized macros (macros containing ## paste symbols) and record their naming rules, which will be the key for macro reverse matching in the subsequent steps; 4) Structured data storage: store all extracted and associated meta-information in JSON format for persistent storage. This standardized storage method ensures data readability and facilitates subsequent processing, providing reliable data flow for the entire system; Output: macro definition table, conditional compilation decision tree / existence condition index, type declaration and dependency relationship metadata. Output: macro definition table, conditional compilation decision tree / existence condition index, type declaration and dependency relationship metadata.

[0050] Step 2: Code preprocessing expansion: Input: Code file library. Processing flow: 1) Collect compilation configuration information from various build systems, supporting multiple mainstream build systems, ensuring the integrity and accuracy of each compilation configuration; 2) For each compilation configuration, call the GCC compiler (GNU Compiler Collection, a compiler suite) for preprocessing (compilation) operations, while preserving key #line mapping information, which records the correspondence between compiled code and unprocessed source code; 3) Store the preprocessing results of different compilation configurations in separate directories to avoid interference. This separate storage method ensures that the code state under different configurations can be accurately distinguished during subsequent analysis. Output: Independent directories of "preprocessed compiled code" divided by compilation configuration and #line mapping.

[0051] The GCC compiler preprocessing process is shown in Figure 3 , which creates an output root directory, iterates through the compilation configurations, creates configuration output directories, iterates through the code files, and preprocesses and stores the preprocessing results.

[0052] Step 3: Analyze preprocessed code to extract symbol information: Based on the preprocessed source code directories generated for each compilation configuration and #line mapping, analyze the "preprocessed code" under each configuration, accurately extract actual symbols (functions / variables / type instances) and reference relationships, and form a "configuration-based" symbol list. Input: Preprocessed source code directories generated for each compilation configuration; corresponding #line mapping. Processing flow: 1) For each preprocessed source code in an independent directory, use Tree-sitter to build a dedicated abstract syntax tree (AST, Abstract Syntax Tree, a tree-like representation used to express source code structure); 2) Systematically traverse the AST to comprehensively extract actual symbol information from the code, including function definitions and calls, variable declarations and uses, type instantiation and reference relationships, etc.; 3) Record metadata information for each discovered symbol, including the symbol's location in the preprocessed code (including #line backtracking information for tracing the original location), symbol type classification, definition, and complete reference list, etc., which will serve as important basis for subsequent symbol association and merging; Output: Symbol / relationship list and AST fragments (including traceable location information) divided by compilation configuration.

[0053] Step 4: Symbol association and merging: Since the goal in building a global symbol table is to obtain the symbol table information of the original code, the symbol resolution performed on the preprocessed code has the following issues: macro expansion interference, for example, the dynamic symbol name generated by a macro function needs to be mapped back to the original macro definition; code location offset, the line numbers of the preprocessed code are inconsistent with the original code. Conditional compilation multiple versions, the same symbol may have different definitions under different compilation configurations. Align the real symbols parsed from the preprocessed compiled code with the macro / conditional context of the original source code that has not been preprocessed, and generate versioned global symbol entries with conditions. Input: 1) macro / conditional compilation / type existing conditional meta information extracted from the original source code metadata that has not been preprocessed; 2)#line mapping (and configuration tags) of code preprocessing expansion; 3) symbol / relationship list obtained by parsing the preprocessed code according to the configuration. Processing flow: 1) Position backtracking: Since preprocessing changes the code structure, the information recorded by the#line instruction is used to accurately map the symbol position in the preprocessed code back to the position in the original source file. It is like giving each symbol a label "I come from which file and which line", ensuring that the true origin of the symbol can be found during subsequent analysis; 2) Macro reverse matching: Reverse the original macro from the expansion result (compiled code library). When you see the expanded symbol, you can identify that it is generated by which macro and what parameters. This process relies on the macro definition information extracted from "extracting metadata from the original source code that has not been preprocessed", especially the naming rules of parameterized macros (such as macros containing##paste symbols), and through pattern matching, the expanded symbol is matched with the original macro definition, while confirming that the macro is indeed available under the current compilation conditions; 3) Conditional fusion: Create a file for a polysemous symbol. The same symbol name may have different definitions under different compilation conditions, and independent records need to be created for each case, and the effective conditions of each record are marked. 4) Conflict detection and default processing: Solve the conflict problem of symbol definition. When the same symbol has multiple different definitions under the same conditions, mark it as a conflict; for symbol references that do not have explicitly specified use conditions, give them a reasonable default configuration to avoid ambiguity; Output: Conditional compilation-aware global symbol table, as shown in Table 2.

[0054] Table 2

[0055]

[0056] Wherein, the definition position of PTR_SIZE is src / main.c:2 (Conditions: CONFIG_64 BIT=1→8, default→4), indicating that PTR_SIZE is defined at line 2 in src / main.c, and the value is 8 when CONFIG_64 BIT=1, and the value is 4 under the default configuration.

[0057] The reference file list of DEVICE_NAME in src / devices.c: 15 (Device_0xFE) and src / log.c: 20 (Device_0xFF) indicates that the DEVICE_NAME macro is expanded as Device_0xFE at line 15 of src / devices.c and as Device_0xFF at line 20 of src / log.c. The meanings of other contents in the table are similar to the above example.

[0058] Cross-file code parsing: After building the global symbol table, cross-file parsing of the UEFI / BIOS code library can be performed to generate code parsing structured data. In order to generate training samples covering different compilation scenarios, all compilation configurations are traversed, and the code files in the code library are parsed under different compilation configuration scenarios, such as Figure 4 As shown in the following table:

[0059] Code parsing and external reference identification: First, use the Tree-sitter parser to generate an abstract syntax tree (AST) for C language source files (code repository). Systematically identify the following key code nodes: function call nodes, macro call nodes, variable usage nodes, structure member access nodes, and protocol GUID (Globally Unique Identifier, a standard for generating unique identifiers in computer systems) reference nodes. Record the #ifdef instruction state through a conditional expression stack to ensure that the parser correctly understands the conditional compilation results. Second, perform local definition checking on the identified symbol nodes by comparing the defined symbols in the current file with the reference symbols. Mark the symbols: if the symbol is defined locally, mark it as an internal reference, and if the symbol is not defined locally, mark it as an external reference. In particular, for symbols generated by macros (such as DEVICE_NAME(123) expanded as Device_123), the system will perform reverse tracking of the macro through the global symbol table, tracing back to the original macro definition, to avoid misjudging the macro-expanded symbols as undefined symbols.

[0060] Dynamic loading of syntax tree fragments: Combine the recognized external reference symbol name and type information into a query condition (query global symbol table, get definition, extract syntax tree fragment, context load, and finally generate structured data), perform an accurate query in the global symbol table according to the current compilation configuration, obtain the symbol definition location information, including the target file name and specific line number. According to the symbol type, different code fragment extraction strategies are adopted: function symbol: extract the complete function body, including the function signature, parameter declaration, local variable definition, and complete function implementation, to ensure the integrity of the call relationship, local variable scope, and control flow; structure / union symbol: extract the complete type definition, including all field definitions and possible conditional compilation fields, to ensure the consistency of data structure layout; header file dependency: when the code fragment references external types, macros, or declarations, synchronize the mounting of related header file fragments to maintain the integrity of the dependency relationship; according to the parsed target file name and line number, locate the symbol node and extract its context code fragment (code fragment is a function or structure) based on the AST syntax tree of the target file. Mount the extracted code corresponding syntax tree fragment and its header file to the current parsing environment.

[0061] Document parsing: After completing the code structure parsing, in order to enhance the semantic understanding ability of the training data, it is necessary to associate the code parsing result with related technical specification documents. By parsing UEFI specification documents, PCIe specification documents (Peripheral Component Interconnect Express, important reference materials for UEFI / BIOS development), Intel EDS register (External Design Specification register document, important reference materials for UEFI / BIOS development) specification documents, etc. technical documents, extract semantic information related to symbols, functions, and protocols in the code, and provide more rich context knowledge for large models.

[0062] Document parsing target and association mechanism: The main goal of document parsing is to establish semantic association between code symbols and technical specifications. By providing official specification definitions and function descriptions for recognized function names, variable names, and protocol GUID symbols, the symbol semantics are enhanced. At the same time, the UEFI API calls in the code are accurately matched with the function signatures, parameter descriptions, and return value definitions in the official documents to provide authoritative API specification support for code understanding. For register access operations in the code, the hardware specification documents are parsed to provide function descriptions and configuration explanations to help understand the intent of the underlying hardware operations. In addition, the protocol implementation in the code needs to be matched with the corresponding standard protocol specification to ensure the accuracy of protocol-level semantic understanding.

[0063] Classification document parsing strategy: According to the structural characteristics of different technical documents, targeted information extraction strategies are adopted: UEFI specification document parsing: For UEFI specification documents, the system extracts the text information near the keywords such as EFI_STATUS and EFIAPI (referring to the content of the previous and next pages where the keywords are located) through keyword positioning technology, thereby obtaining the official explanation of functions and protocols. When dealing with protocol-related content, the system will identify and extract protocol GUID definitions and establish accurate correspondence between GUID and protocol functions. At the same time, the system will also parse API specification information such as function signature, parameter type, and return value description to provide complete specification support for API calls in the code. Intel EDS register document parsing: For the characteristics of Intel EDS register documents, regular matching technology is used to identify register names in the "all uppercase letters + numbers" pattern. After identifying the register, detailed parameter information such as bitfield definition, function description, and configuration options of the register is extracted from the corresponding table. In this way, the register operations in the code can be accurately associated with specific hardware functions, providing hardware specification-level support for register-level code understanding. PCIe specification document parsing: For PCIe specification documents, the focus is on extracting the definition and function description of configuration space registers, which are crucial for understanding PCIe device initialization and configuration code. At the same time, the data structures and communication mechanisms related to the PCIe protocol layer are also parsed to provide protocol-level specification support for PCIe driver code understanding. In addition, the specification definition of PCIe error handling is also a focus of parsing, which helps to understand the complex error handling logic in firmware. Document knowledge base construction: Use different parsing strategies and document parsing tools to structure the PDF document, and finally build a comprehensive document knowledge base. This knowledge base mainly includes a symbol-definition mapping table to establish accurate correspondence between code symbols and official document descriptions. At the same time, an API specification library is established to store function signatures, parameter descriptions, usage examples, and other detailed information. For register operations, a register specification library is constructed, containing register address, bitfield definition, function description, and other key information. In addition, a protocol specification library is established to store protocol GUID, interface definition, implementation requirements, and other protocol-level specification content.

[0064] Code completion training data automatic production: After obtaining the code parsing results and document parsing results, code completion training data can be generated, as shown in Figure 5

[0065] ​Code slicing: intelligently extract logical blocks such as functions, structure definitions, and loops from code parsing results, and cut the code into code segments suitable for training according to logical boundaries (such as function entry, loop start, etc.). In the cutting process, the system will automatically identify and inject dependent external code segments. Ensure that each code segment has complete context information, and provide sufficient semantic basis for subsequent training.

[0066] Document knowledge injection: key text information such as protocol description, register definition, and GUID mapping parsed from PDF is associated with code symbols through two matching strategies. For cases with explicit identifiers, use precise matching to directly map document content to code symbols (functions, variables) through GUID, function name, etc. For cases lacking explicit identification, use "Levenshtein distance (edit distance, the minimum number of editing operations required to convert one string to another) to calculate name similarity + parameter type check" to perform fuzzy matching between text and code symbols, ensuring that document knowledge can be accurately associated with the corresponding code elements. After matching, the system inserts document information into the code segment in the form of standardized annotations, achieving organic integration of knowledge.

[0067] Training data generation: based on the code segments enhanced by the document knowledge base, the system constructs an intelligent training sample generation mechanism. The input part is designed as a complete text sequence containing code context and enhanced information, and the output part is designed as the target code segment to be completed. The system uses random or heuristic strategies to select appropriate breakpoint positions in the code segment, such as incomplete function parameters, missing structure members, and undeclared variables, etc. The code before the breakpoint is used as the input context, and the subsequent part is used as the target completion sequence, forming a high-quality training sample pair.

[0068] Data verification and filtering: to ensure the quality and availability of training samples, the system establishes a multi-level verification mechanism. First, use the global symbol table obtained in step two to verify the legality of the code to be completed, check if the function exists, the parameter type matches, and other key elements, and automatically filter out abnormal samples that cannot be parsed. Second, compile and test the complete code of the generated sample, run the GCC preprocessor in a sandbox environment to ensure the correctness of the code syntax and the feasibility of compilation. Through this strict verification process, the system can output high-quality and reliable training data sets. Finally, store the generated training samples in JSON format, which mainly includes input text, completion target, metadata (such as dependent files, conditional compilation tags), etc. The fields and features of the generated code training data are shown in Table 3.

[0069] Table 3

[0070]

[0071] The application generates a global symbol table containing a compilation condition tag, an original definition position and a reference list by double-stage parsing (original code + pre-processed code) and symbol association and merging, realizes dynamic tracking of symbols and management of condition compilation versions. Cross-file reference association during code parsing is realized by external symbol query, syntax tree pruning (selecting a matching syntax tree version according to the compilation configuration of the current file) and dynamic loading of code fragments. Unstructured hardware documents (UEFI specification, chip manual) are converted into semantic constraints that can be embedded in code by document information extraction, knowledge alignment algorithm (precise matching + fuzzy matching) and knowledge injection mechanism (inserting document description into code fragments in a standardized annotation format).

[0072] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and necessary general hardware platforms, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0073] The embodiment of the application further provides a training data generation device. As shown in the figure, the device comprises: Figure 6

[0074] The first processing unit 61 is configured to process symbols in the source code library and the compiled code library by using the trained parse tree model, to obtain a global symbol table, wherein the global symbol table represents a mapping relationship of symbol types, identifiers, definition positions and reference file lists, and the trained parse tree model is a parse tree model that has injected code query rules, and the code query rules at least represent a mapping relationship of rules, type matching modes and example fragments;

[0075] The second processing unit 62 is configured to parse the source code library by using the trained parse tree model based on the global symbol table, to obtain structured data;

[0076] The third processing unit 63 is configured to associate the structured data and the corresponding related technical specification document, to generate a knowledge base, and to generate training data according to the structured data and the knowledge base, and the related technical specification document is a use specification document related to the source code library and the compiled code library.

[0077] ​In an embodiment of the present application, the first processing unit comprises: a first processing module configured to process the symbols of the source code library using the trained parse tree model to obtain source code multi-dimensional association relationship, the source code multi-dimensional association relationship at least representing a mapping relationship of macro definition information and a logical structure of conditional compilation, the macro definition information including a macro name, a parameter list and an expansion rule; a second processing module configured to store the symbols of the compiled code library of different compilation configurations in independent directories respectively; a third processing module configured to associate the symbols of each independent directory with corresponding line numbers to obtain a compiled code association relationship; and a fourth processing module configured to generate a global symbol table based on the compiled code association relationship and the source code multi-dimensional association relationship.

[0078] In an embodiment of the present application, the first processing module comprises: a first processing submodule configured to process each symbol of the source code library using the trained parse tree model to construct a plurality of first syntax trees; a second processing submodule configured to traverse the first syntax trees according to a code query rule in the trained parse tree model to extract relevant information, the relevant information including macro definition information and a logical structure of conditional compilation; and a third processing submodule configured to construct the source code multi-dimensional association relationship based on the relevant information.

[0079] In an embodiment of the present application, the third processing module comprises: a fourth processing submodule configured to process the symbols of each independent directory using the trained parse tree model to construct a plurality of second syntax trees; a fifth processing submodule configured to traverse each second syntax tree to extract actual symbol information and record corresponding line numbers, the actual symbol information including function definition and call, variable declaration and use, and type instantiation and reference relationship; and a sixth processing submodule configured to construct the compiled code association relationship based on the actual symbol information and the corresponding line numbers.

[0080] In an embodiment of the present application, the fourth processing module comprises: a seventh processing submodule configured to construct a line number association relationship in the process of generating the global symbol table according to the compiled code association relationship and the source code multi-dimensional association relationship, wherein the line number association relationship at least represents a mapping relationship of the symbols in the source code library, the symbols in the compiled code library and the corresponding line numbers; an eighth processing submodule configured to generate a macro association relationship according to the compiled code association relationship and the source code multi-dimensional association relationship, wherein the macro association relationship represents a mapping relationship of the macros in the source code library, the macros in the compiled code library and corresponding trigger parameters, the trigger parameters being used to generate the macros in the compiled code library from the macros in the source code library; and a ninth processing submodule configured to perform conditional fusion processing based on the source code multi-dimensional association relationship, the macro association relationship and the line number association relationship to generate the global symbol table.

[0081] In an embodiment of the present application, the ninth processing submodule comprises: a tenth processing submodule configured to, in the process of generating the global symbol table based on the multi-dimension association relationship of the source code, the macro association relationship and the line number association relationship, and in the case that there are multiple different definitions of the same symbol under the same condition, mark the corresponding symbol as a conflict symbol, and the conflict symbol represents that the symbol does not participate in the process of generating the training data.

[0082] In an embodiment of the present application, the second processing unit comprises: a fifth processing module configured to generate a third syntax tree by processing each symbol of the source code library using the parsed tree model after training, to identify key code nodes, and the key code nodes include function call nodes, macro call nodes, variable use nodes and structure member access nodes; a sixth processing module configured to perform local definition checking on the identified key code nodes, and mark the symbol as an external reference in the case that the corresponding symbol of the key code node is not defined locally; extract definition location information from the global symbol table according to the symbol marked as the external reference, and the definition location information includes a target file name and a specific line number; and a seventh processing module configured to perform context extraction on the key code nodes according to the definition location information, the global symbol table and the third syntax tree, to obtain the structured data.

[0083] In an embodiment of the present application, the third processing unit comprises: an eighth processing module configured to extract a plurality of logical blocks from the structured data, and the logical blocks include function definitions, structure definitions and loop definitions; a ninth processing module configured to cut each logical block into a code segment suitable for training according to a logical boundary, and the logical boundary includes a function entry and a loop starting point; a tenth processing module configured to associate the target text information in the related technical specification document with the code segment, to construct a target association relationship; and an eleventh processing module configured to delete part of the code in the target association relationship, to generate the training data.

[0084] In an embodiment of the present application, the device further comprises: a fourth processing unit configured to, in the process of associating the target text information in the related technical specification document with the code segment to construct the target association relationship, determine that there is a target string, calculate the name similarity using the edit distance algorithm to determine a target name in the case that there is a target string, and the target string is a string lacking of identification, and the target name is at least part of all names; and a fifth processing unit configured to match the parameter type of the target string based on the target name, to match the target name with the corresponding type.

[0085] The features of the embodiments of the training data generation device can be referred to the related descriptions of the embodiments of the training data generation method, which will not be repeated here.

[0086] The embodiment of the present application further provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-mentioned training data generation method embodiments.

[0087] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above-mentioned training data generation method embodiments when running.

[0088] In an example embodiment, the above-mentioned computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0089] The embodiment of the present application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps in any of the above-mentioned training data generation method embodiments.

[0090] The embodiment of the present application further provides another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the above-mentioned training data generation method embodiments.

[0091] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in a general manner in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0092] The above describes in detail the training data generation method and the electronic device provided by the present application. The principles and implementation manners of the present application are described by applying specific examples in this paper, and the above description of the examples is only used to help understand the method of the present application and its core idea. It should be pointed out that, for the ordinary skilled person in the technical field, without departing from the principles of the present application, some improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A method for generating training data, characterized in that, The method comprises the following steps: adopting a post-training parse tree model to process symbols of a source code library to obtain a source code multi-dimensional association relationship, the source code multi-dimensional association relationship at least representing a mapping relationship of macro definition information and a logical structure of conditional compilation, the macro definition information including a macro name, a parameter list and an expansion rule; storing symbols of compiled code libraries with different compilation configurations in independent directories respectively; associating the symbols of each independent directory with corresponding line numbers to obtain a compiled code association relationship; in a process of generating a global symbol table according to the compiled code association relationship and the source code multi-dimensional association relationship, constructing a line number association relationship, wherein the line number association relationship at least represents a mapping relationship of symbols in the source code library, symbols in the compiled code library and corresponding line numbers; generating a macro association relationship according to the compiled code association relationship and the source code multi-dimensional association relationship, wherein the macro association relationship represents a mapping relationship of macros in the source code library, macros in the compiled code library and corresponding trigger parameters, the trigger parameters being used to generate the macros in the compiled code library from the macros in the source code library; performing conditional fusion processing based on the source code multi-dimensional association relationship, the macro association relationship and the line number association relationship to generate the global symbol table, wherein the global symbol table represents a mapping relationship of a symbol type, an identifier, a definition position and a reference file list, the post-training parse tree model being a parse tree model having injected a code query rule, the code query rule at least representing a mapping relationship of a rule, a type matching mode and an example fragment; adopting the post-training parse tree model to process each symbol of the source code library to generate a third syntax tree, to identify key code nodes, the key code nodes including function call nodes, macro call nodes, variable use nodes and structure member access nodes; performing local definition checking on the identified key code nodes, and marking a symbol as an external reference in a case where the symbol corresponding to the key code node is not defined locally; extracting definition position information of the symbol marked as the external reference from the global symbol table, the definition position information including a target file name and a specific line number; performing context extraction on the key code nodes according to the definition position information, the global symbol table and the third syntax tree, to obtain structured data; associating the structured data with corresponding related technical specification documents to generate a knowledge base, and generating training data according to the structured data and the knowledge base, the related technical specification documents being use specification documents related to the source code library and the compiled code library.

2. The method of claim 1, wherein, adopting a post-training parse tree model to process symbols of a source code library to obtain a source code multi-dimensional association relationship, the source code multi-dimensional association relationship at least representing a mapping relationship of macro definition information and a logical structure of conditional compilation, the macro definition information including a macro name, a parameter list and an expansion rule; storing symbols of compiled code libraries with different compilation configurations in independent directories respectively; associating the symbols of each independent directory with corresponding line numbers to obtain a compiled code association relationship; in a process of generating a global symbol table according to the compiled code association relationship and the source code multi-dimensional association relationship, constructing a line number association relationship, wherein the line number association relationship at least represents a mapping relationship of symbols in the source code library, symbols in the compiled code library and corresponding line numbers; generating a macro association relationship according to the compiled code association relationship and the source code multi-dimensional association relationship, wherein the macro association relationship represents a mapping relationship of macros in the source code library, macros in the compiled code library and corresponding trigger parameters, the trigger parameters being used to generate the macros in the compiled code library from the macros in the source code library; performing conditional fusion processing based on the source code multi-dimensional association relationship, the macro association relationship and the line number association relationship to generate the global symbol table, wherein the global symbol table represents a mapping relationship of a symbol type, an identifier, a definition position and a reference file list, the post-training parse tree model being a parse tree model having injected a code query rule, the code query rule at least representing a mapping relationship of a rule, a type matching mode and an example fragment; ​ According to the code query rule in the trained parse tree model, the first syntax tree is traversed to extract relevant information, and the relevant information includes the macro definition information and the logical structure of the conditional compilation; Based on the relevant information, the source code multi-dimensional association relationship is constructed.

3. The method of claim 1, wherein, The symbol of each independent directory is associated with the corresponding line number to obtain a compilation code association relationship, including: The trained parse tree model is used to process the symbol of each independent directory to construct a plurality of second syntax trees; Each second syntax tree is traversed to extract actual symbol information and record the corresponding line number, and the actual symbol information includes function definition and call, variable declaration and use, type instantiation and reference relationship; Based on the actual symbol information and the corresponding line number, the compilation code association relationship is constructed.

4. The method of claim 1, wherein, In the process of condition fusion based on the source code multi-dimensional association relationship, the macro association relationship and the line number association relationship to generate the global symbol table, the method further includes: In the case that the same symbol has multiple different definitions under the same condition, the corresponding symbol is marked as a conflict symbol, and the conflict symbol represents that the symbol does not participate in the generation of the training data flow.

5. The method of claim 1, wherein, Generating training data according to the structured data and the knowledge base includes: Extracting a plurality of logical blocks from the structured data, the logical blocks including function definition, structure definition and loop definition; According to the logical boundary, each logical block is cut into a code segment suitable for training, and the logical boundary includes function entry and loop start point; The target text information in the related technical specification document is associated with the code segment to construct a target association relationship; Part of the code in the target association relationship is deleted to generate the training data.

6. The method of claim 5, wherein, The method further includes: In the process of associating the target text information in the related technical specification document with the code segment to construct a target association relationship, in the case that it is determined that there is a target string, a name similarity is calculated using an edit distance algorithm to determine a target name, the target string is a string lacking of identification, and the target name is at least part of all names; Based on the target name, the parameter type of the target string is matched to match the target name with the corresponding type.

7. An electronic device, comprising: Including: One or more processors, memories, and one or more programs, wherein the one or more programs are stored in the memories and configured to be executed by the one or more processors, and the one or more programs include a program for executing the method in any one of claims 1 to 6.

Images