A method and apparatus for producing multimodal data oriented towards program understanding

Abstract

This application provides a method and apparatus for producing multimodal data for program understanding, belonging to the field of computer technology. The method includes: parsing the source code and binary file of a target program to obtain a source code matching index, multiple binary matching parameters of each function, binary data of multiple modalities, and source code data of multiple modalities. The source code matching index represents the correspondence between each source code data and a source code matching key. For any function in the binary file, the source code matching index is queried sequentially according to the priority order of the multiple binary matching parameters. When the query is successful, multimodal fused data is obtained. Based on the fused data, a multimodal dataset corresponding to the data requirements is generated. This application obtains multimodal data from the low-level machine representation to the high-level semantic description through parsing, matching, and fusion, thereby obtaining a dataset and solving problems such as data integrity, modal richness, and cross-modal pairing.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method and apparatus for producing multimodal data oriented towards program understanding. Background Technology

[0002] With the rapid development of artificial intelligence technology, applying machine learning methods to binary program understanding has become a research hotspot. Binary program data is characterized by strong platform dependence, multiple levels of abstraction, and difficulties in semantic recovery, thus posing unique technical challenges to data production methods. Training high-quality intelligent models of binary programs requires large-scale, diverse binary program data as training corpus.

[0003] In existing technologies, one method for acquiring datasets involves obtaining open-source project source code through code hosting platforms or compiling binary files using open-source software packages. However, such datasets only contain a single modality of source code or binary files, lacking a representation of the binary files or source code. Another method involves establishing a mapping between source code line numbers and binary addresses using debugging information generated during compilation (such as DWARF or PDB formats). This method relies on the existence of debugging information, but actually released binary files typically do not contain debugging information. Furthermore, compiler optimizations can disrupt the one-to-one mapping between source code line numbers and binary addresses, leading to inaccurate pairings. Therefore, dataset acquisition methods relying on debugging information suffer from incomplete or missing pairings between source code and binary addresses.

[0004] In summary, existing binary program datasets have significant shortcomings in terms of data integrity, modal richness, and cross-modal pairing, which restricts the development of software intelligent models. Summary of the Invention

[0005] This application provides a method and apparatus for producing multimodal data for program understanding, aiming to solve the problem that existing binary program datasets have significant shortcomings in terms of data integrity, modal richness, and cross-modal pairing, which restricts the development of software intelligent models.

[0006] Firstly, this application provides a method for producing multimodal data oriented towards program understanding, including: The source code and binary file of the target program are parsed to obtain the source code matching index, as well as multiple binary matching parameters of each function, binary data of multiple modalities, and source code data of multiple modalities. The source code matching index is the correspondence between each source code data and the source code matching key. For any function in the binary file, the source code matching index is queried sequentially using each binary matching parameter according to the priority order of the plurality of binary matching parameters. If the source code matching key corresponding to any binary matching parameter is found, the source code of the function, the binary data of the plurality of modalities, and the source code data of the plurality of modalities are associated based on the query result to obtain fused data.

[0007] In one embodiment, the plurality of binary matching parameters include the binary symbol of the function, the binary address range of the function, and the binary entry address of the function; the source code matching key includes the source code symbol key of the function, the source code address range key of the function, and the source code entry address key of the function. For any function in the binary file, the process of sequentially querying the source code matching index using each of the plurality of binary matching parameters according to their priority order includes: The binary symbol of the function is used to search for the source code symbol key corresponding to the binary symbol in the source code matching index; If the source code symbol key is not found, the binary address range of the function is used to search for the source code address range key corresponding to the binary address range in the source code matching index; If the source code address range key is not found, the binary entry address of the function is used to search for the source code entry address key corresponding to the binary entry address in the source code matching index.

[0008] In one embodiment, before parsing the source code and binary file of the target program, the method further includes: The source code of the target program is compiled to obtain the binary file and a source path mapping table between the source code and the binary file, specifically including: A bootstrap compiler toolchain is constructed, which is generated by iteratively compiling the compiler itself using the compiler source code; Analyze the dependencies of the target program and generate a dependency management strategy; In an isolated compilation environment, the bootstrap compiler toolchain is used to compile the source code of the dependent libraries and the target program in sequence according to the dependency management strategy, so as to obtain the binary file and the source code path mapping table.

[0009] In one embodiment, parsing the source code of the target program includes: Extract the source file path and source code line number of the function from the source code path mapping table; Locate the source code file based on the source file path, and extract the source code fragment of the function from the source code file based on the source code line number; Construct an input sequence containing task instructions and the source code fragment; The input sequence is input into a pre-trained large language model to obtain the natural language functional description and pseudocode representation of the function output by the large language model; The source code fragment, the natural language function description, and the pseudocode representation are used as the source code data for the multiple modalities; Multiple source code matching keys are determined using the source code file, and the source code matching index is constructed based on each source code data and each source code matching key.

[0010] In one embodiment, the step of determining multiple source code matching keys using the source code file and constructing the source code matching index based on each source code data and each source code matching key includes: Extract the source code symbol of each function in the source code file, calculate the hash value of the unique identifier of the binary file and the source code symbol, and obtain the source code symbol key of the function; Extract the source code address range of each function in the source code from the source code path mapping table, calculate the hash value containing the unique identifier of the binary file and the source code address range, and obtain the source code address range key of the function; Extract the source code entry address of each function in the source code file, calculate the hash value containing the unique identifier of the binary file and the source code entry address, and obtain the source code entry address key of the function; Use the source code symbol key, the source code address range key, and the source code entry address key as the multiple source code matching keys; Establish an index relationship to obtain the source code matching index; wherein, the index relationship represents that the source code symbol key, the source code address range key, and the source code entry address key point to the source code data of the multiple modalities.

[0011] In one embodiment, the step of extracting the source code fragment of the function from the source code file according to the source code line number includes: Based on the source code line number, locate an initial length context window in the source code file, and use the source code in the initial length context window as a candidate segment; If the candidate fragment does not contain the start and end identifiers of the function, the length of the context window is expanded by a preset step size, and the candidate fragment is extracted again until the candidate fragment contains the start and end identifiers of the function, or the length of the context window reaches the maximum window length. The final extracted candidate fragment is used as the source code fragment of the function.

[0012] In one embodiment, parsing the binary file of the target program includes: Convert the binary file into assembly code; Convert assembly instructions in assembly code into intermediate representation code containing standard opcodes; The intermediate representation code is then promoted to decompiled code; Parse the symbol table of the binary file to obtain the binary symbol of the function, the binary address range of the function, and the binary entry address of the function; The assembly code, the intermediate representation code, and the decompiled code are used as binary data for the multiple modalities, and the binary symbols, the binary address range, and the binary entry address are used as the multiple binary matching parameters. The binary data of the multiple modalities and the multiple binary matching parameters are serialized into a structured format in units of functions to obtain the parsing result. Write the parsing results into the cache.

[0013] In one embodiment, after converting the assembly instructions in the assembly code into intermediate representation code containing standard opcodes, the method further includes: The intermediate representation code is refined, and the refined intermediate representation code is used as part of the binary data of the multiple modalities; The refinement of the intermediate representation code includes: Based on the intermediate representation code, the basic blocks and jump relationships within the identification function are used to construct a control flow graph. Based on the control flow graph, reverse data flow analysis is performed starting from the function return point to obtain the parameter registers and effective operations that contribute to the final output of the function. Based on all the parameter registers and all the valid operations, temporary variables and redundant operations are filtered out to obtain the refined intermediate representation code.

[0014] In one embodiment, based on the control flow graph, reverse data flow analysis is performed starting from the function return point to obtain parameter registers and valid operations that contribute to the final output of the function, including: When the number of execution paths in the control flow graph does not exceed a preset threshold, all execution paths are traversed to identify the parameter register and the valid operation; When the number of execution paths in the control flow graph exceeds the preset threshold, multiple rounds of random path traversal are performed, and the parameter register and the valid operation are identified in each round of traversal.

[0015] In one embodiment, after obtaining the fused data, the method further includes: Based on the fused data, a multimodal dataset corresponding to the data requirements is generated.

[0016] Secondly, this application also provides a multimodal data production apparatus for program understanding, comprising: The parsing module is used to parse the source code and binary file of the target program to obtain the source code matching index, as well as multiple binary matching parameters of each function, binary data of multiple modalities, and source code data of multiple modalities. The source code matching index is the correspondence between each source code data and the source code matching key. The heterogeneous data pairing module is used to, for any function in the binary file, query the source code matching index sequentially using each of the multiple binary matching parameters according to the priority order of the binary matching parameters. If the source code matching key corresponding to any of the binary matching parameters is found, the module associates the source code of the function, the binary data of the multiple modalities, and the source code data of the multiple modalities based on the query result to obtain fused data.

[0017] Thirdly, this application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned program-oriented multimodal data production methods.

[0018] Fourthly, this application also provides a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the above-described program-oriented multimodal data production methods.

[0019] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements any of the above-described program-oriented multimodal data production methods. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is one of the flowcharts illustrating the multimodal data production method based on program understanding provided in this application; Figure 2 This is the second flowchart of the multimodal data production method based on program understanding provided in this application; Figure 3 This application provides the six modal data machine abstraction hierarchy relationships; Figure 4 This is a flowchart illustrating the heterogeneous data matching and fusion process provided in this application; Figure 5 This is one of the structural schematic diagrams of the multimodal data production device for program understanding provided in this application; Figure 6 This is the second schematic diagram of the multimodal data production device for program understanding provided in this application; Figure 7 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0024] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0025] The multimodal data production method and apparatus for program understanding provided in this application can be applied to scenarios that require large-scale, high-quality, multimodal program datasets, such as training of artificial intelligence models, software security analysis, and reverse engineering assistance.

[0026] The method and apparatus for generating multimodal data for program understanding provided in this application are described below with reference to the accompanying drawings.

[0027] It should be noted that the program-understanding-oriented multimodal data production method provided in this application embodiment is implemented based on a program-understanding-oriented multimodal data production device. The program-understanding-oriented multimodal data production method obtains a complete link of function descriptions from multiple abstract levels, from low-level machine representation to high-level semantic description, by parsing the source code and binary files respectively. It also ensures the accurate alignment and fusion of different modal data at the function level through a multi-level backoff matching strategy, effectively solving the problems of data integrity, modality richness, and cross-modal pairing in the prior art. The resulting large-scale, high-quality multimodal dataset provides a solid data foundation for training a more powerful and general program understanding model.

[0028] This application describes the method for producing multimodal data based on program understanding using a program-oriented understanding-based multimodal data production device as the execution subject.

[0029] Figure 1 This is one of the flowcharts of the multimodal data production method based on program understanding provided in this application. Figure 2 This is the second flowchart of the multimodal data production method based on program understanding provided in this application.

[0030] like Figure 1 As shown, the multimodal data production method for program understanding provided in this application embodiment may include the following steps: S110: Parse the source code and binary file of the target program to obtain the source code matching index, as well as multiple binary matching parameters of each function, binary data of multiple modalities, and source code data of multiple modalities. The source code matching index is the correspondence between each source code data and the source code matching key.

[0031] When parsing binary files, reverse engineering tools or analysis engines are used to perform in-depth analysis to extract their internal structure and content. It should be noted that this parsing process is automated and can process large numbers of binary files in batches and in parallel.

[0032] Binary matching parameters are feature information extracted from binary files that can be used to uniquely identify a function and for subsequent matching queries. These binary matching parameters have different characteristics and reliability, and therefore can be assigned different priorities.

[0033] In one possible implementation, the binary matching parameters may include, but are not limited to: the binary symbol of the function, the binary address range of the function, and the binary entry address of the function. Please refer to [reference needed]. Figure 2 A function's binary symbol typically refers to its name after compiler name mangling; it uniquely identifies a function. A function's binary address range is the starting and ending addresses of the function within the virtual address space of the binary file. A function's binary entry address is the address of its first instruction.

[0034] Multimodal binary data is a data representation extracted from binary files that describes the same function's functionality at different levels of abstraction. Multimodal binary data can include, but is not limited to, assembly code, intermediate representation code, and decompiled code. Assembly code is a sequence of low-level instructions obtained through disassembly and is specific to a particular hardware architecture (e.g., x86, ARM). Intermediate representation code elevates assembly code to a standardized intermediate representation that masks underlying hardware differences, such as platform-independent intermediate representation code (P-code). As a bridging layer, intermediate representation code connects platform-dependent machine code and platform-independent high-level languages, uniformly converting machine instructions from different platforms into semantically equivalent sequences of operations consisting of a compact set of arithmetic and logical operations. Decompiled code further elevates intermediate representation code or assembly code to a form similar to a high-level language (e.g., C-like language), restoring some variable names and control flow structures, resulting in higher readability.

[0035] Understandably, intermediate representation code can also be LLVM (Low Level Virtual Machine) intermediate representation, Valgrind's Executable eXpression intermediate representation (VEX) intermediate representation, custom intermediate representation, etc. The following explanation uses platform-independent intermediate representation code P-code as an example.

[0036] Source code data with multiple modalities obtained from parsing source code refers to data representations extracted or generated from source code files that describe the functionality of the same function from a perspective closer to human understanding. Multiple modal source code data may include, but is not limited to: natural language descriptions and pseudocode for each function. Please refer to [reference needed]. Figure 2 Natural language descriptions are textual descriptions of a function's functionality, such as what algorithm the function implements, what its inputs and outputs are, and what its side effects are. Pseudocode, on the other hand, is an abstract, language-independent description of algorithms that focuses on the logical flow rather than implementation details.

[0037] Source code matching keys are feature information extracted from the source code side that corresponds to binary matching parameters. For example, source code matching keys can include function source code symbol (i.e., function symbol) keys, function source code address range keys, etc. For example, both binary symbol and source code symbol keys are obtained based on function symbols; both binary address range and source code address range keys are obtained based on function address ranges.

[0038] The source code matching index maps the source code matching keys of these functions to records containing source code data for all modalities of that function. This index can be implemented using hash tables, B+ tree indexes, inverted indexes, and Bloom filter pre-filters to support fast queries in subsequent steps. B+ tree indexes support range queries and are suitable for address range matching scenarios. Inverted indexes support multi-keyword combination queries. Bloom filter pre-filters first quickly eliminate non-matching candidates before performing exact matching.

[0039] S120: For any function in the binary file, according to the priority order of the plurality of binary matching parameters, query the source code matching index sequentially using each of the binary matching parameters. If the source code matching key corresponding to any of the binary matching parameters is found, the source code of the function, the binary data of the plurality of modalities, and the source code data of the plurality of modalities are associated based on the query result to obtain fused data.

[0040] Specifically, the priority order is pre-defined based on the accuracy and robustness of the matching parameters. Generally, parameters that are highly accurate but easily optimized (such as symbol names) have the highest priority, while parameters that are relatively stable but may have a small number of conflicts (such as address information) have the next highest priority.

[0041] During the query process, the highest-priority binary matching parameter is first used to query the source code matching index. If a source code matching key (e.g., source code symbol key) corresponding to the current binary matching parameter (e.g., binary symbol) is found in the source code matching index, the match is considered successful. At this point, subsequent queries using lower-priority binary matching parameters are immediately stopped, and the data fusion stage begins. If no match is found using the current-priority binary matching parameter, the process automatically backtracks, repeating the query process using the next-priority binary matching parameter. This multi-level backtracking strategy maximizes the success rate of query matching, even if the highest-priority symbol is stripped, there is still a chance to complete the pairing using address information, etc.

[0042] Understandably, the above query matching process can employ a three-level fallback matching strategy, function signature-based matching, code feature-based matching, semantic similarity-based matching, and other methods.

[0043] After a successful query match, the binary data of multiple modalities of the function is extracted from the parsing result file of the binary file, the source code data of multiple modalities of the function is extracted from the parsing result file of the source code, and the source code of the function is extracted from the source code storage (located through source code path mapping). Then, the source code of the function, the binary data of multiple modalities, and the source code data of the successfully matched multiple modalities are combined into a unified data record (i.e., fused data). This record fully contains the functional description of the same function from multiple abstraction levels from machine code to natural language, that is, each record contains the semantic annotation information of a function.

[0044] like Figure 3 The six modalities of data machine abstraction hierarchy shown, from bottom to top, are: assembly code (platform-dependent layer, bound to specific processor architecture), P-code platform-independent intermediate representation (platform-independent layer, semantic operation sequences that mask hardware differences), decompiled code and original source code (high-level language layer, program representation in C language style), pseudocode (abstract algorithm layer, algorithm description within 30 lines), and natural language description (semantic description layer, textual description of function functionality). These six modalities cover a complete abstraction hierarchy from machine representation to human semantics, with P-code acting as a bridging layer connecting platform-dependent and platform-independent representations.

[0045] The fused data comprehensively covers multiple levels of abstraction, accurately pairs heterogeneous data, and fully preserves the source code traceability chain. Clear transformation relationships and semantic correspondences exist between different modalities, fully considering the diverse needs of program understanding and model training. This fused data can be directly transformed into a foundational model for training software intelligence.

[0046] This application embodiment obtains a complete function description of multiple abstraction levels, from low-level machine representation to high-level semantic description, by parsing the source code and binary files respectively. It also ensures the accurate alignment and fusion of different modal data at the function level through a multi-level backoff matching strategy, effectively solving the problems of data integrity, modal richness, and cross-modal pairing in the prior art. The resulting large-scale, high-quality multimodal dataset provides a solid data foundation for training a more powerful and general program understanding model.

[0047] In existing technologies, datasets typically use pre-compiled system libraries and third-party libraries for binary files, making it impossible to obtain the complete original source code and the source code of dependent libraries, thus compromising the integrity of the dataset. Furthermore, many projects fail to compile due to configuration issues, resulting in a low compilation success rate for existing datasets. This limits the size and diversity of datasets and leads to insufficient compilation configuration coverage.

[0048] Based on the above considerations, in one possible implementation, before parsing the source code and binary file of the target program (step S110), the following steps are also included: S100: Compile the source code of the target program to obtain the binary file and the source code path mapping table between the source code and the binary file.

[0049] The source code of the target program can be a complete collection of source code from any software project to be analyzed and processed, written in a high-level programming language (such as C, C++, Go, Rust, etc.). The source code may include: (1) Target software source code: The source code of the software project obtained from the open-source code repository, including the main program source file, header files, build scripts, etc. Figure 2 As shown; (2) Dependency library source code: The complete source code of third-party libraries and system libraries that the target software depends on, including direct dependencies and transitive dependencies. For example... Figure 2 As shown; (3) Compiler toolchain source code: The source code of tools such as compilers and linkers used to build the target software, which are used to implement self-bootstrapping compilation.

[0050] Specifically, the source code is stored according to the directory structure organized by the software package, preserving the original file hierarchy and naming. The compiled binary files are stored according to the compilation configuration.

[0051] Compilation is the process of converting source code into a binary file executable by the target machine through a series of processes (such as preprocessing, compilation, assembly, and linking). To ensure the accuracy and completeness of the association between the source code and the binary code in subsequent steps, necessary mapping information is preserved during the compilation process.

[0052] Specifically, in step S100, the source code of the target program is compiled to obtain the binary file and a source code path mapping table between the source code and the binary file, specifically including: S1010: Construct a bootstrap compiler toolchain, which is generated by iteratively compiling the compiler itself using the compiler's source code. This produces a fully traceable compilation toolchain, with each component having its corresponding source code record.

[0053] When building a bootstrap compiler toolchain, start with the base compiler and download the source code of the target program's compiler. Then, use the base compiler to compile the target compiler's source code to generate the first version of the target compiler. Next, use the first version of the target compiler to recompile the target compiler's source code to complete the bootstrap process. Then, build the linker, standard library, and other complete toolchain components in the same way. Finally, deploy the built compiler toolchain to an isolated compilation environment.

[0054] Taking the construction of the GNU Compiler Collection (GCC) toolchain as an example, the first step is to use the pre-installed basic compiler to compile the GCC source code (i.e., the source code of the target program), generating the first version of GCC. Then, the first version of GCC is used to recompile the GCC source code itself, completing the bootstrapping process. Finally, each component in the generated GCC toolchain (including the compiler GCC, the assembler (GNU Assembler, AS), the linker (Link eDitor), etc.) has a corresponding source code record. When compiling any software using this toolchain, the generated binary code can be traced back to the toolchain source code. Thus, the compilation toolchain itself is built from source code, eliminating the dependency on pre-compiled system components.

[0055] S1020: Analyze the dependencies of the target program and generate a dependency management strategy.

[0056] When compiling a target program, it typically depends on multiple third-party libraries, which may have interdependencies with each other. If the compilation order is incorrect, it will fail due to missing prerequisite dependencies.

[0057] Therefore, by parsing the target program's build scripts (such as Makefile, CMakeLists.txt, package.json, etc.), all packages that the target program directly or indirectly depends on are automatically identified. These dependencies are then constructed into a Directed Acyclic Graph (DAG), where nodes represent packages and directed edges represent dependencies. This graph is then topologically sorted to obtain a linear order of packages, i.e., the dependency management strategy. In this sequence, any package is listed after the packages it depends on. For example, for a dependency chain `A -> B -> C`, topological sorting will result in a compilation order of `C, B, A`. Thus, the dependency management strategy ensures the correct compilation order and supports the handling of complex dependencies.

[0058] Understandably, dependency management strategies can employ dependency topology sequences, lazy compilation strategies, and pre-computed dependency closures. In a lazy compilation strategy, dependency compilation is recursively triggered when a missing dependency is encountered. In a pre-computed dependency closure, all transitive dependencies are pre-computed, and packages without dependency conflicts are compiled in batches and in parallel.

[0059] S1030: In an isolated compilation environment, the bootstrap compiler toolchain is used to compile the source code of the dependent library and the target program in sequence according to the dependency management strategy to obtain the binary file and the source code path mapping table.

[0060] For each software package, dependencies are compiled sequentially according to the topological order of the dependency management strategy. First, isolated virtual compilation environments are created. These isolated environments are clean and independent temporary environments created for each compilation task, preventing interference from libraries or tools in the host operating system and ensuring repeatability and determinism in the compilation process. This isolation can be achieved using virtual machine technology based on disk image overlay or container technology. Specifically, container images can replace virtual machine disk images, and container layer overlay can replace disk image overlay. Next, install all the package's dependencies (from the compiled local software repository), and download and extract the package source code. Use the bootstrap toolchain built above to compile the dependencies, then collect the compiled artifacts (such as .so and .a library files) and add them to the local software repository. Once all dependent libraries have been compiled in sequence, finally compile the source code of the target program itself.

[0061] Taking the compilation of the zlib compression library as an example: First, the GNU C standard library (glibc standard library) that zlib depends on is resolved; according to the dependency management strategy, the glibc standard library is compiled first, and then zlib is compiled; after compilation, the functions such as memcpy called in the zlib binary file can be traced back to the corresponding implementations in the glibc source code.

[0062] Throughout the compilation process, a detailed record is kept of which version of the source code each compilation artifact (such as .so, .a library files, and binary files) was generated from, in which path, and by which version of the bootstrap toolchain, thus forming a precise source code path mapping table.

[0063] Finally, the SHA256 hash value of each binary file is calculated as a unique identifier for that binary file.

[0064] This results in a set of binary files, each with its own complete source code (including source code of dependent libraries), significantly improving data integrity and identifying unique associations through SHA256 hash values.

[0065] This application embodiment, by constructing and using a bootstrap compiler toolchain, combined with dependency management strategies and an isolated compilation environment, ensures that every piece of code in the binary file generated from the lowest-level dependency libraries to the final target program can be accurately traced back to its original, known source code. This eliminates the problem of missing data at the source, greatly improving the integrity, purity, and reliability of the final multimodal dataset, and providing a pollution-free data foundation for training high-precision program understanding models. Furthermore, this application embodiment, by constructing an isolated compilation environment and employing a dependency management strategy to ensure the correct dependency order, improves the compilation success rate through unified dependency management, and expands the scale and diversity of the dataset.

[0066] In one possible implementation, step S110 involves using reverse engineering techniques to obtain binary matching parameters and binary data for multiple modalities. Reverse engineering is a technique for recovering program structure and semantics from binary files, including disassembly (binary to assembly), decompilation (binary to high-level language), control flow analysis, and data flow analysis. Reverse engineering tools can generate various program representations such as assembly code, intermediate representations (e.g., intermediate representation code), and decompiled code.

[0067] Specifically, in step S110, the binary file of the target program is parsed, including: P10: Read the binary file to be analyzed from the binary file storage, check the content addressing cache (using the SHA256 hash value of the binary file as the key), if the parsing result already exists, reuse the parsing result directly; if the cache is not hit, start the reverse engineering analysis engine and perform the following analysis, extracting all binary matching parameters and binary data simultaneously in word analysis.

[0068] P20: Convert the binary file into assembly code and identify function boundaries.

[0069] The reverse engineering analysis engine reads the machine code (i.e., the sequence of instructions composed of 0s and 1s) from the binary file and translates it into human-readable assembly language mnemonics corresponding to a specific CPU architecture (such as x86-64, ARMv8). For example, the machine code `55 48 89 E5` will be converted to `push rbp; mov rbp, rsp`. This process also identifies the program's code segment and data segment, and initially delineates the boundaries of functions.

[0070] P30: Convert assembly instructions in assembly code into intermediate representation code containing standard opcodes.

[0071] Because assembly code is platform-dependent, with significant differences in instruction sets across different architectures, directly using it for model training can lead to poor model generalization. Therefore, this step elevates it to a higher-level, platform-independent abstraction.

[0072] Intermediate representation code (such as P-code) is a register transfer language used for reverse engineering. It defines a finite set of virtual instructions containing standard opcodes. These standard opcodes cover core semantic categories such as control flow, memory access, and arithmetic logic.

[0073] Specifically, the reverse engineering analysis engine translates each specific assembly instruction into one or more semantically equivalent P-code operation sequences. This conversion shields the differences in the underlying hardware, allowing subsequent analysis and model learning to focus on the program's general semantic logic rather than the implementation details of a specific platform.

[0074] P40: Promote the intermediate representation code to decompiled code and set a single function timeout protection mechanism to prevent processing from being suspended due to complex functions.

[0075] To further improve code readability and make it closer to the thought process of human programmers, this step decompiles the intermediate representation code, that is, recovering the high-level language form of the code from the low-level representation (such as intermediate representation or assembly), typically in a C-like style. This process involves sophisticated program analysis techniques, such as: constructing a Control Flow Graph (CFG) to identify loop (`for`, `while`) and branch (`if-else`, `switch`) structures; performing data flow analysis to recover variable definitions and usages, and attempting to infer their types. For example, a series of complex P-code operations might ultimately be reduced to a single line of concise C code, such as `for (int i = 0; i<10; i++) { ...}`.

[0076] In this step, control flow analysis is also used to identify the basic blocks and function jump relationships of the function, construct the control flow graph of the function, and extract information such as function call relationships based on the control flow graph.

[0077] P50: Parse the symbol table of the binary file to obtain the binary symbol of the function, the binary address range of the function, and the binary entry address of the function.

[0078] P60: The assembly code, the intermediate representation code, and the decompiled code are used as binary data for the multiple modalities, and the binary symbols, the binary address range, and the binary entry address are used as the multiple binary matching parameters; the binary data of the multiple modalities and the multiple binary matching parameters are serialized into a structured format (e.g., JSON object or Protocol Buffers message) in units of functions to obtain the parsing result.

[0079] The parsing results include the unique identifier of the binary file and a list of functions. The parsing record for each function includes the function symbol (i.e., symbol name), entry address, address range, instruction sequence (including intermediate representation code operations), assembly code, the decompiled code, call relationships, and other information.

[0080] For example, for a function that implements quicksort, the binary data extracted by parsing the binary code includes multiple modalities such as: assembly code (e.g., platform-dependent x86-64 instructions such as push rbp; mov rbp, rsp; sub rsp, 0x20), intermediate representation code (e.g., platform-independent semantic operations such as INT_SUB RSP, RSP, 0x20; STORE RAM[RSP], RBP), and decompiled code (e.g., C-like language forms such as void quicksort(int *arr, int low, int high).

[0081] P70: Write the parsing results to the cache.

[0082] In the cache, each binary file corresponds to a parsing result file. Furthermore, when using a distributed cache, the cache is shared across the cluster, improving overall processing efficiency. Fault recovery and resume interrupted downloads are achieved through file-stamp-based state tracking.

[0083] Specifically, the unique identifier (SHA256 hash value) of the binary file is used as the key, and the parsing result is used as the value, stored in a caching system (such as Redis, a local file database, etc.). The next time the same binary file needs to be analyzed, the cache can be queried first using its hash value. If a match is found, the result can be obtained directly, thus skipping the entire time-consuming analysis process (see step P10).

[0084] The embodiments of this application automatically and in parallel extract the program representation modalities of three different levels of abstraction of functions and various parameters for precise matching from a single binary file, ensuring the richness and multi-layered nature of the extracted data content, and significantly improving the efficiency of processing large-scale binary file sets by introducing a caching mechanism.

[0085] Although the intermediate representation code obtained above masks hardware differences, it may still contain a large number of auxiliary or redundant operations generated by the compiler that are unrelated to the core algorithm logic of the program. These operations are necessary for the correct execution of the program, but they will interfere with the learning of artificial intelligence models that aim to understand the semantics of the program.

[0086] Based on the above considerations, in one possible implementation, after converting the assembly instructions in the assembly code into intermediate representation code containing standard opcodes (step P30), the following steps are also included: The intermediate representation code is refined, and the refined intermediate representation code is used as part of the binary data of the multiple modalities.

[0087] The refinement of the intermediate representation code includes: P1: Based on the intermediate representation code, identify the basic blocks and jump relationships within the function, and construct a control flow graph based on the basic blocks and jump relationships.

[0088] P2: Based on the control flow graph, reverse data flow analysis is performed starting from the function return point to obtain the parameter registers and valid operations that contribute to the final output of the function.

[0089] A function return point is a node in the control flow graph that represents the end of function execution and its return value; it typically contains a return instruction. Starting from the final result of a function (i.e., the function's return value), the analysis traces backwards through all operations and data that affected that result. The analysis process may include the following steps: Initialization: First, at the function return point, the register or memory location used to pass the return value (for example, the `RAX` register in the x86-64 architecture) is marked as active.

[0090] Iterative Tracing: Starting from the function return point, traverse backwards along the edges of the control flow graph. For each P-code instruction encountered, check if its output operand (i.e., the register or memory being written to) is in the active set. If the instruction's output is active, it means that the instruction contributes to subsequent active values, so the instruction itself is marked as a valid operation, and all its input operands (i.e., the registers or memory being read) are added to the active set. If the instruction's output is not active, the instruction does not affect the final result.

[0091] Convergence: Repeat the above tracing process until the active set no longer changes.

[0092] Ultimately, all registers marked as active are the parameter registers that contribute to the final output of the function, and all P-code instructions marked as valid operations constitute the valid operations that contribute to the final output of the function.

[0093] P3: Based on all the aforementioned parameter registers and all the aforementioned valid operations, temporary variables and redundant operations are filtered out to obtain the refined intermediate representation code, which is a shorter, more focused sequence of instructions on the core algorithm logic. For example, for a simple addition function, the original P-code may contain 10 instructions (including the function preamble and end), while the refined version may contain only one core `INT_ADD` instruction.

[0094] The embodiments of this application refine the intermediate representation code, which significantly reduces the amount of data compared to the original intermediate representation code, and focuses on semantically relevant core operations, thereby improving the signal-to-noise ratio of downstream model training.

[0095] In one possible implementation, in step P2, based on the control flow graph, reverse data flow analysis is performed starting from the function return point to obtain the parameter registers and valid operations that contribute to the final output of the function, including: P21: When the number of execution paths in the control flow graph does not exceed the preset threshold, all execution paths are traversed to identify the parameter register and the valid operation.

[0096] The number of execution paths is the total number of all possible paths from the entry node to the exit node of the control flow graph. Before performing analysis, the structure of the control flow graph can be preliminarily evaluated, for example, by calculating its cyclomatic complexity or by directly trying path counts to determine its complexity.

[0097] When the number of evaluation paths for a function does not exceed a preset threshold, performing a complete and deterministic dataflow analysis is computationally feasible. In this case, the system traverses all execution paths, i.e., executes the standard reverse dataflow analysis algorithm. This algorithm systematically and deterministically analyzes all possible execution paths through fixed-point iteration until a complete set of registers and operations active under any path is calculated. The results obtained in this way are accurate and complete.

[0098] P22: When the number of execution paths in the control flow graph exceeds a preset threshold, perform multiple rounds of random path traversal, and identify the parameter register and the valid operation in each round of traversal.

[0099] When the number of paths to a function exceeds a preset threshold, the system switches to an analysis method based on a random path sampling algorithm. During random path traversal, instead of attempting to traverse all execution paths, it starts from the function's return point and randomly selects a reverse path to trace back until it reaches the function's entry point. Specifically, when a node has multiple predecessor nodes (i.e., multiple branch convergence points), one of the predecessor nodes is randomly selected to continue the reverse traversal. This process is repeated multiple times (e.g., 100 rounds).

[0100] In each independent round of random path traversal, based on the aforementioned method, the active data flow on the current path is tracked, and the parameter registers and valid operations on that path are identified and marked. The system maintains a global set spanning all rounds (including valid operations and active registers). The results identified in each round of random traversal are incorporated into this global set. Through multiple rounds of random sampling, although only one path is seen each time, after a sufficient number of rounds, it is highly likely to cover most of the important and commonly used execution paths in the function, thereby approximately identifying the vast majority of the core logic.

[0101] This random path sampling method is an application of the Monte Carlo approach. It sacrifices some determinism and completeness in exchange for the ability to process ultra-complex functions within an acceptable timeframe.

[0102] This application's embodiments determine the complexity of a function by checking whether the number of execution paths exceeds a preset threshold, thereby dynamically selecting the analysis mode based on the function's complexity: for simple functions, precise full path traversal is used to ensure the accuracy of the analysis results; for complex functions with a huge number of paths, an efficient random path sampling mode is switched to avoid analysis stagnation or timeouts caused by path explosion. This strategy of intelligently balancing accuracy and efficiency greatly improves the robustness and practicality of the entire data production system when handling various complex programs in the real world, enabling it to efficiently and stably complete the production tasks of large-scale datasets.

[0103] Some existing datasets lack high-level descriptions of function semantics, such as natural language function descriptions, algorithm pseudocode, and function labels. This type of function-level semantic information is crucial for training program understanding models; therefore, the lack of such information reduces the prediction accuracy of these models.

[0104] Based on this consideration, in one possible implementation, step S110 involves parsing the source code of the target program, including: Q10: Extract the source file path and source code line number of the function from the source code path mapping table.

[0105] Q20: Locate the source code file according to the source file path, and extract the source code fragment of the function from the source code file according to the source code line number.

[0106] Based on the path information obtained in the previous step, the system opens the corresponding file on the medium storing the source code to obtain the source code file. Specifically, it can use a fixed window strategy, adaptive context window technology, syntax analysis-based boundary detection, or a hierarchical context strategy to determine the source code fragments of the function. Syntax analysis-based boundary detection uses a program parser to pre-determine the function boundaries before extracting the context. The hierarchical context strategy first extracts a file-level summary and then extracts the function-level detailed context.

[0107] Based on the line numbers of the source code, a segment of code text containing a function is extracted from the source code file and taken as the source code fragment for that function. In a simple implementation, a fixed number of lines (e.g., 200 lines) starting from the given line number can be extracted. In a more efficient implementation, a smarter method is used to ensure that the complete function body is extracted. The resulting source code fragment is the core content subsequently input into the large language model.

[0108] Q30: Construct an input sequence containing task instructions and the source code fragments.

[0109] In one possible implementation, an extended context is first constructed based on source code snippets, including the first 50 lines of the file header declaration, the first 150 lines of the function preamble, the function code, and the last 100 lines of the function postamble, while also providing decompiled code as a reference. The input sequence consists of three parts: task instructions (hints), the extended context and decompiled code, and the source code snippet of the function to be annotated. In the input sequence, special delimiters are typically used between the extended context, decompiled code, task instructions, and the source code snippet of the function to be annotated to help the model distinguish between instructions and the content to be analyzed.

[0110] The task instruction is a piece of natural language text used to guide the behavior of a large language model, such as requesting the model to generate natural language descriptions (semantic layer) and pseudocode (abstract layer). The natural language description includes function specifications, implementation strategies, and textual descriptions of key algorithms. The pseudocode can be an abstract algorithm description of up to 30 lines. Metadata tags include function categories, performance characteristics, and dependencies.

[0111] Q40: Input the input sequence into a pre-trained large language model to obtain the natural language functional description and pseudocode representation of the function output by the large language model, thereby achieving semantic annotation.

[0112] Large language models are natural language processing techniques based on deep learning, capable of understanding and generating human language. They are widely used in tasks such as code understanding and code generation.

[0113] In one possible implementation, the large language model is trained on a dataset consisting of the program's source code and its labeled data.

[0114] Understandably, large language models can be developed using open-source large language models (such as Llama, Qwen, etc., which can be deployed locally without network requests), code-specific models (such as CodeLlama, StarCoder, etc., which are optimized for code tasks), or multi-model integration (using multiple models to generate annotations and improving quality through voting or fusion).

[0115] It should be noted that by controlling the number of concurrent requests to the large language model, it is possible to enable the large language model to process multiple functions in parallel.

[0116] In one possible implementation, the principle and structure of the large language model are as follows: (1) Model Structure: A large language model based on the Transformer architecture is adopted, which consists of multiple layers of self-attention mechanisms and feedforward neural networks. The model captures the dependencies between any positions in the input sequence through the self-attention mechanism, and can understand the syntactic structure and semantic information of the code. The model parameters are usually in the range of billions to hundreds of billions, and it has powerful code understanding and natural language generation capabilities.

[0117] (2) Input processing: The source code text and decompiled code are converted into a word sequence by a tokenizer, and each word is mapped to a high-dimensional vector representation. The input sequence contains three parts: task instructions (prompt words), context code fragments, and function code to be annotated, separated by special markers.

[0118] (3) Reasoning process: The model adopts an autoregressive generation method to predict the output sequence word by word. In the boundary detection stage, the model determines the start and end lines of the function based on the code context; in the annotation generation stage, the model comprehensively understands the control flow, data flow, and call relationships of the function to generate natural language descriptions and pseudocode. The model focuses on key code segments in the input, such as loop structures, conditional branches, and function calls, through an attention mechanism, thereby generating accurate semantic descriptions.

[0119] (4) Output constraints: The output format of the engineering constraint model is prompted, requiring the natural language description to include elements such as functional overview, implementation strategy, and algorithm complexity. The pseudocode is limited to 30 lines and uses a unified pseudocode syntax. The output results are written to the labeled data storage after format parsing and validity verification.

[0120] It should be noted that the large language model achieves breakpoint resumption through a checkpoint mechanism, supporting reliable processing of large-scale data; its concurrency control and error retries ensure the stability of annotation generation.

[0121] The input sequence is fed into a pre-trained large language model, which performs semantic analysis to obtain a natural language functional description and pseudocode representation of the function, thereby achieving semantic annotation.

[0122] Q50: The source code fragment, the natural language function description, and the pseudocode representation are used as the source code data for the multiple modalities.

[0123] The annotation results are formatted and written to a checkpoint file to enable breakpoint resumption. Fault recovery and breakpoint resumption are achieved through state tracking based on file stamps. The annotation record for each function includes: a unique identifier for the binary file, the function's source code symbols (corresponding symbol names), a natural language function description, a pseudocode representation, the detected source language type, function tags, performance characteristics, etc.

[0124] For the quicksort function, the generated annotations include: a natural language description ("This function implements the quicksort algorithm, using a divide-and-conquer strategy, selecting the middle element as the pivot value, recursively sorting the left and right subarrays, with an average time complexity of O(nlog n."), and pseudocode (function quicksort(array, low, high): if low <high: pivot_index = partition(array, low, high); quicksort(array, low, pivot_index - 1);quicksort(array, pivot_index + 1, high))。

[0125] Q60: Use the source code file to determine multiple source code matching keys, and construct the source code matching index based on each source code data and each source code matching key.

[0126] This application's embodiments automatically utilize pre-trained large language models to transform the source code of functions into high-quality, deep semantic descriptions and abstract pseudocode, elevating the source code, which was originally only machine-readable, to a semantic level that is easy for humans to understand, thus enriching the modalities of the dataset.

[0127] In one possible implementation, step Q20, which involves extracting the source code fragment of the function from the source code file based on the source code line number, includes: Q21: Locate an initial length context window in the source code file based on the source code line number, and use the source code in the initial length context window as a candidate segment.

[0128] Specifically, a context window of initial length is obtained by expanding a code region outwards from the source code line number as the center or starting point towards both ends (or one end) of the file. For example, a context window of initial length can be formed by expanding upwards by 50 lines and downwards by 150 lines from the source code line number, resulting in a total length of 200 lines. The initial length is an empirical value designed to cover most small to medium-sized functions with a smaller window to improve processing efficiency. All source code text contained within this initial length context window constitutes the candidate fragments.

[0129] Q 22: If the candidate fragment does not contain the start and end identifiers of the function, the length of the context window is expanded by a preset step size, and the candidate fragment is extracted again until the candidate fragment contains the start and end identifiers of the function, or the length of the context window reaches the maximum window length.

[0130] The start identifier is typically a complete function signature. For example, in C / C++, it might be a line that begins with the return type, includes the function name and a complete parameter list, and ends with `)` or `{`. The end identifier is typically the last `}` that matches the `{` at the beginning of the function body. Start and end identifiers can be detected by matching pairs of curly braces `{}`, parentheses `()`, etc., combined with regular expressions or lightweight parsers. If a candidate segment contains mismatched curly braces, or if no clear function signature is found, it can be determined that a complete function identifier is missing.

[0131] When an incomplete candidate segment is detected, the system automatically expands the truncation range. For example, the length of the context window can be multiplied by a coefficient (such as 1.5 times) or increased by a fixed number of lines (such as 100 lines), and then a larger range of code can be extracted as new candidate segments.

[0132] The newly extracted candidate snippet contains complete function start and end markers. At this point, the loop terminates, resulting in a high-quality, complete function code snippet. If the window length continues to expand but a complete function body cannot be found, to prevent infinite loops and excessive resource consumption, the system sets a maximum window length (e.g., a maximum expansion of 4 times or a maximum of 4000 lines). Once the window length reaches this limit, the loop will be forcibly terminated even if the code is still incomplete. This is a protective mechanism to ensure the stable operation of the system.

[0133] Q23: The last extracted candidate fragment is used as the source code fragment of the function.

[0134] After the above loop terminates, whether it is because a complete function body is successfully found or because the maximum window limit is reached, the system will take the current (i.e. the last) extracted candidate fragment as the source code fragment to be submitted to the large language model.

[0135] This application's embodiments adaptively expand the size of the context window based on the loop probing results of start and end markers, thereby determining the source code fragments of functions and intelligently identifying the precise boundaries of most functions, ensuring the integrity of the data input to the large language model. Compared to simple methods using fixed-length windows, it significantly improves the accuracy of subsequent semantic annotation, avoids inaccurate predictions due to incomplete input information, and thus significantly improves the overall quality and usability of the final generated dataset. Simultaneously, by setting a maximum window length, this method also ensures its robustness and processing efficiency.

[0136] In one possible implementation, step Q60, which involves determining multiple source code matching keys using the source code file and constructing the source code matching index based on each source code data and each source code matching key, includes: Q61: Extract the source code symbol of each function in the source code from the source code file, calculate the hash value of the unique identifier of the binary file and the source code symbol, and obtain the source code symbol key of the function.

[0137] After locating the source code file based on the source code path mapping table (see steps Q10 and Q20), the source code file is first parsed to identify the source code symbol of each function (for example, the source code symbol of the function `int quick_sort(int* arr, intn)` is the string `quick_sort`).

[0138] In one possible implementation, the unique identifier of the binary file and the source code symbol are concatenated, and then a hash value is calculated on the concatenated string (e.g., using MD5 or a faster non-cryptographic hash function such as MurmurHash). This hash value is used as the source code symbol key. It is a fixed-length numeric or string, which is well-suited for use as a key in a hash table.

[0139] Q62: Extract the source code address range of each function in the source code from the source code path mapping table, calculate the hash value containing the unique identifier of the binary file and the source code address range, and obtain the source code address range key of the function.

[0140] By analyzing the source code mapping table, the virtual memory address range corresponding to each function at the source code level is obtained. The unique identifier of the binary file and the source code address range of the function (after being converted to string form) are concatenated, and their hash value is calculated to obtain the source code address range key of the function.

[0141] Q63: Extract the source code entry address of each function in the source code from the source code file, calculate the hash value containing the unique identifier of the binary file and the source code entry address, and obtain the source code entry address key of the function.

[0142] By analyzing the source code mapping table, the virtual entry address corresponding to each function at the source code level is obtained. The unique identifier of the binary file and the source code entry address are concatenated and a hash value is calculated to obtain the source code entry address key of the function.

[0143] Q64: Use the source code symbol key, the source code address range key, and the source code entry address key as the multiple source code matching keys.

[0144] Q65: Establish an index relationship to obtain the source code matching index; wherein, the index relationship represents that the source code symbol key, the source code address range key, and the source code entry address key point to the source code data of the multiple modalities.

[0145] After generating the three types of keys for each function, they are organized into a data structure that allows for fast querying, namely the source code matching index.

[0146] Specifically, index relationships are established by constructing one or more hash tables.

[0147] In one possible implementation, a unified hash table can be constructed, using all three source code matching keys of a function as keys in this hash table. These different keys all point to the same value, which is a record or pointer containing the source code data of the function across multiple modalities. When subsequent steps use keys generated on the binary side for querying, the corresponding source code data can be found quickly, greatly improving matching efficiency.

[0148] This application's embodiments utilize the unique identifier and function characteristics of binary files to calculate a composite hash key, which serves as the source code matching key. This avoids erroneous matching across binary files, solves the problem of cross-file function name or address conflicts, and ensures the global uniqueness and accuracy of the matching key. At the same time, by organizing all keys into a hash table-like index, the potentially time-consuming string or address comparison operations are transformed into efficient constant-time lookup operations, ensuring both matching accuracy and speed.

[0149] In one possible implementation, the plurality of binary matching parameters include the binary symbol of the function, the binary address range of the function, and the binary entry address of the function; the source code matching key includes the source code symbol key of the function, the source code address range key of the function, and the source code entry address key of the function.

[0150] Based on the above, please combine Figure 4 In step S120, the step of querying the source code matching index sequentially using each of the plurality of binary matching parameters according to the priority order of the binary matching parameters for any function in the binary file includes: S1210: Use the binary symbol of the function to search for the source code symbol key corresponding to the binary symbol in the source code matching index. If the source code symbol key is found, then based on the query result, associate the source code of the function, the binary data of the multiple modalities, and the source code data of the multiple modalities to obtain fused data; if the source code symbol key is not found, then execute S1220.

[0151] Compared to address ranges and entry addresses, function symbols contain the most information, therefore matching binary symbols with source code symbols is the highest priority matching method.

[0152] Specifically, the system first extracts the binary symbol from the binary function to be matched, and then uses the binary symbol ( Figure 4 The hash value is calculated using the function decorated name and the unique identifier of the binary file, which serves as the binary symbol key. This binary symbol key is then used to query the source code matching index to determine if there exists a source code symbol key that matches the binary symbol.

[0153] If the search is successful, the matching is complete, and the process proceeds directly to the data fusion stage. If the search fails (for example, because the binary file is a C language program that has undergone complete symbol stripping, resulting in missing symbol information), the process automatically proceeds to the next level of matching.

[0154] S1220: Use the binary address range of the function to search for the source code address range key corresponding to the binary address range in the source code matching index. If the source code address range key is found, then based on the query result, associate the source code of the function, the binary data of the multiple modalities, and the source code data of the multiple modalities to obtain fused data; if the source code address range key is not found, then execute S1230.

[0155] Compiler optimizations (such as function inlining and instruction reordering) may break the strict correspondence between source code line numbers and binary addresses, but the address range of a function as a whole is usually relatively stable. This method is suitable when symbols are removed, but the function structure remains largely intact.

[0156] After the first level of matching fails, the system will extract the binary address range of the current binary function. Figure 4 The hash value is calculated using the binary address range and the unique identifier of the binary file (shown as the starting address). This hash value is then used as the binary address range key. The binary address range key is then used to query the source code matching index to determine if there is a source code address range key that matches the given address range.

[0157] If the search is successful, the matching is complete. If the search fails, the process automatically proceeds to the final level of matching (fallback strategy).

[0158] S1230: Using the binary entry address of the function, search for the source code entry address key corresponding to the binary entry address in the source code matching index. If the source code entry address key is found, then based on the query result, associate the source code of the function, the binary data of the multiple modalities, and the source code data of the multiple modalities to obtain fused data; if the source code entry address key is not found, then output the result of matching failure.

[0159] Regardless of compiler optimizations, the entry point of a function call must be preserved and determined. Therefore, the entry address is a very robust feature. This method can handle some dynamically generated or function-pointer-called functions whose boundaries are difficult to determine in static analysis. Although using the entry address alone may carry a very small risk of address conflicts, it is highly effective as a supplement when the first two strategies fail.

[0160] After the first two levels of matching fail, the system extracts the binary entry address of the current binary function, calculates a hash value using the binary entry address and the unique identifier of the binary file, uses it as the binary entry address key, and then uses the binary entry address key to query whether the corresponding source code entry address key exists in the source code matching index.

[0161] If the search is successful, the matching is complete. If the entry address cannot be matched, the function is marked as unmatchable, and the output result is "match failed".

[0162] like Figure 4 As shown, the left side displays the analysis data generated from parsing the binary file, including the binary file's unique identifier, function names, entry addresses, and other information. The right side displays the annotation data generated from parsing the source code, including the binary file's unique identifier, symbol names, symbol addresses, natural language descriptions, and pseudocode. The three-level backtracking matching strategy in the middle attempts to match symbols in sequence: the first level uses the source code symbol key (Hash(SHA256, function decorated name)) as the query key for symbol name matching; the second level uses the source code address range key (Hash(SHA256, starting address)) as the query key for address range matching; and the third level uses the source code entry address key (Hash(SHA256, entry address)) as the query key for entry point matching. If a previous level fails to match, it automatically backtracks to the next level. After successful matching, the data from each modality are merged to generate a multimodal record containing six program representations.

[0163] For example, in the pairing process of the quicksort function: first, try symbol name matching, with the query key Hash("a1b2c3...", "_Z9quicksortPiii"); if the symbol name matches successfully, output the pair directly; if it fails (e.g., the symbol is removed), try address range matching, with the query key Hash("a1b2c3...", 0x401000); if the address range matches successfully, output the pair; if it still fails, try entry point matching as a last resort.

[0164] In this embodiment, the first-level symbol matching ensures the highest accuracy when information is sufficient. The second-level address range matching and the third-level entry address matching serve as powerful supplements and fallbacks, greatly improving the pairing coverage when symbol information is missing or corrupted. The three-level strategy is designed in the order of decreasing accuracy and increasing coverage, prioritizing the use of high-accuracy symbol name matching. This achieves high-success-rate accurate pairing of heterogeneous data without relying on complete debugging information. This strategy balances accuracy and robustness and is a key technology for realizing the production of large-scale, automated, high-fidelity multimodal datasets.

[0165] In one possible implementation, the program-oriented multimodal data production method also includes: S130: Based on the fused data, generate a multimodal dataset corresponding to the data requirements. Please refer to [reference needed]. Figure 2 .

[0166] Fusion data consists of internal, structured data records, each containing multiple modal representations of the same function. To enable direct use by downstream applications such as AI models, the fusion data needs to be serialized in a specified format (e.g., a standardized training data format) and output as different data files according to data requirements (e.g., training task type). Training tasks can include tasks such as converting intermediate representation code to pseudocode, converting intermediate representation code to natural language description, and generating intermediate representation code from pseudocode.

[0167] The final output multimodal dataset includes: original source code, assembly code, intermediate representation code, decompiled code, pseudocode, and natural language description.

[0168] For example, for the quicksort function, the final output multimodal dataset includes at least: the original source code (the quicksort function implemented in C), assembly code (the sequence of machine instructions for the x86-64 platform), intermediate representation code (the sequence of semantic operations), decompiled code (C-like code recovered from binary), pseudocode (an algorithm description of no more than 30 lines), and natural language description (functional description text). These modal representations represent different levels of abstraction of the same function and can be used to train cross-modal transformation models.

[0169] The output dataset can be in JSON / JSONL format, Parquet columnar storage format, TFRecord format, Arrow format, etc. Parquet columnar storage format is suitable for large-scale data analysis, supporting column pruning and predicate pushdown. TFRecord format is suitable for direct input into TensorFlow training processes. Arrow format supports zero-copy data exchange, suitable for cross-language data processing.

[0170] In one possible implementation, in step S130, this application also generates a dataset statistical report, including the pairing success rate, coverage of each modality, data size, etc.

[0171] The multimodal dataset generated in this application can be used to train basic models in the field of software intelligence, such as program understanding models, vulnerability detection models, reverse engineering auxiliary models, and code generation models, and supports cross-modal program understanding and conversion tasks.

[0172] Based on the above, this application also provides a program-understanding-oriented multimodal data production apparatus. The program-understanding-oriented multimodal data production apparatus and the aforementioned program-understanding-oriented multimodal data production method can be referred to and correspond to each other.

[0173] As an example, such as Figure 5 As shown, the multimodal data production apparatus for program understanding provided in this application includes: The parsing module 510 is used to parse the source code and binary file of the target program to obtain the source code matching index, as well as multiple binary matching parameters of each function, multiple modal binary data, and multiple modal source code data. The source code matching index is the correspondence between each source code data and the source code matching key. The heterogeneous data matching module 520 is used to, for any function in the binary file, query the source code matching index sequentially using each of the multiple binary matching parameters according to the priority order of the multiple binary matching parameters. If the source code matching key corresponding to any of the binary matching parameters is found, the source code of the function, the binary data of the multiple modalities, and the source code data of the multiple modalities are associated based on the query result to obtain fused data.

[0174] This application embodiment obtains a complete function description of multiple abstraction levels, from low-level machine representation to high-level semantic description, by parsing the source code and binary files respectively. It also ensures the accurate alignment and fusion of different modal data at the function level through a multi-level backoff matching strategy, effectively solving the problems of data integrity, modal richness, and cross-modal pairing in the prior art. The resulting large-scale, high-quality multimodal dataset provides a solid data foundation for training a more powerful and general program understanding model.

[0175] Figure 6 This illustration shows an embodiment of the multimodal data production apparatus for program-oriented understanding according to this application, including its composition and data flow. Please refer to the above description and... Figure 6 As shown, the traceable compilation module comprises three components: a bootstrap toolchain builder, a dependency topology sorter, and an isolated compilation environment manager. It is responsible for compiling the source code into traceable binary files. The binary analysis module comprises four components: a reverse analysis engine, a content addressing cache, a P-code refiner, and a CFG builder. It is responsible for extracting assembly code, P-code, and decompiled code from the binary files. The semantic annotation module comprises four components: a boundary detector, an LLM call interface, an adaptive window, and a checkpoint management system. It is responsible for calling a large language model to generate natural language descriptions and pseudocode. The heterogeneous data pairing module comprises three components: an index builder, a three-level backtracking match, and a modality fusion unit. It is responsible for performing function-level pairing of the analyzed data (the parsing results of the binary files) and the annotated data (the parsing results of the source code). The binary analysis module and the semantic annotation module can execute in parallel, and their outputs converge in the heterogeneous data pairing module, ultimately producing a six-modality pairing training dataset.

[0176] It should be noted that the various processing modules in this application adopt a distributed architecture, supporting large-scale parallel processing. The traceable compilation module uses virtual machine isolation to achieve parallel compilation, the binary analysis module uses content-addressed caching to avoid redundant analysis, and the semantic annotation module uses asynchronous pipelines to achieve high concurrency. The overall data production capacity can be linearly scaled with computing resources.

[0177] Figure 7 This is a schematic diagram of the structure of the electronic device provided in this application, such as... Figure 7 As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a program-oriented, multimodal data production method, which includes: The source code of the target program is compiled to obtain a binary file and a source code path mapping table between the source code and the binary file; Parse the binary file to obtain multiple binary matching parameters of each function in the binary file and binary data of multiple modalities; Based on the source code path mapping table, the source code is parsed to obtain source code data of multiple modalities of each function in the source code and source code matching index. The source code matching index is the correspondence between each source code data and the source code matching key. For any function in the binary file, the source code matching index is queried sequentially using each binary matching parameter according to the priority order of the plurality of binary matching parameters. If the source code matching key corresponding to any binary matching parameter is found, the source code of the function, the binary data of the plurality of modalities and the source code data of the plurality of modalities are associated based on the query result to obtain fused data. Based on the fused data, a multimodal dataset corresponding to the data requirements is generated.

[0178] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0179] On the other hand, this application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute the program-understanding-oriented multimodal data production method provided in the above embodiments, the method including: The source code of the target program is compiled to obtain a binary file and a source code path mapping table between the source code and the binary file; Parse the binary file to obtain multiple binary matching parameters of each function in the binary file and binary data of multiple modalities; Based on the source code path mapping table, the source code is parsed to obtain source code data of multiple modalities of each function in the source code and source code matching index. The source code matching index is the correspondence between each source code data and the source code matching key. For any function in the binary file, the source code matching index is queried sequentially using each binary matching parameter according to the priority order of the plurality of binary matching parameters. If the source code matching key corresponding to any binary matching parameter is found, the source code of the function, the binary data of the plurality of modalities and the source code data of the plurality of modalities are associated based on the query result to obtain fused data. Based on the fused data, a multimodal dataset corresponding to the data requirements is generated.

[0180] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the program-oriented multimodal data production method provided in the above embodiments, the method comprising: The source code of the target program is compiled to obtain a binary file and a source code path mapping table between the source code and the binary file; Parse the binary file to obtain multiple binary matching parameters of each function in the binary file and binary data of multiple modalities; Based on the source code path mapping table, the source code is parsed to obtain source code data of multiple modalities of each function in the source code and source code matching index. The source code matching index is the correspondence between each source code data and the source code matching key. For any function in the binary file, the source code matching index is queried sequentially using each binary matching parameter according to the priority order of the plurality of binary matching parameters. If the source code matching key corresponding to any binary matching parameter is found, the source code of the function, the binary data of the plurality of modalities and the source code data of the plurality of modalities are associated based on the query result to obtain fused data. Based on the fused data, a multimodal dataset corresponding to the data requirements is generated.

[0181] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0182] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0183] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for producing multimodal data oriented towards program understanding, characterized in that, include: The source code and binary file of the target program are parsed to obtain the source code matching index, as well as multiple binary matching parameters of each function, binary data of multiple modalities, and source code data of multiple modalities. The source code matching index is the correspondence between each source code data and the source code matching key. For any function in the binary file, the source code matching index is queried sequentially using each binary matching parameter according to the priority order of the plurality of binary matching parameters. If the source code matching key corresponding to any binary matching parameter is found, the source code of the function, the binary data of the plurality of modalities, and the source code data of the plurality of modalities are associated based on the query result to obtain fused data.

2. The multimodal data production method for program-oriented understanding according to claim 1, characterized in that, The plurality of binary matching parameters include the binary symbol of the function, the binary address range of the function, and the binary entry address of the function; the source code matching key includes the source code symbol key of the function, the source code address range key of the function, and the source code entry address key of the function; For any function in the binary file, the process of sequentially querying the source code matching index using each of the plurality of binary matching parameters according to their priority order includes: The binary symbol of the function is used to search for the source code symbol key corresponding to the binary symbol in the source code matching index; If the source code symbol key is not found, the binary address range of the function is used to search for the source code address range key corresponding to the binary address range in the source code matching index; If the source code address range key is not found, the binary entry address of the function is used to search for the source code entry address key corresponding to the binary entry address in the source code matching index.

3. The multimodal data production method for program-oriented understanding according to claim 1, characterized in that, Before parsing the source code and binary files of the target program, the following steps are also included: The source code of the target program is compiled to obtain the binary file and a source path mapping table between the source code and the binary file, specifically including: A bootstrap compiler toolchain is constructed, which is generated by iteratively compiling the compiler itself using the compiler source code; Analyze the dependencies of the target program and generate a dependency management strategy; In an isolated compilation environment, the bootstrap compiler toolchain is used to compile the source code of the dependent libraries and the target program in sequence according to the dependency management strategy, so as to obtain the binary file and the source code path mapping table.

4. The multimodal data production method for program understanding according to claim 3, characterized in that, The source code of the target program is parsed, including: Extract the source file path and source code line number of the function from the source code path mapping table; Locate the source code file based on the source file path, and extract the source code fragment of the function from the source code file based on the source code line number; Construct an input sequence containing task instructions and the source code fragment; The input sequence is input into a pre-trained large language model to obtain the natural language functional description and pseudocode representation of the function output by the large language model; The source code fragment, the natural language function description, and the pseudocode representation are used as the source code data for the multiple modalities; Multiple source code matching keys are determined using the source code file, and the source code matching index is constructed based on each source code data and each source code matching key.

5. The multimodal data production method for program understanding according to claim 4, characterized in that, The step of determining multiple source code matching keys using the source code file and constructing the source code matching index based on each source code data and each source code matching key includes: Extract the source code symbol of each function in the source code file, calculate the hash value of the unique identifier of the binary file and the source code symbol, and obtain the source code symbol key of the function; Extract the source code address range of each function in the source code from the source code path mapping table, calculate the hash value containing the unique identifier of the binary file and the source code address range, and obtain the source code address range key of the function; Extract the source code entry address of each function in the source code file, calculate the hash value containing the unique identifier of the binary file and the source code entry address, and obtain the source code entry address key of the function; Use the source code symbol key, the source code address range key, and the source code entry address key as the multiple source code matching keys; Establish an index relationship to obtain the source code matching index; wherein, the index relationship represents that the source code symbol key, the source code address range key, and the source code entry address key point to the source code data of the multiple modalities.

6. The multimodal data production method for program understanding according to claim 4, characterized in that, The step of extracting the source code fragment of the function from the source code file according to the source code line number includes: Based on the source code line number, locate an initial length context window in the source code file, and use the source code in the initial length context window as a candidate segment; If the candidate fragment does not contain the start and end identifiers of the function, the length of the context window is expanded by a preset step size, and the candidate fragment is extracted again until the candidate fragment contains the start and end identifiers of the function, or the length of the context window reaches the maximum window length. The final extracted candidate fragment is used as the source code fragment of the function.

7. The multimodal data production method for program understanding according to claim 1, characterized in that, The binary file of the target program is parsed, including: Convert the binary file into assembly code; Convert assembly instructions in assembly code into intermediate representation code containing standard opcodes; The intermediate representation code is then promoted to decompiled code; Parse the symbol table of the binary file to obtain the binary symbol of the function, the binary address range of the function, and the binary entry address of the function; The assembly code, the intermediate representation code, and the decompiled code are used as binary data for the multiple modalities, and the binary symbols, the binary address range, and the binary entry address are used as the multiple binary matching parameters. The binary data of the multiple modalities and the multiple binary matching parameters are serialized into a structured format in units of functions to obtain the parsing result. Write the parsing results into the cache.

8. The multimodal data production method for program understanding according to claim 7, characterized in that, After converting the assembly instructions in the assembly code into intermediate representation code containing standard opcodes, it also includes: The intermediate representation code is refined, and the refined intermediate representation code is used as part of the binary data of the multiple modalities; The refinement of the intermediate representation code includes: Based on the intermediate representation code, the basic blocks and jump relationships within the identification function are used to construct a control flow graph. Based on the control flow graph, reverse data flow analysis is performed starting from the function return point to obtain the parameter registers and effective operations that contribute to the final output of the function. Based on all the parameter registers and all the valid operations, temporary variables and redundant operations are filtered out to obtain the refined intermediate representation code.

9. The multimodal data production method for program understanding according to claim 8, characterized in that, Based on the control flow graph, reverse data flow analysis is performed starting from the function return point to obtain the parameter registers and valid operations that contribute to the final output of the function, including: When the number of execution paths in the control flow graph does not exceed a preset threshold, all execution paths are traversed to identify the parameter register and the valid operation; When the number of execution paths in the control flow graph exceeds the preset threshold, multiple rounds of random path traversal are performed, and the parameter register and the valid operation are identified in each round of traversal.

10. The multimodal data production method for program-oriented understanding according to claim 1, characterized in that, After obtaining the fused data, it also includes: Based on the fused data, a multimodal dataset corresponding to the data requirements is generated.

11. A multimodal data production device oriented towards program understanding, characterized in that, include: The parsing module is used to parse the source code and binary file of the target program to obtain the source code matching index, as well as multiple binary matching parameters of each function, binary data of multiple modalities, and source code data of multiple modalities. The source code matching index is the correspondence between each source code data and the source code matching key. The heterogeneous data pairing module is used to, for any function in the binary file, query the source code matching index sequentially using each of the multiple binary matching parameters according to the priority order of the binary matching parameters. If the source code matching key corresponding to any of the binary matching parameters is found, the module associates the source code of the function, the binary data of the multiple modalities, and the source code data of the multiple modalities based on the query result to obtain fused data.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the multimodal data production method for program-oriented understanding as described in any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium, wherein a computer program is stored on the non-transitory computer-readable storage medium, characterized in that, When the computer program is executed by a processor, it implements the multimodal data production method for program-oriented understanding as described in any one of claims 1 to 10.

14. A computer program product, the computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the multimodal data production method for program-oriented understanding as described in any one of claims 1 to 10.

Images