Compiling method, device, equipment, storage medium and program product

Abstract

The application discloses a compiling method and device, equipment, storage medium and program product, and belongs to the technical field of data processing. In the application, in response to a compiling instruction of target software, for each source file in a plurality of source files of the target software, first feature information corresponding to current content of the source file and second feature information corresponding to historical content of the source file are acquired; the first feature information and the second feature information of each source file are compared; for a first source file with an inconsistent comparison result and a second source file associated with the first source file, a compiling process is performed to obtain a current compiling file, and historical compiling files of other source files in the plurality of source files except the first source file and the second source file are acquired; and the current compiling file and the historical compiling files are combined to form a file package of the target software. The application improves the compiling efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a compilation method, apparatus, device, storage medium, and program product. Background Technology

[0002] As front-end application projects grow increasingly large, a medium-sized project typically contains hundreds or even thousands of source files. Front-end build tools are responsible for compiling and packaging these source files to generate runnable artifacts.

[0003] Currently, mainstream front-end compilation solutions employ a full compilation mode, performing compilation processing on all source files each time compilation is triggered. However, developers typically only modify a small number of source files each time, and full compilation results in a significant waste of computing resources on repeatedly compiling unchanged source files, leading to long compilation times and severely impacting compilation efficiency. Summary of the Invention

[0004] This application provides a compilation method, apparatus, device, storage medium, and program product, which can solve the problem of low compilation efficiency. The technical solution is as follows: On the one hand, a compilation method is provided, the method comprising: In response to the compilation instructions of the target software, for each of the multiple source files of the target software, first feature information corresponding to the current content of the source file and second feature information corresponding to the historical content of the source file are obtained; The first feature information of each source file is compared with the second feature information; For the first source file whose comparison result is inconsistent, and the second source file associated with the first source file, a compilation process is performed to obtain the current compiled file, and the historical compiled files of the other source files in the plurality of source files other than the first source file and the second source file are obtained; The compiled files and the historical compiled files are combined to form the target software file package.

[0005] In one possible implementation, the first feature information includes a first hash value obtained by hashing the current content of the source file, and the second feature information includes a second hash value obtained by hashing the historical content of the source file.

[0006] In another possible implementation, the process of determining the second source file associated with the first source file includes: Obtain the dependency relationships between the multiple source files; Based on the dependency relationship, a second source file associated with the first source file is determined.

[0007] In another possible implementation, the dependency relationship includes a dependency graph between the plurality of source files, wherein nodes in the dependency graph represent source files and edges in the dependency graph represent the dependency directions between source files; The step of determining the second source file associated with the first source file based on the dependency relationship includes: Starting from the node corresponding to the first source file, traverse in the opposite direction of the dependency direction corresponding to the edge to obtain the second source file associated with the first source file.

[0008] In another possible implementation, the execution of the compilation process includes: The first source file and the second source file are assigned to a first number of worker threads, which then perform the compilation process in parallel.

[0009] In another possible implementation, before assigning the first source file and the second source file to the first number of worker threads, the method further includes: The first quantity is determined based on the device information of the currently running device, wherein the device information includes at least one of the following: the number of processor cores, the available memory capacity, and historical compilation performance metrics.

[0010] In another possible implementation, before assigning the first source file and the second source file to the first number of worker threads, the method further includes: Determine the data volume of each source file in the first source file and the second source file, and divide the source files in the first source file and the second source file whose data volume is greater than or equal to the specified data volume into multiple sub-files.

[0011] In another possible implementation, the execution of the compilation process includes: Configure the corresponding task number for the compilation process; In response to the completion of the compilation process, the compilation result is obtained, and it is determined whether the task number corresponding to the compilation result is consistent with the task number of the latest configuration. If the task number corresponding to the compilation result is inconsistent with the task number of the latest configuration, the compilation result is discarded.

[0012] On the other hand, a compilation apparatus is provided, the apparatus comprising: The acquisition module is configured to, in response to the compilation instructions of the target software, acquire, for each of the multiple source files of the target software, first feature information corresponding to the current content of the source file and second feature information corresponding to the historical content of the source file; The comparison module is configured to compare the first feature information of each source file with the second feature information; The compilation module is configured to perform compilation processing on a first source file whose comparison result is inconsistent and a second source file associated with the first source file to obtain the current compiled file, and to obtain the historical compiled files of other source files in the plurality of source files other than the first source file and the second source file; The packaging module is configured to combine the currently compiled files and the previously compiled files into a file package for the target software.

[0013] In one possible implementation, the first feature information includes a first hash value obtained by hashing the current content of the source file, and the second feature information includes a second hash value obtained by hashing the historical content of the source file.

[0014] In another possible implementation, the compilation module is used for: Obtain the dependency relationships between the multiple source files; Based on the dependency relationship, a second source file associated with the first source file is determined.

[0015] In another possible implementation, the dependency relationship includes a dependency graph between the plurality of source files, wherein nodes in the dependency graph represent source files and edges in the dependency graph represent the dependency directions between source files; The compilation module is used for: Starting from the node corresponding to the first source file, traverse in the opposite direction of the dependency direction corresponding to the edge to obtain the second source file associated with the first source file.

[0016] In another possible implementation, the compilation module is used for: The first source file and the second source file are assigned to a first number of worker threads, which then perform the compilation process in parallel.

[0017] In another possible implementation, the compilation module is used for: The first quantity is determined based on the device information of the currently running device, wherein the device information includes at least one of the following: the number of processor cores, the available memory capacity, and historical compilation performance metrics.

[0018] In another possible implementation, the compilation module is used for: Determine the data volume of each source file in the first source file and the second source file, and divide the source files in the first source file and the second source file whose data volume is greater than or equal to the specified data volume into multiple sub-files.

[0019] In another possible implementation, the compilation module is used for: Configure the corresponding task number for the compilation process; In response to the completion of the compilation process, the compilation result is obtained, and it is determined whether the task number corresponding to the compilation result is consistent with the task number of the latest configuration. If the task number corresponding to the compilation result is inconsistent with the task number of the latest configuration, the compilation result is discarded.

[0020] On the other hand, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the method described in any of the above.

[0021] On the other hand, a non-transitory computer-readable storage medium is provided, the non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method described in any of the preceding claims.

[0022] On the other hand, a computer program product is provided, including computer program instructions that, when run on a computer, cause the computer to perform the method described in any of the preceding claims.

[0023] The beneficial effects of the technical solution provided in this application are: by comparing the current and historical content of each source file, it is possible to quickly determine which source files have been changed. Only the changed files and their related source files are compiled; other unchanged files directly use the previously compiled results. The workload of each compilation depends only on the actual scope of the changes, rather than the total number of source files in the entire project, which not only ensures the integrity of the output but also improves compilation efficiency. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application; Figure 2 This is a flowchart of the compilation method provided in the embodiments of this application; Figure 3 This is a schematic diagram of the compiler device structure provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0027] With the rapid development of front-end technologies, the scale of modern front-end applications is becoming increasingly large. A medium-sized front-end project typically contains hundreds or even thousands of source files, with complex references and dependencies between them. Front-end compilation tools are responsible for compiling and packaging these source files to generate the final product that can run in a browser. During development, developers need to trigger a compilation to preview the effect every time they modify the code, and the compilation time directly affects development efficiency.

[0028] Currently, mainstream front-end compilation solutions adopt a full compilation mode. Specifically, each time compilation is triggered, the compilation tool obtains all source files in the project, performs compilation processing on each source file, obtains the corresponding compiled file, and finally combines the compiled files of all source files into the output of this compilation. Even if a source file has not changed since the last compilation, it will still be recompiled.

[0029] However, as the project grows, the number of source files increases. But developers usually only modify a small number of source files each time. Compiling all source files every time results in a lot of computing resources being wasted on repeatedly compiling unchanged source files. This leads to long compilation times and seriously affects compilation efficiency.

[0030] This application provides a compilation method. This method can be applied to a terminal. For example... Figure 1 As shown, the terminal may include a processor 110, a memory 120, and a communication component 130.

[0031] The processor 110 can be a central processing unit (CPU), graphics processing unit (GPU), microcontroller unit (MCU), accelerated processing unit (APU), neural processing unit (NPU), tensor processing unit (TPU), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), digital signal processor (DSP), etc.

[0032] The memory 120 may include volatile memory and / or non-volatile memory. Volatile memory may include random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), etc. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, non-volatile random access memory (NVRAM), etc.

[0033] The communication component 130 can be a wireless communication module (WCM), a subscriber identity module (SIM), a universal subscriber identity module (USIM), an optical network unit (ONU), etc. The communication component 130 can be used for communication between the terminal and a server, or other terminals.

[0034] This application provides a compilation method, such as Figure 2 As shown, in some embodiments, the method includes: S201. In response to the compilation instructions of the target software, for each of the multiple source files of the target software, obtain the first feature information corresponding to the current content of the source file and the second feature information corresponding to the historical content of the source file.

[0035] The target software can be a front-end project built based on multi-file collaboration. These multiple source files include, but are not limited to, at least one of TypeScript source files, JavaScript source files, cascading stylesheets (CSS) files, hypertext markup language (HTML) files, and project configuration files. The project configuration files include, but are not limited to, package entry point configuration, module parsing path configuration, and compilation option configuration. The compilation command is used to trigger the compilation and build operation of the target software. The triggering methods for the compilation command include, but are not limited to: the user performing a save operation in an online low-code platform or integrated development environment (IDE), triggering a preview operation, clicking the build button, or automatic triggering after the system detects file changes. In an optional implementation, the compilation command carries a list of source file paths that trigger compilation, used to identify the directly modified files in this compilation. The feature information is a data fingerprint that uniquely represents the file content; when the file content changes arbitrarily, its corresponding feature information changes accordingly.

[0036] In specific implementation, for each source file of the target software, the current content of the source file is read from the currently loaded virtual file system. The virtual file system is a file tree structure maintained in the browser's memory, a mapping relationship with path strings as keys and file content as values. The virtual file system can be directly provided by the file editor of an online IDE, or instantiated locally after being pulled remotely. A hash calculation is performed on the current content to obtain a first hash value, which serves as the first feature information. Simultaneously, the historical content of the source file recorded during the last successful compilation is read from the build cache or local persistent storage, and a hash calculation is performed on the historical content in the same manner to obtain a second hash value, which serves as the second feature information. The historical content is a snapshot of the source file content read and saved from the virtual file system during the last successful compilation. The difference between the historical content and the current content is: if the user edited the source file between two compilations, the current content is inconsistent with the historical content; if the user did not edit it, the two are consistent.

[0037] In one optional implementation, the hash calculation algorithm used includes, but is not limited to, any one of MD5, SHA-1, SHA-256, or CRC32. In one example, the SHA-256 algorithm is used to achieve a balance between high collision resistance and computational efficiency. For small files (e.g., files less than or equal to a preset byte threshold, such as 256KB), the hash calculation is performed directly on the entire file content. For large files (e.g., files larger than the preset byte threshold), the file content is divided into multiple fixed-size data blocks (e.g., each data block is 64KB), the hash value of each data block is calculated separately, and then a second hash calculation is performed on the sequence formed by concatenating the hash values ​​of each block in order to obtain the final file hash value. By using block hashing, when only a partial modification occurs to the file, only the hash value of the modified data block needs to be recalculated, and the hash values ​​of the remaining data blocks can be directly reused, further reducing the hash calculation overhead.

[0038] In one optional implementation, if the source file is being compiled for the first time (i.e., there is no historical content recorded during the previous successful compilation), a preset default feature value (e.g., a string of all zeros) is generated for the source file as the second feature information to ensure that the first feature information and the second feature information are necessarily inconsistent during the first compilation, thereby ensuring that the source file is correctly included in the scope of this compilation.

[0039] S202. Compare the first feature information with the second feature information of each source file.

[0040] The comparison involves performing a content consistency check on each source file separately to determine whether the content of the source file has changed since the last compilation. The comparison result is a binary judgment: consistent (indicating the file has not changed) or inconsistent (indicating the file has changed).

[0041] In practice, for each source file, its corresponding first hash value and second hash value are compared string by string. If the first hash value and the second hash value are exactly the same, it is determined that the current content of the source file is consistent with the historical content, the source file has not been changed, and no recompilation is required; if the first hash value and the second hash value differ in any single bit, it is determined that the current content of the source file is inconsistent with the historical content, the source file has been changed, and recompilation is required.

[0042] In one example, a hash map is maintained, with file paths as keys and hash values ​​as values. This hash map is updated after each successful compilation, recording the content fingerprints of each source file during this compilation. Upon each compilation trigger, all source files in the virtual file system are traversed. For each source file, a first hash value of its current content is calculated, and then a second hash value is retrieved from the hash map using the file path as the key for comparison. All source files with inconsistent hash values ​​are collected into a changed set, and the source files in this changed set are the first source file. All source files with consistent hash values ​​are collected into an unchanged set.

[0043] In another example, to improve comparison efficiency, a parallel comparison strategy can be adopted: all source files are divided into multiple batches according to a preset number, and hash values ​​are calculated and compared sequentially within each batch. Multiple batches can be processed in parallel. For large front-end projects containing thousands of source files, hash comparison of all files can usually be completed within tens of milliseconds.

[0044] By comparing files one by one based on content fingerprints, change detection can be completed with a time complexity that is linearly related to the number of source files, and the detection speed is much faster than full compilation.

[0045] S203. For the first source file whose comparison result is inconsistent, and the second source file associated with the first source file, perform compilation processing to obtain the current compiled file, and obtain the historical compiled files of the other source files in the plurality of source files other than the first source file and the second source file.

[0046] The first source file is one whose current content differs from historical content and requires recompilation. The second source file is one whose content has not changed, but which is affected by changes to the first source file due to a dependency relationship with it and requires synchronous recompilation. The compilation process includes, but is not limited to, at least one of the following: TypeScript to JavaScript transpile processing, code minification processing, tree-shaking processing, and static resource processing. The static resource processing includes, but is not limited to, image compression, stylesheet preprocessing, and font file inlining.

[0047] In practice, the first source file is fed into the compilation pipeline, where it undergoes sequential compilation sub-steps such as transpilation, compression, and tree shaking to obtain the current compiled file corresponding to the first source file. For each second source file associated with the first source file, it is similarly fed into the compilation pipeline for compilation processing, resulting in the current compiled file corresponding to each second source file. It should be noted that although the content of the second source file remains unchanged, its compilation output depends on the compilation output of the first source file (for example, the second source file references a modified exported interface from the first source file), therefore, it also needs to be recompiled to ensure consistency of the output.

[0048] For the other source files in the multiple source files other than the first source file and the second source file (i.e., the source files in the set of unchanged files that are not affected by the changes), since their contents have not changed and they are not affected by the dependencies of the changed files, the historical compilation products generated by the other source files in the last compilation are directly read from the cache as the historical compilation files, and the recompilation step is skipped.

[0049] In one example, the cache is an in-memory key-value store structure, with the file path and / or content hash of the source file as the key and the compilation artifact as the value. In another example, the cache is a content-addressed entry in persistent storage, with a combination of source file identifier, compilation stage identifier, and content feature information as the key and the compilation artifact of the corresponding stage as the value. If the corresponding historical compilation file is not present in the cache (e.g., the cache has been cleared or the entry was evicted due to insufficient storage space), the compilation process for that source file is re-executed.

[0050] Only the changed files and their dependent files are recompiled. Unchanged and unaffected files directly reuse the historical compilation artifacts. This makes the time taken for incremental compilation close to the size of the change rather than the overall project size. In typical use cases where users make small edits (e.g., only 1 to 3 files are modified each time), the compilation time is significantly lower than that of full compilation, achieving a compilation response speed of seconds or even sub-seconds.

[0051] S204. Combine the current compilation files and the historical compilation files into a file package for the target software.

[0052] The file package is a complete collection of build artifacts output by the target software after this compilation, containing compiled files corresponding to all source files in the project, regardless of whether the compiled file is newly generated in this compilation or a historical compiled file reused from the cache. Each compiled file in the file package maintains a path mapping relationship with its corresponding source file, so that upper-layer services can locate the corresponding compiled artifact based on the source file path.

[0053] In practice, the compiled file obtained in this instance is merged with the historical compiled file retrieved from the cache according to the module structure of each source file in the target software to form a complete file package. In one example, the file package includes a transpiled JavaScript module file, a compressed code file, a tree-optimized export set, and processed static resource files. The file package can be directly used for page loading and previewing on the browser side, or it can be further packaged and merged to generate one or more bundle files.

[0054] In one optional implementation, during the process of assembling the file package, the source map files of the currently compiled files and the historically compiled files are merged and updated. For the newly compiled files, a corresponding source map file is generated, recording the mapping relationship between each position in the compiled artifact and the corresponding position in the original source file; for historically compiled files reused from the cache, their existing source map files are directly reused. All source map files are merged into a complete set of mapping files according to the module structure and output along with the file package, so that when debugging on the browser side, the error positions in the compiled artifact can be traced back to the original source code.

[0055] In this embodiment, by combining the current compilation files and historical compilation files into a complete file package, the upper-layer business does not need to distinguish the compilation source of each file to obtain a complete build product consistent with the full compilation result, thus ensuring the integrity and correctness of the incremental compilation product; at the same time, the debuggability of the compilation product is ensured by merging the source map output.

[0056] In some embodiments, the first feature information includes a first hash value obtained by hashing the current content of the source file, and the second feature information includes a second hash value obtained by hashing the historical content of the source file.

[0057] In practice, the current content of the source file is digested using the SHA-256 algorithm to obtain the first hash value; the historical content of the source file is digested using the same algorithm to obtain the second hash value. The SHA-256 algorithm outputs a 256-bit (32-byte) digest value, which is usually represented as a 64-bit hexadecimal string.

[0058] In one implementation, the current content of the source file is a complete byte sequence. A SHA-256 hash operation is directly performed on this byte sequence to obtain a 32-byte first hash value. In another implementation, when the source file is large, the file content is divided into multiple data blocks according to a preset block size (e.g., 64KB). A SHA-256 operation is performed on each data block to obtain a block hash value sequence. These block hash value sequences are then concatenated sequentially and a second SHA-256 operation is performed to obtain the final first hash value. The calculation method for the second hash value is the same as that for the first hash value, except that the calculation object is the historical content recorded during the last compilation.

[0059] In another alternative implementation, for scenarios requiring faster speed but allowing extremely low collision probabilities (such as internal development environments or projects with a small number of files), the CRC32 algorithm can be used instead of the SHA-256 algorithm. CRC32 is significantly faster than SHA-256 and has sufficient ability to distinguish non-maliciously crafted file content. For security-sensitive scenarios (such as open-source collaboration platforms), SHA-256 or a higher-level hash algorithm should be preferred.

[0060] In this embodiment, a hash value is used as the content fingerprint, which can achieve accurate content change identification with minimal storage and computational overhead while ensuring an extremely low probability of collision.

[0061] In some embodiments, the process of determining a second source file associated with the first source file includes: Obtain the dependencies between the multiple source files; determine the second source file associated with the first source file based on the dependencies.

[0062] The dependency relationships represent the import and / or reference relationships between the multiple source files. Specifically, when source file A imports the exported content of source file B through an import or require statement, source file A depends on source file B, and changes to the content of source file B may cause changes to the compilation output of source file A. These dependencies include, but are not limited to: import / export dependencies of ES modules, require dependencies of CommonJS, @import dependencies of CSS, script references and stylesheet references in HTML, and resource reference paths in project configuration files.

[0063] In practice, during the initialization phase of the target software project or whenever the dependency structure may change, static analysis is used to scan each source file for import declarations, dynamic import expressions, require calls, CSS @import directives, HTML script and link tags, and resource reference paths in configuration files to extract import and reference relationships between files. The static analysis process includes, but is not limited to: performing lexical and syntactic analysis on the source files to extract module specifiers from import statements; performing path resolution on the module specifiers, resolving the referenced module specifiers into specific file paths according to preset module resolution rules (e.g., TypeScript's moduleResolution configuration); and constructing a dependency graph with each source file as a node and the import direction as a directed edge. Upon detecting a change in the first source file, other source files with dependency relationships to the first source file are searched in the dependency graph, and the source files affected by the change in the first source file are identified as the second source files.

[0064] In one alternative implementation, the dependency graph is stored using an adjacency list data structure. Each node maintains an outgoing edge list (recording other files it depends on) and an incoming edge list (recording other files that depend on it). When searching for a second source file, the incoming edge list can be used to efficiently locate all downstream files that directly reference the first source file.

[0065] In this embodiment, by obtaining the dependency relationships between source files and determining the associated files accordingly, it can be ensured that not only the changed file itself is recompiled, but all downstream files that are semantically dependent on the changed file are also updated synchronously, avoiding the problem of outdated compilation products caused by cache misses; at the same time, by covering multiple dependency types, the integrity of the dependency relationships is ensured.

[0066] In some embodiments, the dependency relationship includes a dependency graph between the plurality of source files, where nodes in the dependency graph represent source files and edges in the dependency graph represent dependency directions between source files; determining the second source file associated with the first source file based on the dependency relationship includes: starting from the node corresponding to the first source file, traversing in the opposite direction of the dependency direction corresponding to the edge to obtain the second source file associated with the first source file.

[0067] The dependency graph is a directed graph, where nodes are source files and directed edges point from the dependent file to the dependent file. For example, when source file A imports an exported function from source file B using an import statement, there exists a directed edge from node A to node B, indicating that A depends on B, and the dependency direction is A→B. Traversing in the reverse direction of the dependency, that is, traversing from the dependent file (B) towards the dependent file (A), is used to find all files that directly or indirectly reference the first source file.

[0068] In practice, starting from the node corresponding to the first source file, a reverse traversal is performed along the edges of the dependency graph. In one example, a breadth-first search (BFS) algorithm is used, starting from the node corresponding to the first source file and expanding layer by layer along the reverse direction of the directed edges, collecting the source files corresponding to all reachable nodes as the second source file. In another example, a depth-first search (DFS) algorithm is used to perform the same reverse traversal. The traversal terminates when: the current node has no incoming edges in the reverse direction (i.e., no other file references the current node), or the current node has been visited (to avoid getting stuck in an infinite loop when there is a circular dependency).

[0069] For example, if source file C imports source file B, and source file B imports source file A in a dependency graph, then there exist edges C→B and B→A. When source file A changes, traversing backwards from node A, we first reach node B (B depends on A), and then from node B reach node C (C depends on B). Therefore, both source file B and source file C are identified as secondary source files and need to be recompiled along with source file A. Even if source file C itself is not modified and there is no direct dependency between it and source file A, because source file C indirectly depends on source file A (via source file B), the compilation output of source file C may be indirectly affected by changes to source file A and also needs to be recompiled.

[0070] In one alternative implementation, for cases with circular dependencies (e.g., source file X imports source file Y, and source file Y imports source file X), an infinite loop is avoided by maintaining a visited set during traversal. When a node marked as visited is encountered, it is skipped and expansion is stopped.

[0071] In another alternative implementation, for large projects (e.g., projects containing thousands of source files), the dependency graph can be preprocessed to pre-calculate the transitive closure of each node, i.e., pre-record all upstream dependent nodes and all downstream dependent nodes of each node. When a change occurs, the set of downstream dependent nodes of the node corresponding to the first source file is directly queried, without the need for real-time traversal, further shortening the incremental boundary calculation time.

[0072] In this embodiment, the downstream closure of the changed files is accurately calculated by reverse traversing the dependency graph, ensuring that all files semantically affected by the changes are included in the recompilation scope. This avoids the risk of omissions caused by overly narrow propagation, as well as the problem of degrading to a near-full build caused by overly broad propagation.

[0073] In some embodiments, performing the compilation process includes: The first source file and the second source file are assigned to a first number of worker threads, which then perform the compilation process in parallel.

[0074] The worker thread is an independent execution context implemented based on Web Worker technology. In a browser environment, the worker thread is a Web Worker (including Dedicated Worker or Shared Worker), each with its own execution stack, heap memory, and event loop, enabling it to execute computationally intensive tasks in parallel with the browser's main thread without blocking user interface responsiveness. In browser-like runtimes (e.g., based on embedded JavaScript runtimes), the worker thread is an equivalent independent execution context.

[0075] In specific implementation, a compilation task scheduler is constructed, which maintains a task queue and a worker thread pool consisting of the first number of worker threads. Each source file in the set of files to be compiled, consisting of the first source file and the second source file, is encapsulated as a compilation task. The compilation task includes at least the file path, file content, compilation configuration parameters, and compilation type (e.g., transpilation, compression, tree shaking, etc.) of the source file to be compiled. The compilation tasks are placed into the task queue according to a preset strategy, and idle worker threads retrieve tasks from the task queue and execute them.

[0076] The preset strategies include, but are not limited to: First-In-First-Out (FIFO) strategy, which allocates tasks sequentially according to the order in which they are enqueued; priority strategy, which sorts source files according to their topological hierarchy in the dependency graph, prioritizing source files with shallower dependency levels (depended on by more files) to release the compilation wait of downstream files as soon as possible; and load balancing strategy, which dynamically allocates tasks based on the current task completion status and expected remaining workload of each worker thread, assigning tasks to the worker thread with the lightest current load.

[0077] Each worker thread receives its assigned source file in an independent execution context, performs compilation subtasks such as transpilation and compression, and returns the compiled artifacts to the main thread via the postMessage mechanism after compilation is complete. The main thread performs a merge operation after collecting the compiled artifacts from all worker threads. Because multiple worker threads execute compilation in parallel, the parallel computing capabilities of multi-core processors can be fully utilized.

[0078] In one alternative implementation, the worker threads in the worker thread pool can be reused: when a compilation task is completed, the corresponding worker thread is not destroyed, but enters an idle state to wait for the next task, thereby avoiding the overhead of frequently creating and destroying worker threads.

[0079] In this embodiment, by distributing compilation tasks to multiple worker threads for parallel execution, the parallel capabilities of multi-core processors are applied to browser-side compilation scenarios, significantly shortening the clock time for incremental compilation. At the same time, since the compilation tasks are executed in worker threads, the main thread is not blocked by the compilation tasks, maintaining a smooth response of the user interface, allowing users to continue editing files during the compilation process without being affected.

[0080] In some embodiments, before assigning the first source file and the second source file to a first number of worker threads, the method further includes: The first quantity is determined based on the device information of the currently running device, wherein the device information includes at least one of the following: the number of processor cores, the available memory capacity, and historical compilation performance metrics.

[0081] The number of processor cores is obtained through the browser's hardware concurrency interface (such as navigator.hardwareConcurrency), which returns the number of logical processors currently available to the browser. The available memory capacity is obtained through the browser's device memory interface (such as navigator.deviceMemory), which returns the approximate memory capacity of the device (an enumerated value in GB). The historical compilation performance metrics are performance data recorded during each compilation process, including but not limited to the total compilation time, throughput at each stage (number of compilation tasks completed per unit time), actual utilization of worker threads (ratio of actual working time to total working time), and peak memory usage during the compilation process.

[0082] In specific implementation, firstly, the number of processor cores (cores) of the current device is obtained through the hardware concurrency interface, and the available memory capacity (memory_gb) is obtained through the device memory interface. Then, based on the number of processor cores and the available memory capacity, the memory tier (memory_tier) is calculated: for example, when memory_gb is greater than or equal to 8, memory_tier is 3; when memory_gb is greater than or equal to 4 and less than 8, memory_tier is 2; when memory_gb is less than 4, memory_tier is 1. Next, the ratio of actual throughput to theoretical throughput (throughput_ratio) from historical compilation performance metrics is obtained (if no historical data exists, throughput_ratio takes the default value of 1.0). The device capability score is calculated as: score = α × cores + β × memory_tier + γ × throughput_ratio, where α, β, and γ are preset weighting coefficients; in one example, α = 1.0, β = 0.5, and γ = 0.3.

[0083] Subsequently, the first quantity is determined based on the device capability score: first_n = clamp( The function `clamp` restricts the value to the range [W_min, W_max], where W_min is the preset minimum number of worker threads (e.g., 1) and W_max is the preset maximum number of worker threads (e.g., not exceeding the value of `cores`). This indicates rounding the score down.

[0084] For example, for a high-performance desktop computer equipped with an 8-core processor and 16GB of RAM, cores=8, memory_tier=3, throughput_ratio=1.0, score=1.0×8+0.5×3+0.3×1.0=9.8, first_n=clamp(9,1,8)=8, meaning 8 worker threads are allocated to fully utilize the multi-core computing power. For a mid-range laptop equipped with a 4-core processor and 8GB of RAM, cores=4, memory_tier=3, score=5.8, first_n=clamp(5,1,4)=4. For a low-power mobile device equipped with a 2-core processor and 4GB of RAM, cores=2, memory_tier=1, score=2.8, first_n=clamp(2,1,2)=2, avoiding memory pressure and thread contention issues caused by too many worker threads.

[0085] In this embodiment, the number of worker threads is adaptively determined based on device information, so that the compilation concurrency can be dynamically adjusted according to the actual processing power of the terminal device. On high-end devices, the computing power is fully utilized to pursue the fastest compilation speed, while on low-end devices, the concurrency is appropriately reduced to ensure system stability and smooth user interface. This achieves cross-terminal adaptive compilation optimization without requiring developers to manually configure concurrency parameters for different devices.

[0086] In some embodiments, before assigning the first source file and the second source file to a first number of worker threads, the method further includes: Determine the data volume of each source file in the first source file and the second source file, and divide the source files in the first source file and the second source file whose data volume is greater than or equal to the specified data volume into multiple sub-files.

[0087] The data volume refers to the size of the source file in bytes, which can be directly obtained through the file system interface or the length of the content in memory. The specified data volume is a preset shard threshold, which in one example has a default value of 100KB. The shard threshold can be dynamically adjusted according to device performance or project configuration.

[0088] In practice, before assigning files to be compiled to worker threads, the compilation task scheduler first obtains the byte size of each source file to be compiled. For source files with a byte size smaller than the specified data size (hereinafter referred to as small files), no splitting is performed, and the entire file is directly assigned as a compilation task to a single worker thread for compilation. For source files with a byte size greater than or equal to the specified data size (hereinafter referred to as large files), a splitting operation is performed, dividing the large file into multiple sub-files (shards).

[0089] The segmentation methods include, but are not limited to, at least one of the following: First, uniform segmentation by byte range: The file content is divided into multiple consecutive byte segments according to a fixed number of bytes (e.g., the specified data size), with each byte segment serving as a sub-file. This method is simple to implement, but the segmentation boundaries may be located in the middle of statements or declarations, requiring additional concatenation processing during merging. Second, segmentation by the boundaries of the top-level declaration nodes of the abstract syntax tree (AST): First, the source file is parsed to construct a complete AST, and then the file is segmented into multiple semantically complete sub-files according to the boundaries of the top-level declaration nodes (e.g., top-level class declarations, top-level function declarations, top-level variable declarations, etc.) in the AST. Each sub-file contains one or more complete top-level declarations and their internal local declarations, ensuring the semantic integrity within the sub-file. Sub-files obtained by this method can be compiled independently, and merging only requires simple concatenation. Third, segmentation by grouping exported declarations of modules: The exported declarations of the source file are analyzed, and the exported declarations and their dependent internal declarations are grouped together to form multiple exported closure sub-files. Each sub-file corresponds to a portion of the original file's exported interfaces and can be compiled independently.

[0090] The resulting sub-files are distributed to different worker threads for parallel compilation. After each sub-file has been compiled, the compilation outputs of the sub-files are merged according to the structural order of the original file to obtain the complete compilation output of the large file. The merging process includes: concatenating the compilation outputs of the sub-files in the original declaration order, merging the source maps of the sub-files into a complete source map, and verifying whether the merged output covers all the exported interfaces of the original file.

[0091] For example, if a TypeScript source file is 500KB in size and the specified data size is 100KB, it can be divided into 3 to 5 semantically complete sub-files by splitting according to the AST boundary (the specific number depends on the number and distribution of top-level declarations in the file), which are then compiled in parallel by different worker threads and merged into a complete compilation product after compilation.

[0092] In this embodiment, by automatically splitting large files and compiling them in parallel by multiple worker threads, the problem of a single large file slowing down the overall compilation progress as a long-tail task is eliminated.

[0093] In some embodiments, performing the compilation process includes: Configure a corresponding task number for the compilation process; in response to the completion of the compilation process, obtain the compilation result and determine whether the task number corresponding to the compilation result is consistent with the currently latest configured task number; in response to the inconsistency between the task number corresponding to the compilation result and the currently latest configured task number, discard the compilation result.

[0094] The task number is a monotonically increasing sequence number used to uniquely identify a compilation session. Each time a new compilation instruction is triggered, the task number increments, ensuring a clear temporal sequence for task numbers across different compilation sessions. The task number can be a sequence of natural numbers starting from 1, or any other monotonically increasing numerical sequence.

[0095] In practice, a global build sequence number variable `buildSeq` is maintained, initially set to 0. Each time a new compilation instruction is received, the value of `buildSeq` is incremented (e.g., by performing `buildSeq = buildSeq + 1`), and the incremented value is used as the task number for the current compilation session. When a compilation task is assigned to a worker thread, the current task number `buildSeq` is appended to the compilation task's data structure. After the worker thread completes compilation, it includes this task number in the returned compilation result.

[0096] When the main thread receives the asynchronously returned compilation result, it performs a sequence number check: First, it extracts the task number (task_id) carried in the compilation result and reads the latest buildSeq value. If task_id equals buildSeq, it means that the compilation result corresponds to the latest compilation session, the result is not expired, and the compilation result can be merged into the final artifact, triggering the compilation completion event (onBuildComplete). If task_id is less than buildSeq (i.e., inconsistent), it means that in the period between when the compilation task was issued to the worker thread and before the compilation result was returned, the user triggered a new compilation instruction, and an updated compilation session is currently in progress (buildSeq has been incremented). The compilation result is expired and should be actively discarded to avoid the old compilation result overwriting the compilation artifact corresponding to the new edit state.

[0097] In one alternative implementation, while discarding expired compilation results, a cancellation signal can be sent to the corresponding worker thread to notify it to stop the currently executing compilation task and release the occupied computing resources so that the worker thread can take over the task of the new compilation session as soon as possible.

[0098] In another alternative implementation, the number of times expired compilation results are discarded can be counted and reported as an observable metric to analyze user editing habits and compilation frequency, thereby optimizing scheduling strategies (e.g., automatically increasing the delay time for compilation triggering when users edit frequently to reduce unnecessary intermediate compilations).

[0099] For example, a user rapidly edits and saves a file, triggering three compilation instructions within a short period, corresponding to task numbers 1, 2, and 3. If the compilation result for task 1 is returned last (e.g., due to the large number of changed files involved in task 1 and the longer compilation time), then `buildSeq` has been updated to 3. The task number 1 carried by the compilation result for task 1 is inconsistent with `buildSeq=3`, therefore this result is discarded. If the compilation result for task 3 is returned, its task number 3 is consistent with `buildSeq=3`, and this result is adopted and merged into the final output.

[0100] In this embodiment, by constructing a sequence number verification mechanism, in scenarios of rapid continuous saving or frequent editing, it is ensured that only the result of the latest compilation session is adopted, and expired compilation results are actively discarded. This fundamentally solves the problem of compilation result overwriting caused by the last-to-first-to-arrive in asynchronous compilation, and ensures the consistency between the compilation product and the latest editing state.

[0101] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0102] Based on the same inventive concept, and corresponding to the compilation method provided in the embodiments of this application, this application also provides a compilation apparatus.

[0103] refer to Figure 3 The compilation apparatus includes: The acquisition module 301 is configured to, in response to the compilation instructions of the target software, acquire, for each of the multiple source files of the target software, first feature information corresponding to the current content of the source file and second feature information corresponding to the historical content of the source file; The comparison module 302 is configured to compare the first feature information of each source file with the second feature information; The compilation module 303 is configured to perform compilation processing on a first source file whose comparison result is inconsistent and a second source file associated with the first source file to obtain the current compiled file, and to obtain the historical compiled files of other source files in the plurality of source files other than the first source file and the second source file; The packaging module 304 is configured to combine the current compilation files and the historical compilation files into a file package of the target software.

[0104] In one possible implementation, the first feature information includes a first hash value obtained by hashing the current content of the source file, and the second feature information includes a second hash value obtained by hashing the historical content of the source file.

[0105] In another possible implementation, the compilation module 303 is used for: Obtain the dependency relationships between the multiple source files; Based on the dependency relationship, a second source file associated with the first source file is determined.

[0106] In another possible implementation, the dependency relationship includes a dependency graph between the plurality of source files, wherein nodes in the dependency graph represent source files and edges in the dependency graph represent the dependency directions between source files; The compilation module 303 is used for: Starting from the node corresponding to the first source file, traverse in the opposite direction of the dependency direction corresponding to the edge to obtain the second source file associated with the first source file.

[0107] In another possible implementation, the compilation module 303 is used for: The first source file and the second source file are assigned to a first number of worker threads, which then perform the compilation process in parallel.

[0108] In another possible implementation, the compilation module 303 is used for: The first quantity is determined based on the device information of the currently running device, wherein the device information includes at least one of the following: the number of processor cores, the available memory capacity, and historical compilation performance metrics.

[0109] In another possible implementation, the compilation module 303 is used for: Determine the data volume of each source file in the first source file and the second source file, and divide the source files in the first source file and the second source file whose data volume is greater than or equal to the specified data volume into multiple sub-files.

[0110] In another possible implementation, the compilation module 303 is used for: Configure the corresponding task number for the compilation process; In response to the completion of the compilation process, the compilation result is obtained, and it is determined whether the task number corresponding to the compilation result is consistent with the task number of the latest configuration. If the task number corresponding to the compilation result is inconsistent with the task number of the latest configuration, the compilation result is discarded.

[0111] It should be noted that the compilation device provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the compilation device and the compilation method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0112] Based on the same inventive concept, corresponding to the compilation method provided in the embodiments of this application, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the compilation method described in the above embodiments.

[0113] Figure 4 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0114] The processor 1010 can be implemented using a general-purpose CPU (central processing unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0115] The memory 1020 can be implemented in the form of ROM (read-only memory), RAM (random access memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0116] The input / output interface 1030 is used to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0117] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0118] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0119] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0120] The electronic devices described above are used to implement the corresponding compilation methods in the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0121] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to complete the compilation method described above. This computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be ROM (read-only memory), RAM (random access memory), CD-ROM (compact disc read-only memory), magnetic tape, floppy disk, and optical data storage devices, etc.

[0122] In an exemplary embodiment, a computer program product is also provided, including computer program instructions that, when executed on a computer, cause the computer to perform the compilation method described above.

[0123] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0124] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0125] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0126] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A compilation method, characterized in that, include: In response to the compilation instructions of the target software, for each of the multiple source files of the target software, first feature information corresponding to the current content of the source file and second feature information corresponding to the historical content of the source file are obtained; The first feature information of each source file is compared with the second feature information; For the first source file whose comparison result is inconsistent, and the second source file associated with the first source file, a compilation process is performed to obtain the current compiled file, and the historical compiled files of the other source files in the plurality of source files other than the first source file and the second source file are obtained; The compiled files and the historical compiled files are combined to form the target software file package.

2. The compilation method according to claim 1, characterized in that, The first feature information includes a first hash value obtained by hashing the current content of the source file, and the second feature information includes a second hash value obtained by hashing the historical content of the source file.

3. The compilation method according to claim 1, characterized in that, The process of determining the second source file associated with the first source file includes: Obtain the dependency relationships between the multiple source files; Based on the dependency relationship, a second source file associated with the first source file is determined.

4. The compilation method according to claim 3, characterized in that, The dependency relationship includes a dependency graph between the multiple source files, where nodes in the dependency graph represent source files and edges in the dependency graph represent the dependency directions between source files; The step of determining the second source file associated with the first source file based on the dependency relationship includes: Starting from the node corresponding to the first source file, traverse in the opposite direction of the dependency direction corresponding to the edge to obtain the second source file associated with the first source file.

5. The compilation method according to claim 1, characterized in that, The execution of the compilation process includes: The first source file and the second source file are assigned to a first number of worker threads, which then perform the compilation process in parallel.

6. The compilation method according to claim 5, characterized in that, Before allocating the first source file and the second source file to the first number of worker threads, the method further includes: The first quantity is determined based on the device information of the currently running device, wherein the device information includes at least one of the following: the number of processor cores, the available memory capacity, and historical compilation performance metrics.

7. The compilation method according to claim 5, characterized in that, Before allocating the first source file and the second source file to the first number of worker threads, the method further includes: Determine the data volume of each source file in the first source file and the second source file, and divide the source files in the first source file and the second source file whose data volume is greater than or equal to the specified data volume into multiple sub-files.

8. The compilation method according to any one of claims 1 to 7, characterized in that, The execution of the compilation process includes: Configure the corresponding task number for the compilation process; In response to the completion of the compilation process, the compilation result is obtained, and it is determined whether the task number corresponding to the compilation result is consistent with the task number of the latest configuration. If the task number corresponding to the compilation result is inconsistent with the latest configured task number, the compilation result is discarded.

9. A compilation apparatus, characterized in that, include: The acquisition module is configured to, in response to the compilation instructions of the target software, acquire, for each of the multiple source files of the target software, first feature information corresponding to the current content of the source file and second feature information corresponding to the historical content of the source file; The comparison module is configured to compare the first feature information of each source file with the second feature information; The compilation module is configured to perform compilation processing on a first source file whose comparison result is inconsistent and a second source file associated with the first source file to obtain the current compiled file, and to obtain the historical compiled files of other source files in the plurality of source files other than the first source file and the second source file; The packaging module is configured to combine the currently compiled files and the previously compiled files into a file package for the target software.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method described in any one of claims 1 to 8.

12. A computer program product comprising computer program instructions, characterized in that, When the computer program instructions are executed on a computer, the computer causes the computer to perform the method as described in any one of claims 1 to 8.

Images