A multi-warehouse integration construction method, device, equipment and medium

Abstract

The present application relates to the technical field of computers, and in particular relates to a multi-repository integration construction method and device, equipment and medium. The present application uses descriptors as configuration carriers to centrally manage integrated information originally scattered in repositories, scripts, CI configurations and documents, thereby reducing maintenance costs and hidden faults. Secondly, based on descriptors, version constraints and cross-repository constraints are solved to generate a lock file binding an analyzer and a descriptor version, convert complex rules such as multi-variant and cross-repository compatibility into a determined and reproducible commit record number, and solve the problem of incomplete multi-repository version combination expression. Then, the lock file is used to synchronize the code to ensure that the repository combination is completely consistent in any environment, thereby avoiding version drift and construction irreproducibility from the root. Finally, combined with the construction environment and standardized pipeline, the construction behavior is further ensured to be consistent, and multi-repository integration is stable, traceable and efficient.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to a method, apparatus, equipment, and medium for building a multi-warehouse integration system. Background Technology

[0002] In enterprise-level software delivery scenarios involving multiple teams, vendors, and variants, current mainstream multi-repository integration solutions mainly include sub-modules, Manifest-Repo, and script-based integration. These solutions typically only record repository addresses and corresponding version revision numbers in a static list format, lacking a unified description and calculable verification mechanism for product variants, configuration matrices, cross-repository dependency constraints, component compatibility boundaries, and optional components. At the same time, key integration information such as repository manifests, build parameters, toolchain paths, patching strategies, and permission configurations are scattered across multiple carriers such as the main repository, manifest files, CI configurations, build scripts, and documents, failing to form a single, authoritative configuration source and a unified verification mechanism. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a multi-warehouse integration construction method, apparatus, equipment and medium to solve the problems of incomplete expression of multi-warehouse version combination, easy version drift and unreproducible construction, and multi-source truth leading to scattered integration logic, high maintenance costs and easy introduction of hidden faults.

[0004] In a first aspect, embodiments of the present invention provide a method for building a multi-repository integration system, the method comprising: Obtain the key task information for this multi-repository integration and build task, and use the key task information to obtain the descriptor; The corresponding repository is accessed using the key information of the task, and the code of multiple repositories is synchronized based on the commit record number in the lock file associated with the task built based on this multi-repository integration. The lock file is obtained based on the descriptor and repository version constraint solution results. For the multi-repository code, dependency analysis and impact scope calculation are performed in the incremental build scenario to generate a build plan for the affected parts of the build and testing, and a build environment is generated using the descriptor and the multi-repository code; Based on the build plan and the build environment, pipeline orchestration is performed to complete the integrated build of multiple repositories.

[0005] Furthermore, before accessing the corresponding repository using the aforementioned task-critical information, the method further includes: Use the descriptor to read the version constraints and cross-repository constraints of each repository; The parser is invoked to parse the version constraints and cross-repository constraints to obtain a set of repository IDs. The set of IDs includes commit record IDs corresponding to multiple repositories, and each commit record ID corresponds to a code version of a repository. A lock file is generated using the set of numbers, and the lock file is bound to the version of the descriptor and the version of the parser.

[0006] Furthermore, the call parser parses the version constraints and cross-repository constraints to obtain a set of repository IDs, including: The parser is invoked to perform logical conflict detection on the version constraints and cross-warehouse constraints. If a logical conflict exists, constraint conflict diagnosis information is output and the current build task is terminated; or, if no conflict exists, the version constraint and the cross-repository constraint are converted into the commit record number corresponding to each repository, and based on the build number set.

[0007] Furthermore, accessing the corresponding repository using the task-critical information includes: Based on the key information of the task, the corresponding warehouse access domain is determined, and a temporary access credential for the warehouse access domain is generated. The warehouse access domain is used to represent the set of warehouses that are allowed to be accessed in this multi-warehouse integration and construction task, and different warehouse access domains correspond to different sets of warehouses. Use the temporary access credentials to access warehouses in the corresponding warehouse set.

[0008] Furthermore, the step of performing dependency analysis and impact scope calculation for the multi-repository code in an incremental build scenario, and generating a build plan for the affected parts of the build and testing, includes: Obtain the scope of code changes in the incremental build scenario; Based on the dependency configuration corresponding to the descriptor and the commit record number in the lock file, a global dependency graph across repositories is generated. Based on the scope of the code changes and the cross-repository global dependency graph, the impact propagation calculation is performed to obtain the build targets and test set sets affected by the code changes; The build target and the test set are sequentially arranged to obtain a build plan for the parts of the build and test that are affected.

[0009] Furthermore, the calculation of the impact range propagation based on the scope of the code change and the cross-repository global dependency graph is used to obtain the build targets and test set sets affected by the code change, including: Based on the scope of the code change and the cross-repository global dependency graph, locate the initial dependency node corresponding to the changed code; Based on the initial dependency node, a dependency propagation traversal is performed in the cross-repository global dependency graph to obtain all associated dependency nodes; Based on matching all the associated dependency nodes with the corresponding build targets and test sets, a preliminary set of affected objects is obtained; The initial set of affected objects is deduplicated and validated to obtain the final set of build targets and test sets affected by the code changes.

[0010] Furthermore, the step of utilizing the descriptor and the multi-repository code generation build environment includes: Extract the path mapping configuration information from the descriptor; Based on the path mapping configuration in the descriptor, create the directory structure required for this multi-repository integration build task, and configure the build entry script and environment boot script in the directory structure; Based on the directory structure, the build entry script, and the environment bootstrapping script, an environment fingerprint is generated to obtain the build environment.

[0011] Secondly, embodiments of the present invention provide a multi-warehouse integration construction device, the device comprising: The acquisition module is used to acquire key task information for this multi-repository integration and construction task, and to acquire descriptors using the key task information; The processing module is used to access the corresponding repository using the key information of the task, and to synchronize the code of multiple repositories based on the commit record number in the lock file associated with the task built in this multi-repository integration. The lock file is obtained based on the solution results of the descriptor and repository version constraints. The generation module is used to perform dependency analysis and impact scope calculation for the multi-repository code in an incremental build scenario, generate a build plan for the parts affected by building and testing, and generate a build environment using the descriptor and the multi-repository code; The execution module is used to perform pipeline orchestration based on the build plan and the build environment to complete the integrated build of multiple repositories.

[0012] Thirdly, embodiments of the present invention provide a computer device, including: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the method described in the first aspect or any corresponding embodiment thereof.

[0013] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that cause a computer to perform the method described in the first aspect or any of its corresponding embodiments.

[0014] This application uses descriptors as configuration carriers to centrally manage integration information that was originally scattered across repositories, scripts, CI configurations, and documentation, reducing maintenance costs and hidden faults. Secondly, it solves version constraints and cross-repository constraints based on descriptors, generating a lock file that binds the parser and descriptor versions. This transforms complex rules such as multiple variants and cross-repository compatibility into deterministic and reproducible commit record numbers, solving the problem of incomplete expression of multi-repository version combinations. Then, it uses the lock file to synchronize code, ensuring a completely consistent repository combination in any environment, fundamentally preventing version drift and unreproducible builds. Finally, combined with the build environment and standardized pipeline, it further ensures consistent build behavior, achieving stable, traceable, and highly efficient multi-repository integration. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating a multi-warehouse integration construction method according to some embodiments of the present invention; Figure 2 This is a flowchart illustrating another multi-warehouse integration construction method according to some embodiments of the present invention; Figure 3 This is a structural block diagram of a multi-warehouse integrated construction device according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] According to embodiments of the present invention, a method, apparatus, device and medium for building multi-warehouse integration are provided. It should be noted that the steps shown in the flowcharts in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be executed in a different order than that shown here.

[0019] This embodiment provides a method for building a multi-repository integration. Figure 1 This is a flowchart of a multi-warehouse integration construction method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps: Step S101: Obtain the key task information for this multi-repository integration and build task, and use the key task information to obtain the descriptor.

[0020] In this embodiment, a client or CI trigger acts as the main entity, responsible for receiving and parsing externally initiated multi-repository integration build requests to form a unified and standardized task context. Users can select the target project, build variant, entry module, and full or incremental mode via command line or graphical interface, or the continuous integration system can automatically trigger the request, carrying key parameters such as branch, change number, and target platform. Information such as project identifier, variant type, execution entry point, build mode, and trigger source is extracted and integrated into structured key task information.

[0021] After generating the key task information, the executing entity will read the corresponding descriptor from the descriptor repository or platform template based on this information. This descriptor contains core configurations such as a repository list, path mapping, variant definitions, version constraints, and build entry point, serving as the core basis for the entire multi-repository integrated build process. After reading, the existence and accessibility of the descriptor are initially confirmed to ensure that subsequent processes can continue execution based on a valid and effective configuration file, avoiding process anomalies due to missing or incorrect configurations.

[0022] Step S102: Access the corresponding repository using the key information of the task, and synchronize the code of multiple repositories based on the commit record number in the lock file associated with the task built in this multi-repository integration. The lock file is obtained based on the solution results of descriptor and repository version constraints.

[0023] In this embodiment of the application, accessing the corresponding repository using task-critical information includes: Step A1: Determine the corresponding repository access domain based on the key task information and generate temporary access credentials for the repository access domain. The repository access domain is used to represent the set of repositories that are allowed to be accessed in this multi-repository integration and construction task. Different repository access domains correspond to different sets of repositories.

[0024] Specifically, during execution, parameters such as ProjectId, TriggerSource, and Mode in the key task information are parsed. Combined with preset access domain rules, the set of repositories that this multi-repository integration and build task is allowed to access is determined, i.e., the repository access domain. Different task parameters correspond to different access domains. For example, internal development tasks correspond to the internal repository access domain, which includes all code repositories within the enterprise; third-party collaboration tasks correspond to the restricted access domain, which only includes the specified partner repositories and read-only image repositories. This ensures that the access scope accurately matches the task requirements and avoids the leakage of access permissions for irrelevant repositories.

[0025] After determining the repository access domain, the system generates temporary access credentials following the principle of least privilege. The credential's permissions are precisely limited to the minimum scope required for this task: only allowing access to repositories within the corresponding access domain, granting only code read permissions, and setting a specific validity period—valid only during the execution of this build task and automatically expiring after the task ends. Simultaneously, the authorization digest (excluding sensitive plaintext) is written to the audit log, detailing the access domain scope, credential validity time, and permission level, facilitating subsequent audit traceability. This approach effectively reduces the scattered use of long-term credentials, lowers the risk of credential leakage, and avoids security vulnerabilities caused by excessively open permissions.

[0026] Step A2: Access the warehouses in the corresponding warehouse set using the temporary access credentials.

[0027] Specifically, during execution, the system first reads the generated temporary access credential and performs a secondary verification of its validity and permission scope to confirm that the credential has not expired and that the access permissions match the current repository access domain, thus avoiding access failures due to invalid credentials or mismatched permissions. After successful verification, the system establishes connections with the corresponding repositories one by one according to the repository list in the repository access domain. During the connection process, the temporary access credential is carried out for identity authentication to ensure that every repository access is legally authorized.

[0028] For different types of repositories (internal repositories, supplier repositories, and read-only mirror repositories), the system employs corresponding access protocols and authentication methods to ensure compatibility and stability. For internal enterprise repositories, access is achieved via an internal LAN link to improve access speed; for external supplier repositories, encrypted links ensure data transmission security; for read-only mirror repositories, only read operations are triggered, adhering to the permission scope defined by credentials. During access, the connection status is monitored in real time. If authentication failures, link interruptions, or other anomalies occur, a limited number of retries are performed according to a preset retry policy, and structured logs are output to record anomaly details for easy problem localization. Simultaneously, information such as the time, repository address, and operation type of each repository access is recorded.

[0029] In this embodiment of the application, the lock file contains the results of all constraint solving, including key information such as RepoUrl, Path, CommitId, and data digest for each repository. It is also bound to the descriptor version and parser version, ensuring that anyone or any environment using the lock file can synchronize to a completely consistent set of repository code, thus guaranteeing the reproducibility of the build process from the root.

[0030] During execution, the lock file is first read to parse out the commit record number, repository address, and location path for each repository. Then, the internal mirror nodes or cache nodes are queried first to determine if the code object for the corresponding repository and commit record number already exists. If the object exists in the cache, it is retrieved directly from the cache, eliminating the need to fetch from the external source code management system (SCM), significantly improving synchronization efficiency. If the object is missing from the cache, the code for the corresponding commit record number is fetched from the external SCM through the established repository connection. The fetched code object is then stored and deduplicated to avoid storing the same version of code repeatedly, saving storage space. During synchronization, shallow cloning and block fetching optimizations are used based on preset process parameters to reduce network transmission volume. A retry strategy is employed to handle fetching failures caused by network jitter, ensuring that each repository can accurately detect the commit record number specified in the lock file.

[0031] It should be noted that, in response to the problems of complex permission and credential configuration and unstable cross-organization integration, existing technologies require the configuration of multiple credentials for multi-permission domain access, and the credential logic is scattered in scripts and pipeline configurations, resulting in poor reusability, high risk of credential exposure, and easy failures in migration and permission changes.

[0032] This application's embodiments dynamically identify warehouse access domains based on key task information and automatically generate temporary access credentials according to the principle of least privilege. This enables centralized, standardized, and automated management of permissions and credentials, and can be adapted to various permission domains such as internal warehouses, supplier warehouses, and open-source mirrors. It reduces the use of long-term high-privilege credentials, narrows the security risk surface, improves configuration reusability and environment portability, and significantly reduces the probability of access failures and synchronization anomalies during permission changes, warehouse migrations, and cross-organizational collaborations, thereby improving the security and stability of multi-warehouse integration.

[0033] Step S103: For multi-repository code, perform dependency analysis and impact scope calculation in the incremental build scenario, generate a build plan for the parts affected by the build and testing, and generate a build environment using descriptors and multi-repository code.

[0034] In this embodiment of the application, for multi-repository code, dependency analysis and impact scope calculation are performed in an incremental build scenario to generate a build plan for the affected parts of the build and testing, including: Step B1: Obtain the scope of code changes in the incremental build scenario.

[0035] After the incremental build mode is initiated, the system first extracts the baseline commit record number from the standardized task context and locked files, using it as the baseline version for code comparison. Simultaneously, it reads the current code status of multiple synchronized repositories and uses version comparison tools to examine code changes repository by repository and file by file. It accurately identifies newly added, modified, and deleted code files and modules, filtering out irrelevant configurations, comments, and other changes that do not affect the compilation logic. This results in a structured code change list that can be used for dependency analysis, clearly marking the changed repository paths, file names, and code modules. This ensures that the scope of changes is clear, accurate, and free of redundant information, providing reliable input for subsequent dependency propagation and impact analysis.

[0036] Furthermore, by combining the change number, commit history, and branch difference information carried in the task context, the scope of code changes can be further refined and verified, excluding changes unrelated to business logic, and ensuring that only valid changes that affect the build results, functional logic, and test cases are retained. Simultaneously, the final determined scope of code changes is stored in association with the task information, forming a traceable change record. This avoids misjudging the scope of impact due to invalid changes, ensuring that subsequent analysis focuses on truly effective code modifications, and improving the accuracy and execution efficiency of incremental builds.

[0037] Step B2: Generate a global dependency graph across repositories based on the dependency configuration corresponding to the descriptor and the commit record number in the lock file.

[0038] First, the system reads the verified descriptors and extracts complete dependency configuration information, including inter-repository dependencies, module reference rules, component hierarchy, and interface relationships. This clarifies the compilation, runtime, and test dependencies between repositories and modules. Simultaneously, it loads all repository commit record numbers bound to the lock file to ensure dependency analysis is based on a definite and reproducible code version. The system parses the relationships between code units layer by layer, using repositories as units, modules as nodes, and dependencies as edges, forming a basic dependency structure covering all participating repositories and ensuring complete matching of dependencies with the current code version. Based on this basic dependency structure, the system integrates and normalizes cross-repository dependencies, unifying the dependency representation format across different repositories, eliminating parsing anomalies caused by path and naming differences, and verifying the validity and completeness of dependencies, eliminating invalid, circular, and conflicting dependency links. Finally, it generates a structured, traversable, and computable cross-repository global dependency graph. This dependency graph fully records the propagation relationships between all build targets, code modules, and test cases in a multi-repository environment, supporting rapid location and hierarchical propagation of change impacts.

[0039] Step B3: Perform impact propagation calculations based on the scope of code changes and the cross-repository global dependency graph to obtain the build targets and test set sets affected by the code changes.

[0040] Specifically, based on the scope of code changes and the cross-repository global dependency graph, an impact propagation calculation is performed to obtain the set of build targets and test sets affected by the code changes. This includes: locating the initial dependency node corresponding to the changed code based on the scope of code changes and the cross-repository global dependency graph; performing dependency propagation traversal in the cross-repository global dependency graph based on the initial dependency node to obtain all related dependency nodes; matching the corresponding build targets and test sets based on all related dependency nodes to obtain a preliminary set of affected objects; and performing deduplication and validity verification on the preliminary set of affected objects to obtain the final set of build targets and test sets affected by the code changes.

[0041] First, the scope of code changes and the cross-repository global dependency graph are loaded and matched together. The core is to accurately locate the initial dependency node corresponding to the code change in the dependency graph. The scope of code changes includes information such as specific repository paths, file names, and code modules. These changes are parsed one by one, and according to the node mapping rules of the dependency graph, each changed code unit (such as function, class, module) is mapped to the corresponding node in the cross-repository global dependency graph. These nodes are the initial dependency nodes—the core nodes where code changes directly occur and serve as the starting point for the impact propagation.

[0042] During the localization process, the matching relationship between change items and dependency graph nodes is verified to eliminate matching deviations caused by path differences and inconsistent naming conventions, ensuring that every valid code change accurately corresponds to a unique node in the dependency graph. Simultaneously, the initial dependency node's attribute information, including its repository, module type, dependency level, and associated interfaces, is marked to distinguish core change nodes from secondary change nodes, providing a clear starting point for subsequent dependency propagation traversal. Furthermore, the localization process and results of the initial dependency node are recorded to form a matching log, facilitating subsequent traceability and problem investigation, ensuring the accuracy and completeness of the initial node localization, and laying the foundation for subsequent impact range propagation calculations.

[0043] After identifying the initial dependency nodes, the system, following preset dependency propagation rules, performs a comprehensive, multi-level dependency propagation traversal across the global dependency graph. The core objective is to fully capture all direct and indirect dependency nodes affected by the initial change. The propagation traversal process follows the "forward propagation" principle, starting from the initial change node and traversing its directly dependent parent nodes and indirectly dependent deep nodes layer by layer, covering all dependency types such as compile-time dependencies, runtime dependencies, and test dependencies, without omitting any node associated with the initial change.

[0044] During the traversal, the traversal path and associated node information are recorded in real time. Each traversed node is marked, distinguishing between directly related nodes (directly connected to the initial node) and indirectly related nodes (related to the initial node through intermediate nodes). The dependency relationship types and propagation paths between nodes are also recorded to facilitate subsequent tracing of impact chains. For cross-repository dependencies, the validity of dependency chains between repositories is rigorously verified to ensure uninterrupted and complete propagation of nodes across repositories. After traversal, the system integrates all marked associated nodes to form a complete set of associated dependency nodes. This set includes all nodes whose behavior may have changed due to initial code modifications and require re-verification, providing comprehensive node information for subsequent matching of target builds and test suites.

[0045] Next, the entire set of related dependency nodes is loaded. Combining the build target configuration in the descriptor, test suite association rules, and code version information determined in the lock file, association matching is performed. The core is to establish a correspondence between dependency nodes and specific build targets and test suites, forming a preliminary set of affected objects. First, the attributes of each related dependency node are parsed to determine its repository, module, and functional module. Then, according to preset matching rules, the node is associated with its corresponding build target—for example, the compilation target and packaging target corresponding to a changed module will both be matched as affected build targets. Simultaneously, based on the functional logic corresponding to the related dependency node, relevant test suites, including unit tests, integration tests, and regression tests, are matched to ensure that all test cases related to the changed node are included in the affected scope.

[0046] During the matching process, priority is given to matching the core build goals and key test sets corresponding to directly related nodes, and then gradually expanded to the secondary build goals and auxiliary test sets corresponding to indirectly related nodes to ensure the comprehensiveness of the matching. After the matching is completed, the system integrates all related build goals and test sets to form a preliminary set of affected objects. This set contains all objects that may be affected by code changes and need to be rebuilt and tested, providing a foundation for subsequent deduplication and verification.

[0047] Finally, for the initial set of affected objects, deduplication is performed first. The core of this process is to remove duplicate build targets and test sets from the set, avoiding duplicate inclusion issues caused by multiple paths during dependency propagation. Each build target and test set in the set undergoes unique identifier verification. Based on key information such as its repository, target name, and test case ID, duplicates are removed to ensure that each affected object is retained only once. This simplifies the subsequent build plan orchestration process and improves execution efficiency.

[0048] After deduplication, the system performs validity checks on the remaining affected objects, eliminating invalid, expired, and irrelevant objects. For example, it verifies whether the build target exists in the current code version, whether the test cases are still valid, and whether they fall within the execution scope of the current build task. Simultaneously, it verifies the correlation between the affected objects and the code changes, removing objects corresponding to irrelevant nodes caused by dependency propagation misjudgments. During the verification process, the validity of each object is confirmed one by one, taking into account task context, descriptor configuration, and locked file versions. Invalid objects are marked and deleted, ultimately resulting in a concise, accurate, and directly executable set of final build targets and test suites affected by the code changes. This provides a reliable basis for subsequent build planning orchestration.

[0049] For example, in a multi-repository integration build task, the code change scope is the "user authentication module" (corresponding code file AuthService.java) in repository A. In the cross-repository global dependency graph, the corresponding node for this module is Node-A. First, Node-A is located as the initial dependency node. Then, dependency propagation traversal is performed, and it is found that Node-A directly depends on the "encryption tool module" (Node-B) in repository B. Node-B in turn depends on the "security configuration module" (Node-C) in repository C. At the same time, Node-A indirectly depends on the "logging module" (Node-D) in repository D. Thus, all related dependency nodes {Node-A, Node-B, Node-C, Node-D} are obtained. Next, the build targets corresponding to these nodes are matched, namely the "user authentication module compilation target" of repository A, the "encryption tool compilation target" of repository B, the "security configuration compilation target" of repository C, and the "log module compilation target" of repository D, as well as the corresponding test sets (user authentication unit test set, encryption tool integration test set, security configuration regression test set, and log module test set), forming a preliminary set of affected objects. Finally, the set is deduplicated, duplicate test cases are removed, and all build targets and test sets are verified to be within the scope of this build and valid. The final affected set containing 4 build targets and 4 test sets is obtained and used for subsequent build plan orchestration.

[0050] Step B4 involves sequentially orchestrating the build targets and test sets to obtain a build plan for the parts of the build and test that are affected.

[0051] Based on the affected build targets and test suites, and in conjunction with the build rules, execution order, dependency priority, and parallel execution strategies in the descriptors, an ordered orchestration is performed. Following the principles of compilation first, tests next, and dependencies first, the compilation order of each build target and the execution order of each test suite are determined. Simultaneously, the degree of parallelism is set according to system resource configuration to optimize the execution flow and improve overall build efficiency. The orchestration process adheres to dependency constraints, ensuring that dependent targets are built first, basic modules are compiled first, and core tests are run first, avoiding build failures or test ineffectiveness due to incorrect ordering.

[0052] After orchestrating the execution order and parallel strategies, the system integrates build parameters, environment configurations, log output rules, and failure retry strategies to generate a structured, directly executable incremental build plan. This plan clearly identifies the path, version, dependencies, and execution method of each build target, as well as the scope, validation rules, and result judgment criteria for each test suite. The plan only includes affected build and test content and excludes irrelevant tasks, significantly reducing pipeline execution time.

[0053] It's worth noting that by using dependency configuration based on descriptors and the code version determined by locking files, a global dependency graph is generated across repositories. This enables unified explicit modeling of dependencies across multiple repositories, overcoming the shortcomings of traditional solutions that only focus on code retrieval and lack dependency analysis capabilities. By locating the initial dependency node through the scope of code changes and performing a full-link traversal in the dependency graph, all build targets and test sets affected by changes can be automatically and accurately calculated, replacing traditional manual judgment or coarse-grained full build methods. This solution can significantly shorten the CI pipeline execution cycle, reduce invalid compilations and tests, and avoid missed or incorrect tests due to insufficient experience. It also improves regression accuracy and delivery efficiency, providing reliable incremental build support for multi-variant, multi-repository collaborative development.

[0054] In this embodiment of the application, a build environment is generated using descriptors and multi-repository code, including: Step C1: Extract the path mapping configuration information from the descriptor.

[0055] After verifying the validity of the descriptor, the unique descriptor file corresponding to the current build task is located from the standardized task context. All configuration items within the descriptor are read according to preset parsing rules, with a focus on extracting path mapping configuration information used for multi-repository code organization. This configuration information includes key content such as the root directory of each repository in the local workspace, the location path of submodules, the directory for storing resource files, the build output directory, and the script execution directory. Simultaneously, additional constraints such as path conflict verification rules, directory permission configurations, and cross-repository path relationships are parsed to ensure that the extracted path information is complete, standardized, and directly usable for directory creation.

[0056] The extracted path mapping information is then formatted to unify path separators, conversion rules for absolute and relative paths, and directory naming conventions. Invalid characters and redundant configurations are removed, while the most core executable path data is retained. The processed path configuration is then bound to the task context to form a transferable and parsable structured path information, providing an accurate basis for subsequent directory structure creation.

[0057] Step C2: Based on the path mapping configuration in the descriptor, create the directory structure required for this multi-repository integrated build task, and configure the build entry script and environment boot script in the directory structure.

[0058] Based on path mapping configuration information, a multi-level directory structure is created layer by layer in the build workspace. Following the rules defined by the descriptors, code, configurations, resources, and output from multiple repositories are allocated to designated directory locations, ensuring that files from different repositories do not overlap, conflict, or get lost, forming a unified and well-organized multi-repository integrated directory layout. After the directory structure is created, the system generates a common build entry script in the specified root directory or entry directory according to the task type and build platform requirements. This script encapsulates common logic such as build commands, parameter passing, log output, and exception handling, achieving a unified build entry point throughout the entire process. Simultaneously, the system deploys an environment bootstrapping script to automatically detect toolchain versions, install dependent components, configure environment variables, and load build parameters.

[0059] Step C3 generates an environment fingerprint based on the directory structure, the build entry script, and the environment bootstrap script, thus obtaining the build environment.

[0060] After completing the standard directory creation and script configuration, all key characteristic information of the current build environment is collected, including directory structure layout, repository location path, build entry script version, environment bootstrap script content, operating system type, toolchain version number, core dependency version, environment variable configuration, build parameters, and other information. This information is integrated and encoded according to a fixed format to generate a unique environment fingerprint that identifies the current build environment. This fingerprint is tamper-proof, unique, and verifiable, enabling rapid differentiation between different build environments and ensuring complete consistency between environments with the same fingerprint. Finally, the system associates and stores the environment fingerprint with task information, locked files, and descriptor versions, forming a complete and traceable build environment. This completely eliminates environmental differences between different devices, users, and nodes, ensuring that incremental build plans can be executed in a stable, consistent, and reproducible environment.

[0061] It's important to note that existing technologies address the issues of tightly coupled workspace build environments, slow developer onboarding, and poor reusability. These technologies rely on fixed directory structures, toolchain locations, and environment variables, leading to significant differences between environments, cumbersome configurations, and issues such as being able to pull code but not compile it, or inconsistent compilation results. This solution automatically creates a standardized directory structure based on path mappings in the descriptor, deploys a unified build entry script and environment bootstrapping script, eliminating strong dependence on fixed environments. Furthermore, by generating unique environment fingerprints to identify the build environment, it ensures that different operating systems and devices can be quickly initialized to a consistent state. This significantly reduces the environment configuration costs for new members and new machines, improves consistency between local development and CI builds, and enables replicable, reproducible, and rapidly reusable build environments, effectively improving overall delivery efficiency.

[0062] Step S104: Perform pipeline orchestration based on the build plan and build environment to complete the multi-repository integrated build.

[0063] In this embodiment, based on an incremental build plan and a standardized build environment, the pipeline orchestration module executes ordered scheduling according to the complete stages of compilation, testing, packaging, and release. During the compilation stage, the system selects affected build targets according to the build plan, supporting parallel compilation and caching acceleration to improve compilation speed. During the testing stage, only test cases within the affected scope are run, reducing the execution time of invalid tests. During the packaging stage, the build artifacts are organized, compressed, and signed. During the release stage, legitimate artifacts are pushed to the enterprise artifact repository. Each stage outputs clear status codes and structured logs, facilitating rapid location of failures and causes of anomalies.

[0064] After the entire pipeline is completed, the system archives all build artifacts to the designated artifact repository and generates a complete traceability record, including key information such as repository combination, build parameters, environment information, artifact hash, and storage address. It also outputs easily readable and auditable summary information. If quality issues arise, the traceability record allows for rapid reproduction of the original repository version, environment configuration, and build process, pinpointing the root cause of the problem. The entire process achieves end-to-end automation from code synchronization, environment building, incremental execution to artifact archiving.

[0065] In this embodiment of the application, before accessing the corresponding repository using task-critical information, such as Figure 2 As shown, the method also includes: Step S201: Use descriptors to read the version constraints and cross-repository constraints of each repository.

[0066] Based on verified descriptors, the system reads all constraint information related to repository versions from the configuration file according to preset parsing rules, extracting both individual repository version constraints and cross-repository constraints that link repositories. Version constraints include conditions such as branch name, tag rules, commit scope, and version number range to limit the code version of a single repository. Cross-repository constraints include rules such as interface compatibility requirements, component mutual exclusion rules, dependency version consistency, and collaborative update restrictions to ensure version matching between multiple repositories.

[0067] The system performs structured parsing and categorized storage of the read constraint information, distinguishing between mandatory and optional constraints, and marking constraint priorities and effective scopes. This ensures that all constraints are complete, clear, and computable, providing standardized and consistent input data for subsequent version solving by the parser and preventing parsing failures due to missing or incorrect constraint information. During the reading and parsing of constraint information, basic legality checks are performed simultaneously to verify the syntactic correctness of version constraints, the logical rationality of cross-warehouse constraints, and to detect obvious format errors, invalid paths, missing warehouse identifiers, etc., initially marking abnormal constraints. Simultaneously, the system associates the parsed constraint information with the task context, recording the constraint source, descriptor version, warehouse list, and other related information to form a traceable constraint data structure. Through standardized reading and preprocessing, the system ensures that version constraints and cross-warehouse constraints can be correctly identified and calculated by the parser, providing stable and reliable data support for subsequent version conflict detection and commit record number solving.

[0068] Step S202: Call the parser to parse the version constraints and cross-repository constraints to obtain a set of numbers for all repositories. The set of numbers includes the commit record numbers corresponding to multiple repositories, and each commit record number corresponds to the code version of a repository.

[0069] In this embodiment of the application, the parser is invoked to parse the version constraints and cross-repository constraints to obtain a set of numbers for all repositories. This includes: invoking the parser to perform logical conflict detection on the version constraints and cross-repository constraints; if a logical conflict exists, constraint conflict diagnosis information is output and the current build task is terminated; or, if no conflict exists, the version constraints and cross-repository constraints are converted into commit record numbers corresponding to each repository, and based on the set of build numbers.

[0070] Specifically, the parser is invoked to perform logical conflict detection on version constraints and cross-repository constraints. The parser performs a full-dimensional compatibility check based on all repository version restrictions and cross-repository association rules extracted from the descriptor. It checks key aspects such as whether the version constraints of a single repository are valid, whether the dependencies between different repositories match, whether there is an intersection between version ranges, and whether the compatibility rules can be satisfied. At the same time, it traverses all constraint combinations according to the preset solution strategy and priority rules to determine whether there is a repository version combination that can satisfy all constraints at the same time, ensuring that all constraints are logically achievable, without contradictions, and without mutual exclusion.

[0071] During conflict detection, the resolver accurately locates the location and cause of the conflict, records information such as the name of the conflict repository, the content of the conflict constraints, the incompatible version range, and the cross-repository rules that cannot be met, and forms a structured conflict detection result. This provides a clear basis for subsequent anomaly handling. The entire detection process follows calculable and verifiable execution logic to ensure that the detection results are objective and accurate, and to avoid version anomalies in the subsequent build process due to missed constraints or judgment errors.

[0072] If the detection finds an irreconcilable logical conflict between version constraints and cross-repository constraints, it will automatically output diagnostic information containing conflict details, clearly showing the conflict item, the cause of the conflict and optional repair suggestions, and immediately terminate the current multi-repository integration build task to prevent the process from continuing under incorrect constraints and avoid producing unreproducible or unstable build results.

[0073] If the test confirms that all constraints are conflict-free and fully satisfyable, the parser will transform the dynamic and variable constraints into unique commit record numbers for each repository. Following the version-optimal matching rules, it will select a compliant code version for each repository and integrate all commit record numbers into a set of numbers, providing deterministic version data for subsequent lock file generation. After completing the version conversion, the parser will verify the set of numbers to ensure that each repository corresponds to a unique and valid commit record number, and that all numbers meet the original constraints and cross-repository compatibility requirements, guaranteeing the reproducibility and consistency of version combinations. The verified set of numbers will be passed to the next stage as core data, achieving a standardized conversion from flexible constraints to fixed versions, ensuring accurate matching and stable use of code versions across multiple repositories.

[0074] Step S203: Generate a lock file using the number set, and bind the lock file to the descriptor version and the parser version.

[0075] A lock file is generated using a set of commit IDs, and this lock file is bound to the descriptor version and the parser version. Based on the commit record ID set output by the parser, the system generates a lock file according to a standardized file structure. The file fully records key information such as the address of each repository, its local path, the corresponding commit record ID, and data checksums, ensuring that the lock file uniquely identifies a set of reproducible multi-repository code. During the generation process, the lock file is formatted to ensure a standardized file structure, complete fields, and ease of parsing. File verification information is also added to prevent file tampering, ensuring that the build process can execute code synchronization based on a trusted lock file.

[0076] After the lock file is generated, the system strongly binds it to the currently used descriptor version and the combined parser version, writing version association information into the lock file header. This ensures that the same descriptor and parser versions are used when reproducing a build in any environment or by any user, guaranteeing complete consistency between parsing behavior and version resolution results. The system also stores the lock file in association with the build task context, recording audit information such as generation time, task identifier, and operator, forming a traceable, verifiable, and shareable version locking result. This provides a unified and reliable version basis for multi-repository code synchronization, incremental build analysis, and standardized environment generation.

[0077] For example, a build task includes repositories A, B, and C. The version constraint for repository A in the descriptor is matching the v2.x tag, and the constraint for repository B is matching the latest commit on the main branch. Cross-repository constraints require repository B to be version compatible with repository A. After the parser detects no conflicts, it assigns commit record numbers 1a2b3c to repository A corresponding to the tag, 4d5e6f to repository B (which is compatible), and 7g8h9i to repository C (which is a matching commit record), forming a set of numbers. The system generates a lock file based on this set and binds the descriptor version v1.0 to the parser version v3.1, ensuring that all subsequent processes use this fixed version combination.

[0078] It should be noted that existing technologies, due to multiple sources of information, suffer from fragmented integration logic and high maintenance costs. These technologies disperse information such as repository lists, build parameters, and toolchain paths across multiple carriers, including the main repository, configuration files, scripts, and documents, lacking a unified authoritative configuration source. This leads to issues such as asynchronous updates, numerous hidden faults, and high maintenance difficulty. This application's embodiment uses a descriptor as the sole authoritative configuration carrier, managing all integration-related information uniformly. This ensures a unique configuration entry point, standardized structure, parsability, and verifiability. Any change only needs to be modified in one place for global effect, avoiding problems such as incorrect branch pulls, missed repository pulls, and inconsistent parameters caused by asynchronous configuration sources. This results in highly convergent and traceable integration logic, significantly reducing maintenance costs in multi-team, multi-supplier collaboration scenarios and improving the stability of the overall delivery process.

[0079] This embodiment also provides a multi-warehouse integration construction apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0080] This embodiment provides a multi-warehouse integrated construction device, such as... Figure 3 As shown, it includes: The acquisition module 301 is used to acquire the key task information of this multi-repository integration and build task, and to acquire the descriptor using the key task information; The processing module 302 is used to access the corresponding repository using the key information of the task, and to synchronize the code of multiple repositories based on the commit record number in the lock file associated with the task built in this multi-repository integration. The lock file is obtained based on the solution results of descriptor and repository version constraints. The generation module 303 is used to perform dependency analysis and impact scope calculation for multi-repository code in incremental build scenarios, generate build plans for the parts affected by building and testing, and generate build environments using descriptors and multi-repository code; Execution module 304 is used to perform pipeline orchestration based on the build plan and build environment to complete multi-repository integrated build.

[0081] In this embodiment of the application, the apparatus further includes: a construction module, configured to read the version constraints and cross-repository constraints of each repository using a descriptor; call a parser to parse the version constraints and cross-repository constraints to obtain a set of numbers for all repositories, wherein the set of numbers includes commit record numbers corresponding to multiple repositories, and each commit record number corresponds to a code version of a repository; generate a lock file using the set of numbers, and bind the lock file to the version of the descriptor and the version of the parser.

[0082] In this embodiment of the application, the construction module is used to call the parser to perform logical conflict detection on version constraints and cross-repository constraints; if a logical conflict exists, constraint conflict diagnosis information is output and the current construction task is terminated; or, if no conflict exists, the version constraints and cross-repository constraints are converted into the commit record numbers corresponding to each repository, and based on the set of construction numbers.

[0083] In this embodiment of the application, the acquisition module 301 is specifically used to determine the corresponding warehouse access domain based on the key information of the task, and generate a temporary access credential for the warehouse access domain. The warehouse access domain is used to represent the set of warehouses that are allowed to be accessed in this multi-warehouse integration and construction task. Different warehouse access domains correspond to different sets of warehouses. The temporary access credential is used to access the warehouses in the corresponding set of warehouses.

[0084] In this embodiment, the generation module 303 is specifically used to obtain the scope of code changes in the incremental build scenario; generate a cross-repository global dependency graph based on the dependency configuration corresponding to the descriptor and the commit record number in the lock file; perform impact scope propagation calculation based on the scope of code changes and the cross-repository global dependency graph to obtain the build targets and test set sets affected by the code changes; and perform sequential arrangement of the build targets and test set sets to obtain the build plan for the parts of the build and test affected by the code changes.

[0085] In this embodiment of the application, the generation module 303 is specifically used to locate the initial dependency node corresponding to the changed code based on the scope of code change and the cross-repository global dependency graph; perform dependency propagation traversal in the cross-repository global dependency graph based on the initial dependency node to obtain all related dependency nodes; match the corresponding build target and test set based on all related dependency nodes to obtain a preliminary set of affected objects; and perform deduplication and validity verification on the preliminary set of affected objects to obtain the final set of build targets and test sets affected by the code change.

[0086] In this embodiment of the application, the generation module 303 is specifically used to extract the path mapping configuration information in the descriptor; create the directory structure required for this multi-repository integration build task according to the path mapping configuration in the descriptor, and configure the build entry script and environment boot script in the directory structure; generate the environment fingerprint based on the directory structure, the build entry script and the environment boot script to obtain the build environment.

[0087] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 4As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system).

[0088] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0089] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0090] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device as shown by a landing page for an app. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0091] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0092] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0093] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0094] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for building a multi-warehouse integration system, characterized in that, The method includes: Obtain the key task information for this multi-repository integration and build task, and use the key task information to obtain the descriptor; The corresponding repository is accessed using the key information of the task, and the code of multiple repositories is synchronized based on the commit record number in the lock file associated with the task built based on this multi-repository integration. The lock file is obtained based on the descriptor and repository version constraint solution results. For the multi-repository code, dependency analysis and impact scope calculation are performed in the incremental build scenario to generate a build plan for the affected parts of the build and testing, and a build environment is generated using the descriptor and the multi-repository code; Based on the build plan and the build environment, pipeline orchestration is performed to complete the integrated build of multiple repositories.

2. The method according to claim 1, characterized in that, Before accessing the corresponding repository using the task-critical information, the method further includes: Use the descriptor to read the version constraints and cross-repository constraints of each repository; The parser is invoked to parse the version constraints and cross-repository constraints to obtain a set of repository IDs. The set of IDs includes commit record IDs corresponding to multiple repositories, and each commit record ID corresponds to a code version of a repository. A lock file is generated using the set of numbers, and the lock file is bound to the version of the descriptor and the version of the parser.

3. The method according to claim 2, characterized in that, The call parser parses the version constraints and cross-repository constraints to obtain a set of repository IDs, including: The parser is invoked to perform logical conflict detection on the version constraints and cross-warehouse constraints. If a logical conflict exists, constraint conflict diagnosis information is output and the current build task is terminated; or, if no conflict exists, the version constraint and the cross-repository constraint are converted into the commit record number corresponding to each repository, and based on the build number set.

4. The method according to claim 1, characterized in that, Accessing the corresponding repository using the key task information includes: Based on the key information of the task, the corresponding warehouse access domain is determined, and a temporary access credential for the warehouse access domain is generated. The warehouse access domain is used to represent the set of warehouses that are allowed to be accessed in this multi-warehouse integration and construction task, and different warehouse access domains correspond to different sets of warehouses. Use the temporary access credentials to access warehouses in the corresponding warehouse set.

5. The method according to claim 1, characterized in that, The process of performing dependency analysis and impact scope calculation for the multi-repository code in an incremental build scenario, and generating a build plan for the affected parts of the build and testing, includes: Obtain the scope of code changes in the incremental build scenario; Based on the dependency configuration corresponding to the descriptor and the commit record number in the lock file, a global dependency graph across repositories is generated. Based on the scope of the code changes and the cross-repository global dependency graph, the impact propagation calculation is performed to obtain the build targets and test set sets affected by the code changes; The build target and the test set are sequentially arranged to obtain a build plan for the parts of the build and test that are affected.

6. The method according to claim 5, characterized in that, The calculation of the impact range is performed based on the scope of the code change and the cross-repository global dependency graph to obtain the build targets and test set sets affected by the code change, including: Based on the scope of the code change and the cross-repository global dependency graph, locate the initial dependency node corresponding to the changed code; Based on the initial dependency node, a dependency propagation traversal is performed in the cross-repository global dependency graph to obtain all associated dependency nodes; Based on matching all the associated dependency nodes with the corresponding build targets and test sets, a preliminary set of affected objects is obtained; The initial set of affected objects is deduplicated and validated to obtain the final set of build targets and test sets affected by the code changes.

7. The method according to claim 1, characterized in that, The process of generating a build environment using the descriptor and the multi-repository code includes: Extract the path mapping configuration information from the descriptor; Based on the path mapping configuration in the descriptor, create the directory structure required for this multi-repository integration build task, and configure the build entry script and environment boot script in the directory structure; Based on the directory structure, the build entry script, and the environment bootstrapping script, an environment fingerprint is generated to obtain the build environment.

8. A multi-warehouse integrated construction device, characterized in that, The device includes: The acquisition module is used to acquire key task information for this multi-repository integration and construction task, and to acquire descriptors using the key task information; The processing module is used to access the corresponding repository using the key information of the task, and to synchronize the code of multiple repositories based on the commit record number in the lock file associated with the task built in this multi-repository integration. The lock file is obtained based on the solution results of the descriptor and repository version constraints. The generation module is used to perform dependency analysis and impact scope calculation for the multi-repository code in an incremental build scenario, generate a build plan for the parts affected by building and testing, and generate a build environment using the descriptor and the multi-repository code; The execution module is used to perform pipeline orchestration based on the build plan and the build environment to complete the integrated build of multiple repositories.

9. An electronic device, characterized in that, include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method of any one of claims 1 to 7.

Images