Automatic system and method for customizing, distributing and deploying embedded system software

Abstract

The invention discloses an automatic system for customizing and distributing and deploying system software of an embedded system. The automatic system comprises a configuration management module, a configuration management module and a configuration management module, wherein the configuration management module is used for storing a declarative configuration file in a Git version control system; the automatic execution module is realized by a CI / CD server, the CI / CD server is closely integrated with a Git warehouse of the configuration management module through a Webhook mechanism, and a CI / CD assembly line can be automatically triggered once the change of a configuration file is detected to be pushed to a specified branch; the product management module is used for archiving, storing and distributing the constructed product after the automatic execution module successfully completes the construction; and the end-side deployment module is used for deploying the system software on development equipment according to requirements for rapid deployment, verification and rollback use.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of embedded systems, automated operation and maintenance, software distribution, continuous integration and continuous delivery, and in particular to an automated system and method for customizing and distributing and deploying system software for embedded systems BACKGROUND

[0002] With the rapid development of technologies such as the Internet of Things, edge computing, and artificial intelligence, embedded systems have penetrated into various industries such as automobiles, medical treatment, industrial control, and consumer electronics. The market demand for more powerful, more intelligent, and more interconnected devices has led to an exponential increase in the complexity of embedded system software. This increase in complexity has brought unprecedented challenges to traditional embedded software development processes.

[0003] Unlike general-purpose enterprise software development, embedded development faces a series of unique constraints and difficulties. First, there is a close coupling between hardware and software, and software must run on specific, resource-constrained hardware platforms that often have strict limitations on processing power, memory, and storage space. Second, many embedded applications, such as automotive safety systems or industrial robots, need to interact and control complex peripheral hardware, requiring the integration of corresponding firmware and drivers into the system to support the hardware. In addition, due to the limited performance of embedded hardware, the development process often requires the use of complex cross-compilation toolchains to compile code that can run on target embedded devices on more powerful and feature-rich host machines (such as PCs). These factors together make the setup, maintenance, and synchronization of embedded development environments an extremely complex and time-consuming task.

[0004] Under this complex development background, the management of software dependencies and system configurations has become a key bottleneck that restricts development efficiency and project quality. When the development team wants to add new functions to the system, they often need to introduce new libraries, drivers, or middleware for the new functions. Traditional workflows rely heavily on manual operations, and developers need to manually download source code, determine appropriate dependency versions through repeated modifications and testing, modify complex build scripts (such as Makefile or BitBake recipes), resolve version conflicts between different dependencies, and ensure that the development environments of the entire team are consistent. This process is not only tedious but also prone to errors, easily falling into the "dependency hell" problem of different feature dependencies conflicting with different dependency versions. Specifically, existing solutions have the following many deficiencies in actual application: Fragile Development Environments: In collaborative projects, each developer needs to configure a complete cross-compilation environment on their local machine. Due to differences in operating systems, library file paths, toolchain versions, etc., software that compiles and runs successfully on one machine may fail to build or run abnormally on another. The effort to ensure successful operation across different machines wastes significant debugging time and reduces team collaboration efficiency. Furthermore, manually maintaining the environment is extremely fragile; any improper system updates or configuration modifications can corrupt the entire development environment and are difficult to repair.

[0005] Manual operations are inefficient and inaccurate: An embedded system software stack typically contains hundreds of open-source and proprietary software packages. Managing the direct and indirect dependencies between these packages is a daunting task. Adding, removing, or updating a dependent component often triggers a chain reaction, requiring developers to invest significant time and effort in analyzing and resolving dependency issues. Furthermore, in enterprise environments, besides dependency version compatibility issues, software license compliance, software package security vulnerabilities, and patching must also be considered in dependency management. Manual management heavily relies on expert experience, which is not only inefficient but also leads to a sharp increase in maintenance costs as projects grow. Finally, the manual packaging process struggles to accurately manage artifacts, potentially resulting in the omission of critical files that prevent the program from running correctly, or the incorrect inclusion of irrelevant or even sensitive files, leading to resource waste and security incidents.

[0006] The complexity of build artifact distribution and deployment: Even if the build system successfully generates system images and SDKs, efficiently and reliably distributing these build artifacts to team members and deploying them to their development environments remains a complex and critical step. The traditional method of developers manually downloading, unzipping, and configuring is not only inefficient but also risks introducing human error and disrupting environment consistency. Existing toolchains generally lack an integrated, automated mechanism to bridge the "last mile" from requirement updates to development environment updates.

[0007] Version management is challenging: During software iteration and updates, tracking and locating problems often requires repeatedly switching between different software versions for comparison before the source of the issue can be determined. Efficient and comprehensive version management can significantly improve the speed of switching between different versions, thereby increasing work efficiency and reducing software maintenance costs. Manual distribution methods are difficult to manage software versions efficiently and may result in incorrect version distributions, causing chaos in the workflow.

[0008] In summary, current embedded system development faces significant efficiency bottlenecks in system software configuration, management, and deployment, failing to meet the demands for rapid iteration, high quality, and high reliability. Therefore, there is an urgent need for an efficient, reliable, and secure automated system capable of automating the entire process of system software customization, distribution, and deployment. This would simplify developer operations, reduce the possibility of human error, and improve the efficiency of development and management. Summary of the Invention

[0009] The purpose of this invention is to provide an automated system and method for customizing, distributing, and deploying system software for embedded systems. This aims to solve the problems of cumbersome, error-prone, and time-consuming software dependency and system configuration management processes, as well as complex development environment versions and disorganized artifact storage in existing embedded development workflows. This invention fully automates complex build, distribution, and deployment tasks by introducing version-controlled declarative configuration and a CI / CD automated pipeline. It enables system software iteration based on updated requirements with zero developer intervention, thereby significantly improving developer efficiency, shortening development cycles, and ensuring the reliability and version consistency of the development environment during collaborative development.

[0010] The technical solution of this invention is an automated system for customizing, distributing, and deploying system software for embedded systems. This system enables developers to manage the system software stack in a simple and intuitive way, and utilizes existing, robust version management and collaboration systems to achieve synchronization among different developers. By applying modern DevOps practices to the field of embedded development, this invention implements and optimizes the entire workflow from system configuration to developer environment updates.

[0011] The technical solution of this invention logically consists of four core modules, which work together to form a closed-loop automated process: Configuration Management Module: This module is the starting point of the entire process and the sole user interface. Its core is a declarative configuration file stored in the Git version control system. The configuration file supports various common formats, such as YAML or JSON. Developers no longer need to directly manipulate complex build scripts; instead, they can modify this simple and easy-to-read configuration file to record their customizations of the system software, such as which libraries to add, which system services to enable, or which compilation parameters to set. Furthermore, to accommodate the needs of different developers at different levels, the configuration file can be automatically generated through a streamlined configuration assistant, or it can be directly written or modified by developers. Finally, the module uses Git for version control, ensuring that every system configuration change has a clear and traceable historical record, and that it can be easily rolled back to any historical version, providing a solid foundation for system stability and maintainability.

[0012] The automated execution module is a core component of the system, implemented by a CI / CD server (such as GitLab CI / CD or Jenkins). The CI / CD server is tightly integrated with the Git repository of the configuration management module via a webhook mechanism. Once a change to the configuration file is detected and pushed to a designated branch, the CI / CD pipeline is automatically triggered. The pipeline executes a series of predefined, automated tasks, including: preparing a clean build environment using Docker; parsing the contents of the configuration file; converting the configuration content into configuration items and instructions that the underlying build system (such as Yocto or Buildroot) can understand; parsing software dependencies and preparing source code and configuration; and finally, performing the time-consuming cross-compilation and packaging process.

[0013] Artifact Management Module: After the automated execution module successfully completes the build, its build artifacts require a reliable channel for archiving, storage, and distribution. Build artifacts typically include a complete root file system image, a software development kit (SDK) accompanying the build image, and deployment scripts for the image. The artifact management module is responsible for managing and providing build artifacts. All build artifacts are stored in a storage backend according to the module's rules. The storage backend can be a professional artifact repository (such as Artifactory or Nexus), or a simple HTTP file server or cloud storage service (such as AWS S3). Based on the hash value of the Git commit in the configuration file, all build artifacts are assigned a unique version identifier and uploaded to the storage backend. This centralized management approach ensures that all developers can obtain build artifacts from an authoritative, single source, while using the commit hash to strictly bind the configuration file to the system software, avoiding version confusion.

[0014] Edge Deployment Module: This module is responsible for deploying the system software on development devices as needed, enabling rapid deployment, verification, and rollback. This module is essentially a collection of designs and corresponding automated operation scripts. By retrieving from a pre-defined storage backend, the module obtains the latest version of the current system software, along with the corresponding root file system, toolchain, and deployment scripts. Then, following the instructions in the deployment scripts, the module can upgrade or modify the system software on the current development device.

[0015] These four modules constitute a complete, automated data flow closed loop. The specific steps of the system operation are as follows: Step 1 Configuration Update: Developers generate new configuration files based on development needs, either through configuration management tools or manually, and push them to the Git repository.

[0016] Step 2: Pipeline Invocation: After receiving the synchronized commit from the developer, the Git server invokes the Webhook to start the pipeline automation program. Commit synchronization ensures that the developer's local commit records and remote commit records are synchronized, avoiding conflicts and ensuring that changes are recorded. Step 3: Configuration Parsing: In the first stage of the pipeline, the system calls a program to parse the configuration, including the target platform architecture, system kernel configuration, root file system type and configuration method, dependencies and sources of dependent software packages, and the format and storage method of the artifacts. The results are saved as intermediate artifacts for debugging and other processing. Simultaneously, the system verifies the configuration file for correctness and the existence of unresolved conflicts. If the configuration file fails verification, the pipeline terminates immediately, and an email report of the problem is sent to the developer who submitted the commit.

[0017] Step 4 Environment Preparation: In the second stage of the pipeline, the system prepares the environment for construction based on the parsed configuration. The system starts a Docker container and downloads and configures various software packages according to the requirements of the configuration file for use in the next step.

[0018] Step 5: Build and Debris Generation: In the third stage of the pipeline, the system compiles the kernel, root file system, and various dependent packages in sequence, and generates the final build artifacts according to the definitions in the configuration file. The build artifacts contain various metadata such as the commit hash, build platform, and build time, facilitating quick identification of the build source during debugging.

[0019] Step 6: Debris Archiving: In the final stage of the pipeline, the system uploads the packaged build artifacts to the storage backend, using the commit hash of the build configuration file as the identifier for each artifact. Simultaneously, the storage backend adds detailed tags to the artifacts based on the build metadata, facilitating indexing by developers.

[0020] Step 7 Development Equipment Update: On the embedded development equipment, developers retrieve the latest system software version from the storage backend server by calling a pre-configured general script. Then, according to the developer's needs, the corresponding hierarchical deployment script is called to complete the update of the development equipment.

[0021] Developers commit configuration changes via Git, triggering an automated CI / CD pipeline build and release. The updates are then seamlessly applied to the embedded development environment using edge tools. This approach significantly simplifies operations and improves efficiency by shifting the developer's focus from "how to build" to "what is needed," fundamentally ensuring the consistency and reproducibility of the development environment.

[0022] CI / CD pipelines are core tools for achieving automation and efficient delivery in modern software development. At its core, they automate the processes of continuous integration (CI) and continuous deployment / delivery (CD) to quickly and securely deploy validated code to the target environment.

[0023] Webhook is an event-driven automated communication mechanism. Its core function is to enable real-time transmission of event information between two systems (or services) to trigger subsequent actions. It is widely used in CI / CD pipelines and third-party integration scenarios, such as automatically triggering a build after code submission or notifying business systems after payment completion. When a specific event occurs in the source system (such as GitHub or a payment platform), such as a code push or successful order payment, it proactively sends an HTTP request (usually a POST) to a preset URL in the target system (such as Jenkins or a business server), carrying event-related data. Upon receiving the request, the target system parses the data and executes preset logic (such as triggering a pipeline or updating the order status).

[0024] Beneficial Effects: This invention is an automated system for customizing and distributing embedded system software. It enables developers to manage the system software stack in a simple and intuitive way, and leverages existing, robust version management and collaboration systems to achieve synchronization among different developers. This system applies modern DevOps practices to the embedded development field, realizing and optimizing a complete workflow from system configuration to developer environment updates.

[0025] This invention fundamentally guarantees software quality and the reliability of the development process through an immutable and traceable automated process. Developers can use the system of this invention to efficiently deploy and verify embedded system software on embedded devices, achieve rapid comparison between different versions of software, and accurately locate the problem based on the information saved by the system according to this invention when errors occur during software verification.

[0026] The system of this invention allows developers to submit configuration changes via Git, triggering automatic building and release of the CI / CD pipeline, and finally seamlessly applying the updates to the embedded development environment through edge tools. By shifting the developer's focus from "how to build" to "what is needed", the system of this invention greatly simplifies the operation, improves efficiency, and fundamentally ensures the consistency and reproducibility of the development environment. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the system structure of the present invention; Figure 2 A flowchart for the steps of CI / CD automation. Detailed Implementation

[0028] The core idea of ​​the automated system and method proposed in this invention lies in decoupling the developer's "intent" from the "implementation" of the build process, and seamlessly connecting the two using a CI / CD toolchain. Developers express their requirements in a high-level, declarative manner, while the system is responsible for translating these requirements into low-level, complex build operations and automating subsequent distribution and deployment. The specific implementation methods of the four core modules constituting this system will be described in detail below.

[0029] 1. Configuration Management Module This module serves as the entry point for the entire automation process. It is designed to provide an extremely simple, intuitive, and powerful interface for developers to define and modify the software configuration of embedded systems.

[0030] Step 1: Design and Implementation of Declarative Configuration Files The core of this invention is the concept of "configuration as code." All configuration items of the system software, including kernel version, root file system image version, additional software packages, artifact format, and related configuration and compilation parameters, are defined through a text file. This file uses a highly readable and clearly structured format, such as YAML or JSON.

[0031] Taking the most commonly used format in this system as an example, the configuration file config.yml is written in YAML (YAML Ain't Markup Language) format. YAML is widely used in various tools, frameworks, and applications to define parameters, rules, or processes. Its core advantages are clear structure, high readability, greater conciseness than JSON, and lighter weight than XML.

[0032] A typical config.yml configuration file example is as follows: # config.yml # Define platform parameters target_arch: 'arm' target_endian: 'little' ... # Define system kernel version kernel_channel: 'analogdevicesinc / linux' kernel_commit: '6bb2ef19eae7b4ebda9978ee75d67ceadcd31cd6' # Kernel Configuration kernel_base_config: 'kernel / defconfig / analog / ad9361.kernel.config' kernel_config: - 'CONFIG_REMOTEPROC=y' - 'CONFIG_ZYNQ_REMOTEPROC=y' ... # Custom kernel module modules: - name: 'rtl8192eu-gh-mange' version: '528ae31705764d78cc117abd604d9b799bd52543' recipe: 'modules / thirdparty / rtl8192eu-gh-mange / recipe.yaml' ... # Define the root file system version rootfs_type: 'buildroot' buildroot_version: '2021.11.3' # Root file system configuration buildroot_base_config: 'rootfs / defconfig / buildroot / 20220526.config' buildroot_config: - 'BR2_PACKAGE_ZSH=y' - 'BR2_PACKAGE_NANO=y' - 'BR2_PACKAGE_NANO_TINY=y' ... # Custom Package packages: - name: 'python-numpy' version: '1.19.2' recipe: 'packages / br2_imported / python / numpy / 1.19.2 / recipe.yaml' dependency: - 'buildroot / python' - name: 'webd' version: '76dd4501da49c85ed5591dc72e62fc8cd0e2e608' recipe: 'packages / thirdparty / webd / recipe.yaml' dependency: - 'kernel / network' ... # Define product format artifacts: tar: enable: true cpio: enable: true compress: 'gzip' yaffs2: enable: false NFS: enable: true deploy_host: '10.10.10.10' deploy_path: ' / nfsboot / debug / rootfs' deploy_auth: 'xxxxxxxx' ... Through configuration files, developers can clearly see and define the system's basic attributes, the system kernel and included software packages and their versions, the system image loading method, and more. They can not only define basic software versions but also control complex feature enablement configurations, such as specific kernel characteristics and additional software package functionalities. This declarative configuration approach makes generating, modifying, and reviewing configurations simple and intuitive. Team members can review changes to system configurations as they would review business code, while also ensuring machine readability of the configurations.

[0033] Step 2: Git-based version control integration The aforementioned configuration files, along with necessary dependency configuration files, are stored in a dedicated Git repository. Git, as the de facto standard for version control in software development today, brings numerous benefits to configuration management: A complete change history: Every modification to config.yml is recorded as a Git commit. Using the `git log` command, you can clearly trace every evolution of the system configuration, understanding "who, when, why, and what changes were made." A comprehensive history can significantly reduce the time needed for troubleshooting.

[0034] Atomicity and Traceability: Each commit is an atomic change. Based on the commit's command hash, it's easy to locate the corresponding artifact or trace back to the problematic configuration file. If a configuration update introduces a problem, the system configuration can be easily rolled back to any previous stable state using the `git revert` command.

[0035] Branching and Collaboration: Development teams can leverage Git's branching feature for parallel development of new features. For example, a team can test a new library on a feature branch without affecting the stability of the main development branch. Once the feature is validated, the configuration changes can be merged into the main branch via a merge request.

[0036] By incorporating system configuration into Git management, this invention endows embedded system software stack development management with traceability, stability, and maintainability. Each built system artifact can be precisely traced back to a specific Git commit, providing a solid foundation for troubleshooting and version control.

[0037] 2. CI / CD Automation Execution Module This module is the core execution engine of this invention. It transforms the "intents" defined in the configuration management module into actual, deployable software artifacts. This module is driven by a CI / CD server (such as GitLab CI), and its workflow is defined in a pipeline configuration file, as follows: Step 1: Automatic Triggering of the Production Line The CI / CD server connects to the Git repository via a webhook. When a developer pushes changes to config.yml to the repository, the Git server determines the branch where the commit is located (by convention, commits on specific branches do not trigger pipelines). For commits that require pipeline operations, the CI / CD server is immediately notified, thus automatically triggering the predefined pipeline.

[0038] Step 2: Dynamic preparation of the environment The first stage of the pipeline is preparing the build environment. To avoid inconsistencies in the build environment, this invention employs Docker containerization technology. The system maintains a Docker image containing all necessary cross-compilation toolchains, libraries, and build tools. Each job in the pipeline runs in a brand-new, isolated container launched based on this image. This ensures that every build takes place in an identical and pristine environment, achieving 100% reproducibility of the build process.

[0039] Step 3: Configuration resolution and automated build execution At this stage, one of the scripts in the pipeline will perform the following operations: l Parse configuration: Read the config.yml file and extract information such as the build framework, kernel configuration, packages and versions to be built, and build artifact format.

[0040] l Obtain dependencies: Based on the parsing results, obtain the necessary software packages, source code, and other resources for the build process and place them into the build environment.

[0041] The build process involves two main steps: First, translating YAML configurations into instructions required by the underlying build system. For example, it dynamically generates Yocto's `local.conf` file or automatically executes buildroot commands like `make defconfig` to generate and modify build recipes. Second, it generates corresponding compilation and installation scripts based on custom kernel modules and package configurations. For instance, it configures module dependencies and deployment locations and methods based on the custom kernel module's `recipe.yaml` file. This abstraction is crucial, completely isolating developers from the internal details of complex tools like Yocto.

[0042] The build process involves invoking commands from the underlying build system to perform the actual compilation, linking, and packaging. Because the build process is centralized on the server, high-performance hardware and caching mechanisms can be leveraged to significantly reduce build time. For scenarios where problems might arise from the incorrect use of expired caches, developers can manually control whether caching is enabled or disabled through configuration files, including config.yml.

[0043] Step 4: Product packaging and deployment script generation After successful construction, the pipeline enters its final stage: l Collecting and Polishing Output: The script automatically collects all useful output from the build directory, such as the generated root filesystem archive (rootfs.tar.gz), the SDK installation script (sdk-installer.sh), and other binary files. If defined, the script will polish the output in a user-defined way, such as adding specific version information and resource files.

[0044] l Generating a deployment script: This is a key innovation of this invention. The pipeline not only packages the binary output but also dynamically generates a deployment script. This script contains all the commands needed to apply the update on the developer's host and slave devices, such as unzipping the SDK to a specified directory, setting environment variables, and updating symbolic links. This script can be called by the pre-installed automatic update script on the device, enabling one-click automated updates.

[0045] l Organize and package: Organize all collected artifacts into a directory named after the commit hash according to the configuration, and distinguish different artifact formats through subdirectories.

[0046] 3. Product Distribution Module This module is responsible for systematically storing and providing the build artifacts generated by the CI / CD automation module, and is a key module for ensuring the automation of development environment deployment.

[0047] The products generated on the production line are automatically packaged and uploaded to a centralized storage location. Currently, the product warehouse can be: Professional artifact repository systems: such as Sonatype Nexus and JFrog Artifactory. These tools offer robust version control, access control, and caching capabilities, minimizing the difficulty of artifact management.

[0048] l General-purpose object storage services: such as Amazon S3, Google Cloud Storage, MinIO, etc. Because object storage services are widely used in many fields, this invention supports using object storage services as the storage backend for artifacts. This allows for the reuse of existing storage infrastructure, thereby achieving highly available, scalable, and cost-effective artifact storage.

[0049] A simple HTTP file server: For smaller systems, a simple web server is sufficient for the storage backend. Alternatively, the system can utilize existing Git and Git LFS systems to store build artifacts, avoiding additional setup work.

[0050] Local NFS Server: To meet the needs of real-time, non-destructive testing, the system supports direct deployment of build artifacts to the NFS endpoint specified in config.yml. Developers can directly specify NFS boot in the bootloader during testing, enabling them to test the latest system software without modifying the existing firmware of the development board, and to flexibly switch between different versions with virtually no time cost.

[0051] Regardless of the storage backend used, unified management and flexible access to build artifacts can be guaranteed.

[0052] In addition, to facilitate developers in finding and managing artifacts of specific versions, and to enable automated scripts to process artifacts, this module also maintains several auxiliary files. For example: `latest`: An alias that is a symbolic link to the latest version of the current branch. Developers can use this alias to directly access the latest version of the artifacts, making it convenient for them to write scripts to process the artifacts.

[0053] `manifest.json`: A JSON file that records metadata for all historical builds, including Git commit hashes, build times, version tags, and corresponding artifact package URLs and checksum hashes. This allows developers to quickly browse version history and retrieve artifacts as needed without accessing the original Git repository, facilitating the automated reproduction and debugging of issues in specific versions.

[0054] A typical manifest.json file looks like this: { "version": "v1", "latestRelease": { "versionTag": "20220704", "gitCommitHash": "4f99389ebb0ef4bf013d4ce0d186f13baa7ab144" }, "artifactVersions": [ { "versionTag": "20220526", "gitCommitHash":"2baa8a9573bd81d7607ab9ee080dee5bbf71dee2", "buildTimestamp": "2022-05-26T14:26:13Z", "buildBy": "nju-git-ci", "artifactType": "tarball", "targetPlatform": "ad9361z7035 / buildroot", "deploymentScript": "deploy_zynq_buildroot.sh", "artifactUrl": "20220526 / rootfs.tar", "artifactChecksum":"sha256-32f40ac631fca03109b9a5c6a034c4aa3f5a530ec9aa91ad5c315fb2aa7dd663" }, { "versionTag": "20220531", "gitCommitHash":"92e91510281cf5bc6336049ee48fb7acb5466341", "buildTimestamp": "2022-05-31T09:12:58Z", "buildBy": "nju-git-ci", "artifactType": "tarball", "targetPlatform": "ad9361z7035 / buildroot", "deploymentScript": "deploy_zynq_buildroot.sh", "artifactUrl": "20220531 / rootfs.tar", "artifactChecksum":"sha256-3c6821e73bb22a12f6c95bd0da0e79b869c52a26f172be208f89f3770ff63f7e" }, { "versionTag":"20220704", "gitCommitHash":"4f99389ebb0ef4bf013d4ce0d186f13baa7ab144", "buildTimestamp": "2022-07-04T23:46:32Z", "buildBy": "nju-git-ci", "artifactType": "tarball", "targetPlatform": "ad9361z7035 / buildroot", "deploymentScript": "deploy_zynq_buildroot.sh", "artifactUrl": "20220704 / rootfs.tar", "artifactChecksum":"sha256-1449125d53eea760ef248ad204f366e7651d7036416c55d59de545d555e7ed17" } ] } 4. End-side deployment module This module provides a concise interface for developers, simplifying the complex environment update process into a single command and offering a flexible and convenient system software deployment experience.

[0055] Step 1: Design of end-side tools Deploying embedded system software is a complex and highly customizable process, requiring extensive adjustments based on different configurations. Therefore, this invention breaks down the deployment process into multiple parts, forming a three-layer deployment script architecture: "Version-specific scripts": The parts that need to be customized according to the configuration file are generated as part of the artifact and stored in the artifact repository. Each generated artifact has its own unique script. These scripts are called by "platform-common scripts" during deployment to complete the deployment of the actual artifact. For example, a certain software needs to compile and generate a shader cache based on the specific characteristics of the hardware platform during deployment to improve the performance of the graphical interface. In artifact versions that do not include this specific software, obviously this script for compiling the shader cache is not needed; therefore, this script will be saved as a "version-specific script" as part of the artifact.

[0056] "Platform Common Scripts": These common scripts, related to the platform / deployment method, are highly coupled with the configuration management and automation pipeline modules, and are therefore stored in a Git repository. These scripts are typically not modified; they are only updated when the configuration management and automation pipeline modules are maintained or updated. During deployment, these scripts are called by "user scripts," which then select "version-specific scripts" for further invocation based on the parameters provided.

[0057] "User Script": This is a lightweight command-line script, essentially a wrapper around user-input parameters. The user script's job is to analyze the deployment platform, version, and other parameters based on the user's input, and then call specific "platform-common scripts" for further operations. Due to its layered script design, the user script requires minimal updates, providing developers with a simple and convenient deployment experience.

[0058] Step 2: Automated Product Retrieval When developers execute a command in their terminal to start a deployment, the deployment module performs the following actions: l Connect to the product distribution module according to the configuration.

[0059] Based on user input, determine the method of calling the secondary script and read the version manifest file (latest or manifest.json, depending on user input) to obtain the download URL of the target artifact package.

[0060] Use tools such as curl or wget to download the specified artifact package to a local temporary directory.

[0061] In actual development, developers often submit a new version of a configuration file (config.yml) and then need to quickly test the artifacts generated from this new configuration file. Since embedded system software is complex and takes a considerable amount of time to build, starting deployment immediately could easily cause the deployment process to fail because the build artifacts are not yet ready. Therefore, if the script does not find the required commit in the build artifacts, it will search in Git commits and pipeline jobs. If found, it means the artifact build has started but is not yet complete. In this case, the deployment module will not terminate due to "artifact not found," but will wait for the corresponding pipeline to complete before continuing the deployment process. If the pipeline fails, the script will prompt the user before terminating the deployment process. With this design, developers can update the configuration file and start deployment simultaneously, without having to wait for the pipeline to complete before initiating automated deployment on the device.

[0062] Step 3: Automated Deployment Execution Once the download is complete, the script will automatically execute the following steps: Reboot the device and enter the rescue partition. The rescue system is a streamlined, independent operating system with basic system maintenance and update functions. It is stored in a dedicated partition on the device, separate from the main system. The rescue system ensures that even if system software updates fail, the embedded device will not become unusable, guaranteeing reliable and secure deployment.

[0063] The script verifies the integrity of downloaded files. It retrieves the checksum of the artifact files from the artifact repository's metadata and verifies the integrity of the local files to prevent system deployment issues caused by file corruption.

[0064] The script reads the partition flags. The device's system partition is divided into two partitions, A and B. The currently deployed system resides on partition A, and the updated system will be deployed to partition B. The script reads the mapping from the current A and B partitions to the physical partitions, ensuring that the new system is correctly deployed to partition B. The A and B partition mechanism guarantees the safety and reliability of the update. Even if the new system fails to boot, the partition flags can be rolled back through the recovery system, preventing the embedded device from becoming unusable.

[0065] l Invokes the secondary deployment script. The secondary script is responsible for handling all platform-related, tedious configuration tasks until deployment is complete.

[0066] l Switch partition flags. Modify the flags of partitions A and B to direct the next boot to the new system partition. The new system deployment is now complete.

[0067] The system of this invention creates a perfect closed loop for updating the entire embedded system software. Developers' only task is to modify a simple configuration file, push it to Git, run the deployment script, and wait for the results. This yields a fully functional embedded development environment that is completely synchronized with the latest configuration and consistent with all other team members. This system not only significantly improves efficiency, but more importantly, it fundamentally guarantees software quality and the reliability of the development process through immutable, traceable, and automated processes.

[0068] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An automated system for system software customization and distribution deployment for embedded systems, characterized in that, The system consists of four modules, which work together to form a closed-loop automation process: Configuration management module: The configuration management module is the starting point of the entire process and the only user interaction interface. It is a declarative configuration file stored in the Git version control system. The configuration file supports multiple common formats, and the module uses Git for version control, so that each system configuration change has a clear and traceable history, and can be rolled back to any historical version. Automatic execution module: This module is implemented by a CI / CD server. The CI / CD server is tightly integrated with the Git repository of the configuration management module through the Webhook mechanism. Once the configuration file changes are pushed to the specified branch, the CI / CD pipeline will be automatically triggered. The pipeline will perform a series of predefined and automated tasks, including: using Docker to prepare a clean build environment, parsing the contents of the configuration file, converting the configuration content into configuration items and instructions that the underlying build system understands, resolving software dependencies and preparing source code and configuration; Finally, perform time-consuming cross-compilation and packaging process. Product management module: When the automatic execution module successfully completes the build, the build product is archived, stored and distributed through this module. The build product usually includes a complete root file system image, a software development kit (SDK) that matches the build image, and an image deployment script. The product management module manages and provides build products. All build products are stored in the storage backend according to the rules specified by the module. The storage backend is a professional artifact repository or an HTTP file server or a cloud storage service. According to the Git commit hash value of the configuration file, all build products will be assigned a unique version identifier and uploaded to the storage backend. End-side deployment module: Deploy system software on development devices as needed for quick deployment, verification, and rollback. This module is a collection of design and corresponding automation operation scripts. By retrieving from the predetermined storage backend, the module can obtain the latest version of the current system software, as well as the version corresponding root file system, supporting tool chain and deployment script. Then, the module can upgrade or modify the current system software on the development device according to the instructions of the deployment script. These four modules form a complete and automated data flow closed loop, and the steps of the system work are as follows: Step 1: Configuration update: The developer generates a new configuration file according to the development requirements through the configuration management tool and pushes it to the Git repository. Step 2: Pipeline call: After the Git server receives the developer's synchronized commit, it calls the Webhook to start the pipeline automation program. Step 3: Configuration parsing: In the first stage of the pipeline, the system call program parses the configuration, including the architecture of the target platform, the system kernel configuration, the type and configuration method of the root file system, the dependency and source of dependent software packages, the format and storage method of the product, and saves the results as intermediate products for debugging and other processing. At the same time, the system verifies whether the configuration file is correct and whether there are conflicts that cannot be resolved. If the configuration file cannot be verified, the pipeline operation is terminated directly, and an email is sent to the developer who submitted the commit to report the problem; Step 4: Environment preparation: In the second stage of the pipeline, the system prepares the build environment according to the parsed configuration. The system starts a Docker container and downloads and configures various software packages in the container according to the requirements of the configuration file for the next step; Step 5: Build and product generation: In the third stage of the pipeline, the system compiles the kernel, root file system, and various dependent software packages in sequence, and generates the final build product according to the configuration file definition. The build product contains the commit hash, build platform, build time, and other metadata, which can be used to quickly determine the source of the build during debugging; Step 6: Product archiving: In the last stage of the pipeline, the system uploads the packaged build product to the storage backend, using the commit hash of the configuration file used for building as the identifier of the build product. At the same time, the storage backend adds detailed tags to the product based on the build metadata, making it easy for developers to index the build product; Step 7: Development device update: On the embedded development device, the developer retrieves the latest system software version from the storage backend server by calling the pre-configured general script, and then calls the hierarchical deployment script corresponding to the version to complete the update of the development device.

2. The automated system of claim 1, wherein, The CI / CD pipeline automation execution steps convert the "intent" defined in the configuration management module into actual, deployable software products; Step 1: Automatic triggering of the pipeline The CI / CD server connects to the Git repository through Webhook; when the developer pushes the modified configuration file to the repository, the Git server judges according to the branch where the commit is located, and notifies the CI / CD server immediately for the commit that needs to be operated by the pipeline, thereby automatically triggering the pre-defined pipeline; Step 2: Dynamic preparation of the build environment The first stage of the pipeline is to prepare the build environment; Docker container technology is used; the system maintains a Docker image containing all the necessary cross-compilation tool chains, libraries, and build tools; each job of the pipeline runs in a new, isolated container based on this image; Step 3: Configuration parsing and automated build execution A script in the pipeline performs the following operations: Parse the configuration: read the configuration file and extract the build framework, kernel configuration, software packages and versions to be built, and product format information; Build instruction generation: on one hand, the configuration file content is translated into the instruction required by the underlying build system; on the other hand, the corresponding compiling and installing script is generated according to the configuration of the customized kernel module and software package; Build execution: the command of the underlying build system is invoked to perform the actual compiling, linking and packaging work; since the build process is centrally performed on the server, high-performance hardware and cache mechanism can be utilized to significantly shorten the build time; for the scenario that may be incorrectly used due to the expired cache, the developer can manually control the enablement of the cache through the configuration file; Step 4: product packaging and deployment script generation After the build is successful, the pipeline enters the last stage: Collection and modification of products: the script automatically collects all useful outputs from the build directory; if defined, the script performs user-defined modification on the products, including adding specific version information, resource files; Generation of deployment script: the pipeline not only packages the binary products, but also dynamically generates a deployment script; the script contains all the commands required for applying this update on the developer host machine and the slave machine, including decompressing the SDK to the specified directory, setting the environment variable, updating the symbolic link; The script can be called by the automatic update script pre-installed on the device to realize one-key automatic update; Arrangement and packaging: all collected products are arranged and packaged according to the configuration in the directory named by the commit hash, and different product formats are distinguished by subdirectories.

3. The automated system of claim 1, wherein, The work of the side deployment module includes 1) Design of side tool: the deployment process is divided into multiple parts to form a three-layer deployment script architecture: "Version-specific script": the part that needs to be customized according to the configuration file is generated as part of the product and saved in the product repository; each generated product has its own unique script; these scripts are called by "platform common script" to complete the actual deployment of the product during deployment; "Platform common script": the common script related to the platform / deployment method is essentially highly coupled with the configuration management and automation pipeline module, and therefore is saved in the Git repository; These scripts usually will not be modified, and only when the configuration management and automation pipeline module is maintained and updated, the corresponding scripts will be updated; these scripts are called by "user script" during deployment, and then they select "version-specific script" for further calling according to the parameters called; "User script": this is a lightweight command-line script that is a wrapper for some user input parameters; the work of "user script" is to analyze the platform, version and other parameters of the deployment according to the user input content, and then call the specific "platform common script" for further operation; due to the layered script design, the user script basically does not need to be updated to provide a simple and convenient deployment experience for developers; 2) The side deployment module performs automatic product pulling When the developer executes the command in the terminal to start a deployment, the side deployment module performs the following operations: According to the configuration, connect to the product distribution module; According to the user input, determine the calling mode of the secondary script, and read the version list file to obtain the download URL of the target product package; Use tools such as curl or wget to download the specified product package to a local temporary directory; In actual development, it is common for developers to first submit a new version of the configuration file config.yml, and then need to use the product generated by the new configuration file for testing as soon as possible; If the script does not find the required commit in the build product, it will instead look for it in the Git commit and pipeline job, and wait for the corresponding pipeline to complete before continuing the deployment process; If the pipeline fails, the script will prompt the user and then terminate the deployment process; 3) Automatic deployment execution After downloading, the script will automatically perform the following steps: Restart the device and enter the rescue system partition; The rescue system is an independent operating system with basic system maintenance and update functions, and is stored in a dedicated partition of the device, independent of the main system; The rescue system can ensure that the update of system software will not cause the embedded device to "brick" even if it fails, ensuring the reliability and security of deployment; Verify the integrity of the downloaded file; The script obtains the verification hash of the product file from the metadata of the product warehouse, verifies the integrity of the local file, and prevents file corruption from causing problems with system deployment; Read the partition flag; The system partition of the device is divided into AB two partitions, the partition where the current deployment system is located is A partition, and the partition where the updated system will be deployed is B partition; The script reads the mapping of the current AB partition to the physical partition to ensure that the new system is correctly deployed to the B partition; The AB partition mechanism ensures the safety and reliability of the update, even if the new system cannot start, it can also roll back the partition flag through the rescue system to avoid the embedded device "brick"; Call the secondary deployment script; The secondary script is responsible for handling all platform-related and tedious configuration tasks until the deployment is complete; Switch the partition flag; Modify the AB partition flag to point to the new system partition for the next startup, and complete the deployment of the new system.

Images