Lightweight processing method for general software

Abstract

The invention relates to the field of embedded systems, and discloses a lightweight processing method for universal software, which comprises the following steps of: setting a front-end configuration page, customizing a corresponding component configuration list by a user according to a specific function or service demand through the front-end configuration page, the component configuration list is used for guiding the system manager to screen out required components from a software repository, and packaging the screened components to form embedded firmware; the types of the components comprise a core component, a supplier component, a third-party component and a configuration file, and the constituent elements of the components comprise an application process, a dynamic library and a static library; the technical problems that due to a traditional full-static compiling scheme, the program size is increased exponentially, and maintenance is difficult are solved.

Description

Technical Field

[0001] The present invention relates to the field of embedded systems, and in particular to a lightweight processing method for general software. Background Art

[0002] In the embedded world, hardware resources are limited. To save hardware costs, many embedded devices use lower-cost, lower-power CPUs, memory, and disks. These components are particularly demanding, with some requiring only a dozen MB of memory. With limited hardware resources, efficiently managing and utilizing them becomes a key issue in embedded system development.

[0003] A common approach is to use a fully static compilation solution, linking all program source files into a single executable process. The advantage of this approach is that the generated program does not require additional dynamic loading at runtime, resulting in fast startup and high execution efficiency. However, when the application software needs to be expanded, static recompilation must be performed to generate a new executable process. As demand continues to expand, a large number of static processes will accumulate in the system. These processes cannot share code segments, resulting in an exponential increase in program size. Furthermore, fully static compilation makes it difficult to modularize the software, increasing the difficulty of maintenance and upgrades.

[0004] To address the low code reuse and large program size associated with static compilation, the concept of dynamic libraries was introduced in the software field. Dynamic libraries allow different processes to share the same code at runtime, significantly reducing program size. However, in embedded systems, since processes are the smallest unit of resource allocation for the operating system, when multiple processes simultaneously load a dynamic library, the operating system creates a data segment for each process. Although these data segments are typically read-only, they still occupy valuable memory resources, causing the program's running memory to remain high and reducing memory utilization. Summary of the Invention

[0005] The present invention aims to provide a lightweight processing method for general software to solve the technical problem that traditional fully static compilation solutions cause program size to grow exponentially and become difficult to maintain.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions: a lightweight processing method for general-purpose software, comprising setting up a front-end configuration page, through which a user customizes a corresponding component configuration list according to specific functional or service requirements. The component configuration list is used to guide a system manager to filter required components from a software repository and package the filtered components into embedded firmware; Component types include core components, vendor components, third-party components, and configuration files. Component components include application processes, dynamic libraries, and static libraries.

[0007] The principle and advantages of this solution are as follows: In practical application, this solution provides users with a customized entry point by setting up a front-end configuration page. Users can customize the corresponding component configuration list on the front-end configuration page based on specific functional or service requirements. This component configuration list serves as a key guide, allowing the system manager to accurately select the required components from the software repository. The selected components are then packaged and ultimately formed into embedded firmware.

[0008] With traditional fully static compilation solutions, as demand increases, a large number of static processes accumulate in the system. These processes cannot share code segments, causing the program size to expand rapidly. This solution, however, uses a component-based customization approach, allowing users to select only the components they need based on their actual needs. This avoids unnecessary code loading, effectively controls program size, and prevents exponential growth.

[0009] Fully static compilation makes software modularization difficult, increasing the complexity of maintenance and upgrades. This solution, through component-based design, breaks the software into multiple independent components, each with distinct functions and responsibilities. When software maintenance or upgrades are required, only the specific components need to be modified, eliminating the need for large-scale changes to the entire software. This significantly reduces the complexity of maintenance and upgrades and improves software maintainability.

[0010] Component-based design allows software to be developed and organized in components. These components are relatively independent, offering excellent reusability and interchangeability. Different components can be independently developed, tested, and deployed, allowing developers to flexibly combine components based on actual needs and quickly build software systems that meet diverse functional requirements. This effectively increases the software's modularity, flexibility, and scalability.

[0011] Preferably, as an improvement, in addition to the configuration file, each type of component in the software warehouse has an independent property file for describing basic information of the component; When users perform customization operations in the front-end configuration interface, they obtain the dependencies of each component and whether it can be tailored based on the component property file, delete unnecessary components, and retain necessary components to form a component configuration list.

[0012] The beneficial effect of this improvement is that in traditional methods, due to the inability to accurately determine the dependencies between components, some components that are not actually needed may be loaded, resulting in excessive memory usage. This improvement solution provides information on whether a component can be tailored through a property file. When users customize the component configuration list, they can delete unnecessary components according to actual needs and retain only necessary components. The system will also automatically tailor components that have no dependencies or can be tailored based on this information during packaging, further reducing the size of the program, reducing memory usage during program runtime, improving memory utilization, and enabling embedded devices to run more efficiently with limited hardware resources.

[0013] Preferably, as an improvement, when the system manager receives a component configuration list customized by a front-end user and triggers firmware packaging, a clipping unit is used to clip components without dependencies in the component configuration list according to component dependencies to form a software component list, and the component details in the software component list are used for component packaging to form the final firmware.

[0014] The beneficial effect of this improvement is that the trimming unit acts as an automated tool and plays a role when the system manager receives the component configuration list customized by the front-end user and triggers the firmware packaging. It accurately identifies and trims the components without dependencies in the component configuration list based on the dependencies recorded in the component property file to form a software component list, and then packages the components based on this list to form the final firmware. This process realizes the automated screening and optimization of components, which is more efficient and accurate than the previous manual deletion of components by users, and avoids possible omissions or errors that may occur humanly. At the same time, automated trimming can accurately remove unnecessary components, effectively reduce the size of the program, reduce memory and disk usage, improve resource utilization, and solve the problems of code segments that cannot be reused and the program size being too large under the fully static compilation scheme.

[0015] Preferably, as an improvement, component dependencies are located in component property files, and the dependencies between components will eventually generate a directed acyclic graph; Dependencies include strong dependencies and weak dependencies. Strong dependencies mean that the current component will not function properly without the dependent party. Weak dependencies mean that only some functions of the current component will be unavailable without the dependent party, but the component can still function normally. In a strong dependency relationship, when the current component is not pruned or deleted, its dependent party cannot be pruned or deleted.

[0016] This improvement has the beneficial effect of recording component dependencies in a properties file and ultimately generating a directed acyclic graph, presenting the dependencies between components in an intuitive and clear manner. This allows developers and maintainers to more easily understand and analyze the relationships between components, helping them make more informed decisions during software development and maintenance, and avoiding errors and conflicts caused by unclear dependencies.

[0017] By defining strong and weak dependencies, the system can perform more refined component tailoring based on different dependency levels. In strong dependencies, it is clearly stipulated that if the current component has not been tailored or deleted, its dependents cannot be tailored or deleted, ensuring the integrity of component operation. In weak dependencies, the dependent components can be tailored under certain conditions. This further optimizes program size and resource usage while ensuring the normal operation of basic functions, improving system flexibility and scalability.

[0018] Preferably, as an improvement, when the firmware is running, the components are loaded and started according to the dependency relationship, with the dependent components started first, and then the dependent components.

[0019] The beneficial effect of this improvement is that when the firmware is running, it is loaded and started strictly according to the dependency relationship of the components, with dependent components started first, and then dependent components. This orderly startup method ensures that the program can run normally according to the expected dependency order, avoiding resource conflicts or program errors caused by improper startup order. For example, some components may need to rely on services or data provided by other components during initialization. If the startup order is wrong, these components may not work properly, which in turn affects the stability of the entire system. By loading and starting components according to the dependency relationship, a solid guarantee is provided for the stable operation of the program, and indirectly supports the efficient use of hardware resources, because stable system operation can reduce resource waste and repeated operations caused by program errors.

[0020] Preferably, as an improvement, when the firmware is running, a component prototype process is first created, and the prototype process is responsible for loading the common resources shared by all components; When the component manager needs to create a specific component, it derives a new process from the component prototype process through the fork system call. The newly derived component process initially shares exactly the same memory space with the component prototype process, including all loaded common resources.

[0021] The beneficial effect of this improvement is that when the firmware is running, a component prototype process is first created to load common resources shared by all components. When a specific component needs to be created, a new process is derived from the component prototype process through the fork system call. The newly derived component process initially shares exactly the same memory space with the component prototype process, including all loaded common resources. This method minimizes memory usage by sharing common memory, avoiding the waste of resources caused by multiple processes loading the same resources at the same time without modification during dynamic library loading. For example, multiple components may need to use some common library functions or data structures. By sharing common resources, these components do not need to load a copy of the same resources each, which greatly reduces memory usage, improves memory utilization, and effectively solves the problem of high memory usage.

[0022] Preferably, as an improvement, when a component instance needs to modify its shared public resources, the system triggers the write-time copy mechanism, allocates new memory space for the component instance, and copies the resources that need to be modified from the component prototype process to the newly allocated memory space; other read-only component instances continue to share the public resources in the component prototype process.

[0023] The beneficial effect of this improvement is that when a component instance needs to modify its shared public resources, the system triggers the copy-on-write mechanism, allocates new memory space for the component instance, and copies the resources that need to be modified from the component prototype process to the newly allocated memory space, while other read-only component instances continue to share the public resources in the component prototype process. This mechanism implements modification isolation, ensuring that the component that needs to modify resources can perform the modification operation independently without affecting the normal use of public resources by other components. It also avoids allocating independent memory space for public resources for each component instance. This further optimizes memory usage, improves memory utilization, and reduces memory usage while ensuring the integrity of program functions.

[0024] Preferably, as an improvement, the component's property file is edited by a corresponding form during the development process, and the form includes the name, version, cropping mark and dependency relationship; the cropping mark indicates whether the current component can be cropped, and the dependency relationship is used to describe the pre-dependent components and the corresponding dependency level.

[0025] This improvement has the beneficial effect of recording component name, version, trimming mark, and dependencies in properties files by editing the corresponding form during the development process, making component management more standardized and systematic. Developers and maintainers can quickly understand basic component information and features by viewing the properties files, facilitating component selection, configuration, and maintenance, thereby improving work efficiency.

[0026] The cropping mark explicitly indicates whether the current component can be cropped, providing a basis for the system to crop components during the packaging process, allowing the system to flexibly remove unnecessary components based on actual needs and optimize program size. Dependency relationships are used to describe pre-dependent components and their corresponding dependency levels. This helps the system accurately analyze the relationships between components, implement a reasonable component loading and startup sequence, and ensure system stability and functional integrity during component cropping. This improves the flexibility and scalability of component management and adapts to the needs of different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a firmware packaging flow chart of an embodiment of the present invention.

[0028] Figure 2 Schematic diagram of dependency relationships in an embodiment of the present invention.

[0029] Figure 3 for Figure 2 Startup sequence diagram of dependencies.

[0030] Figure 4 This is a memory layout diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following is further described in detail through specific implementation methods: Example Basically as attached Figure 1 As shown, a lightweight processing method for general software includes: Users can customize the component configuration list based on specific functional or service requirements through the front-end configuration page. The component configuration list is used to guide the system manager to filter the required components from the software repository. The front-end and back-end design for user configuration can adopt a general B / S architecture.

[0032] Component types include core components, vendor components, third-party components, and configuration files. Core components are the core modules that make up the embedded software. They are developed internally by the development team and provide the basic functions and key services of the system. Vendor components are specialized modules provided by external suppliers and usually contain support for specific hardware or proprietary functions. Third-party components are modules from the open source community or other third parties that provide common functions, promote development efficiency and code reuse. Configuration files are used to define system behavior, module parameters, and operating environment files to ensure that the software can run correctly under different conditions. The constituent elements of a component include application processes, dynamic libraries, and static libraries.

[0033] In addition to the configuration file, each component in the software repository has an independent property file. The property file is used to describe the basic information of the component, including the component name, source, version information, pre-dependent components, required configuration files, approximate storage space required, approximate CPU usage required, etc. This file is edited by the corresponding form during the software development process, and the form can be saved in database, JSON, or XML format.

[0034] Property files can be configured as either a summary node form or a single component form. The summary node form consolidates key information about all components in the software repository, including the total number of components, the number of components by type, version distribution, and a status overview. As an entry point or overview page for the software management system, the summary node form helps developers quickly locate components requiring management or view specific information.

[0035] A form for detailed information about a single component. Specific form fields that can be designed include: Cropped: true indicates that the current component can be cropped, and the binary file and resource file will be deleted during packaging.

[0036] Name: The process name or library name, which must match the full word of the binary file.

[0037] Version: The version number of the current component.

[0038] Description: used to describe the information of the current node.

[0039] License: Only required for opensource, used to describe the license of the current open source library. Multiple licenses are separated by semicolons.

[0040] Tag: Used to add a tag to the current binary file for easy searching. Multiple tags are separated by semicolons.

[0041] Dependency (dep): used to describe pre-dependent components, divided into three types of dependencies: must, weak, and runtime, and supports wildcard matching expressions; for example, * must: strong dependency; * weak: weak dependency; * runtime: strong dependency, but it is a non-component dependency; multiple dependencies are separated by semicolons.

[0042] Resource: divided into RAM, ROM, and files resources, which describe the memory size, disk size, and file list required by the component; * RAM: The memory size required by the current component; supports KB / MB / GB, * ROM: The disk size required by the current component, the unit supports KB / MB; * Files: The file list required by the current component, the absolute path.

[0043] When users perform customization operations in the front-end configuration interface, they can obtain the dependencies of each component and whether it can be tailored and adjusted based on the component property file, delete unnecessary components, and retain necessary components to form a component configuration list.

[0044] When the system manager receives a customized component configuration list from a front-end user and triggers firmware packaging, the cutting unit cuts out components in the component configuration list that do not have dependencies based on their dependencies to form a software component list. The components in the software component list are then packaged to form the final firmware.

[0045] Among them, the trimming unit is an automated tool or script that identifies and trims unnecessary components based on predefined rules or configuration files. The trimming process may involve multiple steps such as code analysis, dependency resolution, and resource usage assessment. When users customize the component configuration list on the front end, some components that are not required and have no dependencies will be screened out based on the dependencies. Although this can also achieve the purpose of trimming components, the effect may be affected by the user's professional knowledge and experience. If the user does not accurately understand the system requirements or component dependencies, it may lead to over- or under-trimming, thereby affecting the system's functionality or performance. The trimming unit can more accurately identify and trim unnecessary components, thereby reducing the firmware size, optimizing system performance, and potentially improving system reliability. Because the trimming process is automated, it can reduce human errors and improve trimming efficiency.

[0046] The tailoring unit and front-end custom filtering address different requirements. The tailoring unit is suitable for embedded systems that require large-scale deployment and have strict requirements on system performance and resource usage. Automated tailoring ensures consistency and efficiency across all deployed systems. Front-end custom deletion is more suitable for scenarios that require flexible customization and specific requirements for system functionality. Users can freely select and delete unnecessary components within the front-end interface based on their actual needs.

[0047] When the firmware is running, components are loaded and started according to their dependencies, prioritizing dependent components before starting dependent components, ensuring that the program starts in the expected dependency order. If a component has no dependencies, it can start directly without waiting. If a component has strong dependencies, it must wait for the predecessor dependencies to start before starting. If a component has weak dependencies, it can still start directly without waiting, but it must implement fault tolerance within the component for scenarios where the predecessor dependencies have not yet started.

[0048] The dependency relationship is located in the component's property file. This property file is written into the component when the component is developed. The dependency relationship between the components will eventually generate a directed acyclic graph. Figure 2 As shown in the figure, the figure contains strong dependencies and weak dependencies. Strong dependency means that the current component will not be able to operate normally if the dependent party is missing. Strong dependency corresponds to the must in the form field, which is represented by a solid line in the figure. Weak dependency means that when the current component lacks the dependent party, only some functions cannot be used but it can still operate normally. Weak dependency corresponds to the weak in the form field, which is represented by a dotted line in the figure. Figure 2 Taking the dependency connection relationship as an example, since component A has no dependencies, it can be directly deleted from the component list during the firmware packaging process. However, since component G is strongly dependent on components E and H, the deletion condition for component G is that both components E and H are deleted.

[0049] As attached Figure 3 As shown, the attached Figure 2 The startup order of components in the dependency relationship. Component A, which has no dependencies, components B and F, which are strongly dependent and have no preceding strong dependencies, and component G, which has no strong dependencies, must start first. Next, components C and G, whose preceding strong dependencies have already started, start. Finally, components E and H, whose preceding strong dependencies have already started, start.

[0050] During the operation of embedded firmware, a large number of components in the system only need to read and not modify public resources (i.e., read-only but not write-only). In order to avoid memory waste caused by repeated loading of dynamic library data segments, we adopted a "write-time reuse" memory optimization strategy.

[0051] As attached Figure 4 As shown, the upper dotted box is the independent memory of each component, and the lower dotted box is the public memory. This method places the component prototype process in the public memory area. When reading only and not writing, the component prototype is only called to the corresponding component through fork, and no actual copy is made.

[0052] When a program starts, a component prototype process (Module Prototype) is first created. This prototype process is responsible for loading common resources that may be shared by all components, including but not limited to configuration files, system handles, dynamic link libraries, etc. These resources exist in memory in read-only form for subsequent component reuse.

[0053] When the Module Manager needs to create a concrete module, the firmware's core scheduling module, responsible for monitoring system status and component requirements, does not directly load all of the component's resources. Instead, it derives a new process from the component prototype process via a fork() system call. Due to the nature of fork(), the newly derived component process initially shares the exact same memory space as the component prototype process, including all loaded common resources. This means that multiple component instances can share common resources in the same memory without modifying any of the common resources, significantly reducing memory usage.

[0054] When a component instance needs to modify its shared public resources, a write operation occurs, triggering the system's Copy-On-Write (COW) mechanism. At this point, the system allocates new memory space for the component instance and copies the modified resources from the component prototype process into this newly allocated memory space. Thereafter, all write operations for this component instance will be performed within this newly allocated memory space, without affecting other component instances still using the shared resources. Other read-only component instances continue to share the shared resources in the component prototype process, without requiring any memory copies.

[0055] In this way, the "write-time reuse" solution effectively utilizes the read-only and non-write characteristics of components in embedded systems, minimizes memory usage by sharing common memory, and provides the necessary isolation when modifications are required to ensure system stability and reliability.

[0056] The present invention has been put into practical use in the fields of embedded vehicle-mounted and security monitoring equipment, organizing and arranging embedded software in a more orderly manner while minimizing the hard disk and memory resources required by the embedded software.

[0057] The actual operating environment in which it is put into use: Total memory: 38MB in total, with 12MB of remaining memory available during operation; Total disk: 11MB in total, with 2MB of free space remaining during operation; CPU: The single-core A7 processor using ARMv7 architecture can run smoothly and has extremely low requirements for CPU performance.

[0058] The above is only an embodiment of the present invention, and the common knowledge such as the specific technical solutions and / or characteristics in the solution are not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several variations and improvements can be made, which should also be regarded as the scope of protection of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the description can be used to interpret the content of the claims.

Claims

1. A lightweight processing method for general software, characterized by: This includes setting up a front-end configuration page, through which users can customize a corresponding component configuration list based on specific functions or service requirements. The component configuration list is used to guide the system manager to filter out required components from the software warehouse and package the filtered components into embedded firmware; Component types include core components, vendor components, third-party components, and configuration files. Component components include application processes, dynamic libraries, and static libraries.

2. The lightweight processing method for general software according to claim 1, characterized in that: In addition to the configuration files, each type of component in the software repository has an independent property file that describes the basic information of the component; When users perform customization operations in the front-end configuration interface, they obtain the dependencies of each component and whether it can be tailored based on the component property file, delete unnecessary components, and retain necessary components to form a component configuration list.

3. The lightweight processing method for general software according to claim 2, characterized in that: When the system manager receives the component configuration list customized by the front-end user and triggers firmware packaging, it uses the cutting unit to cut the components without dependencies in the component configuration list according to the component dependencies to form a software component list, and then packages the components according to the component details in the software component list to form the final firmware.

4. The lightweight processing method for general software according to claim 3, characterized in that: Component dependencies are located in the component's property files, and the dependencies between components will eventually generate a directed acyclic graph; Dependencies include strong dependencies and weak dependencies. Strong dependencies mean that the current component will not function properly without the dependent party. Weak dependencies mean that only some functions of the current component will be unavailable without the dependent party, but the component can still function normally. In a strong dependency relationship, when the current component is not pruned or deleted, its dependent party cannot be pruned or deleted.

5. The lightweight processing method for general software according to claim 4, characterized in that: When the firmware is running, components are loaded and started according to their dependencies. Dependent components are started first, followed by dependent components.

6. The lightweight processing method for general software according to claim 5, characterized in that: When the firmware is running, a component prototype process is first created. This prototype process is responsible for loading the common resources shared by all components. When the component manager needs to create a specific component, it derives a new process from the component prototype process through the fork system call. The newly derived component process initially shares exactly the same memory space with the component prototype process, including all loaded common resources.

7. The lightweight processing method for general software according to claim 6, characterized in that: When a component instance needs to modify its shared public resources, the system triggers the copy-on-write mechanism to allocate new memory space for the component instance and copy the resources that need to be modified from the component prototype process to the newly allocated memory space; Other read-only component instances continue to share the common resources in the component prototype process.

8. The lightweight processing method for general software according to claim 7, characterized in that: The component's property file is edited by the corresponding form during the development process. The form includes the name, version, cropping mark and dependency relationship; the cropping mark indicates whether the current component can be cropped, and the dependency relationship is used to describe the pre-dependent components and the corresponding dependency level.

Images