Automatic machine learning collaborative optimization method and system based on hardware perception and program product

Abstract

The invention discloses an automatic machine learning collaborative optimization method based on hardware perception. The method is used for design of an integrated learning field programmable gate array accelerator in edge artificial intelligence. The method aims at solving the balance challenge between algorithm performance and hardware resource consumption. The method comprises the following steps: defining a joint optimization space containing algorithm hyper-parameters and FPGA hardware configuration parameters; a collaborative optimization objective function is established, the function is determined by algorithm performance indexes and predicted FPGA hardware resource occupancy, and resource prediction quantifies hardware influence according to algorithm hyper-parameters; and searching according to the objective function in the joint optimization space by utilizing an AutoML optimization algorithm so as to identify the optimal configuration of the balance performance and resources. According to the method, the problems of low resource efficiency and high development cost caused by separate optimization of an algorithm and hardware in the prior art are effectively solved, balanced design is realized, automation is enhanced, and complexity is reduced.

Description

Technical Field

[0001] The present invention relates to the field of edge artificial intelligence technology, and in particular to a hardware-aware automatic machine learning collaborative optimization method, system, and program product for the design of integrated learning FPGA accelerators in edge artificial intelligence, aiming to solve the balance problem between algorithm performance and hardware resource consumption. Background Art

[0002] In the field of edge artificial intelligence, deploying ensemble learning models on hardware accelerators such as FPGAs to achieve high performance and low power consumption is an important research direction. However, existing technologies face many challenges in designing such accelerators.

[0003] On the one hand, algorithm optimization and hardware design are usually carried out independently, resulting in low resource efficiency. If you only focus on algorithm performance optimization, you may ignore compatibility issues when the model is deployed on hardware, such as insufficient parallelism, or it may be difficult to take into account the optimization of hardware resources. On the contrary, if you only focus on optimizing hardware resources, it may lead to degraded model performance. Therefore, how to strike a balance between algorithm performance and hardware resource usage is a difficult problem. On the other hand, the degree of automation is insufficient and the development cost is high. Although the existing AutoML-driven algorithm design method can realize automatic optimization of algorithm hyperparameters, manual intervention in hardware configuration is still required during the deployment process, and parameters such as hardware bit width and parallelism are manually adjusted. The adjustment process often relies on empirical settings, and a single optimization takes a long time. In addition, the existing technology lacks the ability to automatically generate HLS code and produce IP cores, which further increases the complexity and cost of development.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0005] In view of this, the present invention provides a hardware-aware automatic machine learning collaborative optimization method for the design of integrated learning FPGA accelerators in edge artificial intelligence, aiming to solve the problem of difficult balance between algorithm performance and hardware resource consumption in the existing technology, reduce manual intervention and development costs, and realize algorithm-hardware collaborative optimization and full-process automation.

[0006] An embodiment of the present invention provides a hardware-aware automated machine learning collaborative optimization method for designing an integrated learning FPGA accelerator in edge artificial intelligence, comprising the following steps:

[0007] Construct a joint optimization space that includes the algorithm hyperparameters of the ensemble learning model and the target FPGA hardware configuration parameters;

[0008] defining a collaborative optimization objective function, where the objective function is determined based on an algorithm performance evaluation metric of the ensemble learning model and an expected FPGA hardware resource usage, wherein the expected FPGA hardware resource usage is derived based on predictions of at least some algorithm hyperparameters in the joint optimization space to quantify the impact of the algorithm hyperparameters on hardware resource consumption;

[0009] An automatic machine learning optimization algorithm is used to search within the joint optimization space based on the collaborative optimization objective function. The objective function guides the search process towards a configuration that takes into account both the algorithm performance evaluation indicators and the expected FPGA hardware resource usage, so as to determine an optimal configuration. The optimal configuration includes a set of algorithm hyperparameters and hardware configuration parameters.

[0010] In some optional embodiments, the algorithm hyperparameters include at least the number of trees (ne) and the maximum depth (d) of the ensemble learning model, which directly affect the computational complexity of the model and the potential hardware implementation scale.

[0011] In some optional embodiments, the hardware configuration parameters include at least data bit width and parallelism factor (PF), which directly affect resource consumption and computing speed of hardware implementation.

[0012] In some optional embodiments, the parallelism factor (PF) is a high-level synthesis (HLS) loop unrolling factor applied to the core processing loop of the ensemble learning model.

[0013] In some optional embodiments, the expected FPGA hardware resource usage is configured to be calculated using a pre-established hardware resource usage prediction model (SHR), which receives algorithm hyperparameters as input and establishes a quantitative relationship between algorithm complexity and hardware resource consumption.

[0014] In some optional embodiments, the hardware resource occupancy prediction model (SHR) is pre-established by the following steps:

[0015] Obtain the actual hardware resource usage data of the ensemble learning model deployed on the target FPGA hardware platform when using various combination parameters;

[0016] The SHR model is obtained by performing function fitting based on the actual hardware resource usage data and the corresponding algorithm hyperparameters.

[0017] In some optional embodiments, the functional form of the SHR model includes terms related to the number of trees (ne) and terms related to an exponential function (2^d) of the number of trees (ne) and the maximum depth (d). This specific functional form reflects the nonlinear growth relationship between the structural characteristics of the integrated learning model (the number and depth of trees) and the hardware resources occupied (especially the resources related to the decision logic).

[0018] In some optional embodiments, the expected FPGA hardware resource occupancy or actual hardware resource occupancy data is configured as the occupancy of a lookup table (LUT). The lookup table (LUT) is a key resource for implementing combinational logic in the FPGA, and its occupancy directly reflects the mapping cost of the algorithm logic complexity on the hardware.

[0019] In some optional embodiments, the collaborative optimization objective function (Lcod) is determined by transforming the algorithm performance evaluation index (AUC) (for example, taking a negative value) and performing a weighted summation with the normalized expected FPGA hardware resource occupancy (N(SHR)). This weighted summation method explicitly unifies the two mutually restrictive factors of algorithm performance and predicted hardware cost into a single goal to guide the optimization process to achieve balance.

[0020] In some optional embodiments, the automated machine learning optimization algorithm employs a Bayesian Optimization Over Band (BOHB) algorithm to efficiently explore and converge to the optimal configuration in the joint optimization space.

[0021] In some optional embodiments, before constructing the joint optimization space, a step of screening the input features of the integrated learning model is also included to reduce the model complexity and potential resource requirements for subsequent hardware implementation.

[0022] The screening steps include:

[0023] Select the first feature subset using a method based on model feature importance scoring;

[0024] A recursive feature elimination (RFE) method is used to select a second feature subset from the first feature subset according to a preset performance index evaluation criterion, where the number of features in the second feature subset is less than or equal to that in the first feature subset.

[0025] In some optional embodiments, the method further includes a step of automatically generating hardware description language code for configuring the target FPGA hardware platform based on the algorithm hyperparameters and hardware configuration parameters determined in the optimal configuration, thereby achieving an automated transition from optimized configuration to hardware implementation.

[0026] The step of automatically generating hardware description language code is completed using a high-level synthesis (HLS) tool.

[0027] In some optional embodiments, a step of generating a bitstream file based on the hardware description language code is further included to complete the final FPGA deployment.

[0028] An embodiment of the present invention further provides a hardware-aware automatic machine learning collaborative optimization system for implementing the aforementioned hardware-aware automatic machine learning collaborative optimization method, comprising:

[0029] a parameter space management module configured to store and manage a joint optimization space containing algorithm hyperparameters of the ensemble learning model and target FPGA hardware configuration parameters;

[0030] a hardware resource prediction module configured to receive at least a portion of the algorithm hyperparameters as input and output a corresponding expected FPGA hardware resource usage based on a pre-established hardware resource usage prediction model (SHR);

[0031] An objective function processing module, connected to the hardware resource prediction module, configured to receive the expected FPGA hardware resource usage and calculate a value of a collaborative optimization objective function in combination with an algorithm performance evaluation indicator of the ensemble learning model;

[0032] The automated machine learning optimization engine, connected to the parameter space management module and the objective function processing unit, is configured as follows:

[0033] Select candidate combinations of algorithm hyperparameters and hardware configuration parameters in the joint optimization space;

[0034] Providing the algorithm hyperparameters in the candidate combination to the hardware resource prediction module;

[0035] receiving a collaborative optimization objective function value calculated by an objective function processing unit;

[0036] The subsequent parameter combination selection is guided by the collaborative optimization objective function value until an optimal configuration is determined. The optimal configuration includes a set of algorithm hyperparameters and hardware configuration parameters.

[0037] An embodiment of the present invention provides a hardware-aware automatic machine learning collaborative optimization program product, which includes computer instructions that, when executed by a processor, implement the steps of the above-mentioned hardware-aware automatic machine learning collaborative optimization method.

[0038] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

[0039] The hardware-aware automatic machine learning collaborative optimization method, system, and program product of the present invention have the following beneficial effects:

[0040] In edge artificial intelligence applications, the present invention usually needs to implement integrated learning models on resource-constrained hardware platforms such as FPGAs. By establishing a quantitative relationship between algorithm complexity and hardware resource consumption, and unifying algorithm performance and hardware cost into a single goal, the optimization process can be guided to achieve a balance, and ultimately a solution that takes into account both algorithm performance and hardware efficiency can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Other features, objects and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0042] Figure 1 is a flowchart of a hardware-aware automatic machine learning collaborative optimization method according to an embodiment of the present invention;

[0043] Figure 2 1 is a schematic diagram of the structure of a hardware-aware automatic machine learning collaborative optimization system according to an embodiment of the present invention;

[0044] Figure 3 1. It is a schematic diagram of a collaborative optimization algorithm framework of a hardware-aware automatic machine learning collaborative optimization system according to an embodiment of the present invention;

[0045] Figure 4 This is an AutoML-driven hyperparameter search process in a hardware-aware automatic machine learning collaborative optimization method according to an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of automatically generating IP cores in a hardware-aware automatic machine learning collaborative optimization method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0048] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0049] The flowcharts shown in the accompanying drawings are merely exemplary and do not necessarily include all steps. For example, some steps may be decomposed, while some steps may be combined or partially combined. Therefore, the actual execution order may change according to actual circumstances.

[0050] The present invention provides a hardware-aware automated machine learning collaborative optimization method. The method first constructs a joint optimization space containing the algorithm hyperparameters of an ensemble learning model and the target FPGA hardware configuration parameters. Then, a collaborative optimization objective function based on the algorithm performance evaluation index of the ensemble learning model and the expected FPGA hardware resource usage is defined. Finally, an automated machine learning optimization algorithm is used to search within the joint optimization space based on the collaborative optimization objective function. The objective function guides the search process toward a configuration that takes into account both the algorithm performance evaluation index and the expected FPGA hardware resource usage to determine the optimal configuration. The optimal configuration includes a set of algorithm hyperparameters and hardware configuration parameters. The present invention can effectively solve the problem of the difficulty in balancing algorithm performance and hardware resource consumption in the prior art, reduce manual intervention and development costs, and achieve algorithm-hardware collaborative optimization and full-process automation. Specifically, by constructing a joint optimization space, the present invention can closely integrate algorithm design and hardware implementation, avoiding the performance bottlenecks and resource waste caused by independent optimization of algorithms and hardware in traditional methods. By defining the collaborative optimization objective function, the present invention can unify the two mutually restrictive factors of algorithm performance and hardware resource consumption into a single goal, thereby guiding the optimization process to find a balance point. By employing an automated machine learning optimization algorithm, the present invention can efficiently search for the optimal combination of algorithm hyperparameters and hardware configuration parameters within a high-dimensional joint optimization space, reducing the need for manual parameter tuning and improving optimization efficiency. Therefore, the present invention can achieve a dual improvement in algorithm performance and hardware efficiency, and while meeting hardware resource constraints, it maximizes algorithm performance, providing an effective solution for the deployment of edge AI applications.

[0051] like Figure 1 As shown, the present invention provides a hardware-aware automatic machine learning collaborative optimization method for the design of integrated learning FPGA accelerators in edge artificial intelligence, aiming to solve the balance problem between algorithm performance and hardware resource consumption, including the following steps:

[0052] P100. Construct a joint optimization space that includes the algorithm hyperparameters of the ensemble learning model and the target FPGA hardware configuration parameters. Ensemble learning models, as an effective machine learning method, construct a strong learner by combining multiple weak learners and can be applied to edge AI applications. Algorithm hyperparameters, such as the number of trees and maximum depth, directly affect the complexity and performance of the ensemble learning model. Target FPGA hardware configuration parameters, such as data bit width and parallelism factor, determine the model's resource consumption and computational speed on the FPGA. Constructing a joint optimization space for algorithm hyperparameters and hardware configuration parameters aims to synergize algorithm design and hardware implementation, avoiding performance bottlenecks and resource waste caused by independent optimization of algorithms and hardware in traditional methods. In this embodiment of the present invention, the ensemble learning model can be, but is not limited to, XGBoost or EBM, and can also be any other ensemble learning algorithm suitable for FPGA acceleration. Algorithm hyperparameters can include the number of trees, maximum depth, learning rate, and regularization coefficient. The value ranges of these parameters can be adjusted based on the specific application and dataset. Hardware configuration parameters can include FPGA bit widths (e.g., 8 / 16 / 32 bits) and parallelism. The choice of these parameters directly impacts hardware resource utilization and model inference speed. This joint optimization space can be freely defined based on actual needs, encompassing a wider range of algorithms and hyperparameters. However, its core principle is to place algorithms and hardware parameters within the same space for joint optimization.

[0053] By constructing a joint optimization space for algorithm hyperparameters and hardware configuration parameters, the present invention tightly integrates algorithm design and hardware implementation, overcoming the performance bottlenecks and resource waste caused by independent optimization of algorithms and hardware in traditional approaches. Furthermore, the design of this joint optimization space provides a foundation for the subsequent definition of collaborative optimization objective functions and the search for optimal configurations. This allows the optimization process to simultaneously consider algorithm performance and hardware resource constraints, ultimately achieving the optimal configuration that maximizes algorithm performance while satisfying hardware resource constraints.

[0054] P200. Define a collaborative optimization objective function. The objective function is determined based on the algorithm performance evaluation indicators of the ensemble learning model and the expected FPGA hardware resource usage. The algorithm performance evaluation indicators are used to quantify the prediction accuracy or classification ability of the ensemble learning model, while the expected FPGA hardware resource usage reflects the hardware resource cost required to implement the model on the FPGA. The collaborative optimization objective function is designed to unify the two mutually constraining factors of algorithm performance and hardware resource consumption into a single goal, thereby guiding the optimization process to find a balance point, that is, to maximize algorithm performance while satisfying hardware resource constraints. In this embodiment of the present invention, the algorithm performance evaluation indicators may be, but are not limited to, AUC (Area Under Curve), accuracy, F1-score, etc. Appropriate evaluation indicators can be selected based on the specific application scenario and dataset. The expected FPGA hardware resource usage can be predicted using a hardware resource usage prediction model, namely, SHR. This model accepts algorithm hyperparameters as input and outputs corresponding hardware resource usage estimates, such as the number of LUTs (Look-Up Tables) and the number of Flip-Flops (FFs). The specific form of the collaborative optimization objective function can be adjusted based on actual needs. For example, a weighted summation approach can be used to combine the algorithm performance evaluation metric and the hardware resource usage, where the weights can be set based on the user's different preferences for algorithm performance and hardware resources. Furthermore, the hardware resource usage can be normalized to eliminate dimensional differences and make it comparable with the algorithm performance evaluation metric. For example, the objective function Lcod can be defined as: Lcod = w1*(-AUC)+w2*N(SHR), where AUC is the algorithm performance evaluation metric, SHR is the expected FPGA hardware resource usage, N(SHR) is the normalized hardware resource usage, and w1 and w2 are weights used to adjust the balance between algorithm performance and hardware resources. This objective function represents algorithm performance through a negative AUC because the goal of the optimization algorithm is to minimize the objective function, and improving the AUC is the desired outcome.

[0055] By defining a collaborative optimization objective function based on algorithm performance evaluation indicators and expected FPGA hardware resource usage, the present invention can unify the two mutually constraining factors of algorithm performance and hardware resource consumption into one goal, thereby guiding the optimization process to find a balance point. This collaborative optimization method can avoid the performance bottlenecks and resource waste problems caused by independent optimization of algorithms and hardware in traditional methods, and achieve a dual improvement in algorithm performance and hardware efficiency. In addition, by adjusting the weights in the objective function, users can flexibly adjust the balance between algorithm performance and hardware resources according to their own needs to meet different application scenarios and hardware platform constraints.

[0056] P300, using an automated machine learning optimization algorithm, searches within the joint optimization space based on a collaborative optimization objective function. The objective function guides the search process toward a configuration that balances algorithm performance evaluation indicators and expected FPGA hardware resource usage to determine the optimal configuration. The optimal configuration includes a set of algorithm hyperparameters and hardware configuration parameters. An automated machine learning optimization algorithm is a technology that can automatically search and optimize model hyperparameters. It can effectively improve model performance and reduce the need for manual parameter adjustment. In the present invention, an automated machine learning optimization algorithm is used to search for the optimal combination of algorithm hyperparameters and hardware configuration parameters within the joint optimization space to achieve a balance between algorithm performance and hardware resource consumption. The collaborative optimization objective function serves as a guide for the search, guiding the optimization process toward a configuration that balances algorithm performance evaluation indicators and expected FPGA hardware resource usage. In an embodiment of the present invention, the automated machine learning optimization algorithm can use the Bayesian optimization hyperband algorithm (BOHB), but is not limited thereto. Any other optimization algorithm suitable for high-dimensional space search, such as TPE (Tree-structured Parzen Estimator) and SMAC (Sequential Model-based Algorithm Configuration), can also be used. The BOHB algorithm combines the advantages of Bayesian optimization and the hyperband algorithm, effectively exploring and converging to the optimal configuration in high-dimensional space. During the search process, the BOHB algorithm builds a probabilistic model based on historical search results to predict the performance of different parameter combinations and selects the next parameter combination to evaluate based on this model. To improve search efficiency, the BOHB algorithm also utilizes the hyperband algorithm, which evaluates multiple parameter combinations in parallel during each iteration and dynamically adjusts the search strategy based on the evaluation results. The optimal configuration is the combination of algorithm hyperparameters and hardware configuration parameters within the joint optimization space that enables the collaborative optimization objective function to achieve the optimal value. This optimal configuration includes the algorithm hyperparameters of the ensemble learning model, such as the number of trees and maximum depth, and the target FPGA hardware configuration parameters, such as data bit width and parallelism factor. The values of these parameters jointly determine the model's performance and resource consumption on the FPGA.

[0057] By adopting an automatic machine learning optimization algorithm, the present invention can efficiently search for the optimal combination of algorithm hyperparameters and hardware configuration parameters in a high-dimensional joint optimization space, thereby achieving a balance between algorithm performance and hardware resource consumption. This automated search method can not only improve optimization efficiency, but also reduce the need for manual parameter adjustment and reduce development costs. In addition, due to the guidance of the collaborative optimization objective function, the search process can proceed in the direction of taking into account both algorithm performance and hardware resources, thereby obtaining the optimal configuration that maximizes algorithm performance while meeting hardware resource constraints.

[0058] In some embodiments, algorithm hyperparameters include the number of trees (ne) and maximum depth (d) of the ensemble learning model. The number of trees (ne) represents the number of weak learners in the ensemble learning model, such as the number of decision trees in XGBoost. The maximum depth (d) represents the maximum number of layers in each decision tree. The number of trees and maximum depth are key factors affecting model complexity, directly affecting the model's predictive and generalization capabilities. At the same time, these two parameters also directly affect the scale of hardware implementation. For example, a larger number of trees results in more computing units and storage units, while a larger maximum depth results in more complex decision logic. In this embodiment, the value range of the number of trees can be set to [50, 100, 150, 200], and the value range of the maximum depth can be set to [3, 4, 5, 6]. These value ranges can be adjusted according to the specific application and dataset. For example, for complex datasets, the number of trees and maximum depth can be appropriately increased to improve model performance; for resource-constrained hardware platforms, the number of trees and maximum depth can be appropriately reduced to reduce hardware resource consumption. In addition to the number of trees and the maximum depth, other algorithm hyperparameters such as learning rate and regularization coefficient can also be included. The choice of these parameters will also affect model performance and hardware implementation.

[0059] By optimizing two key algorithm hyperparameters—the number of trees and maximum depth—this paper effectively controls the complexity of ensemble learning models and strikes a balance between algorithm performance and hardware resource consumption. Through collaborative optimization, we can find the optimal number of trees and maximum depth configuration that maximizes algorithm performance while satisfying hardware resource constraints, thereby enabling energy-efficient edge computing.

[0060] In some embodiments, hardware configuration parameters include data bit width and parallelism factor (PF). Data bit width refers to the width of the data type used in the hardware implementation, such as 8 bits, 16 bits, or 32 bits. Data bit width directly affects the accuracy and resource consumption of the hardware implementation. A smaller data bit width can reduce hardware resource consumption but may result in a loss of accuracy; a larger data bit width can improve accuracy but increase hardware resource consumption. The parallelism factor (PF) refers to the degree to which data is processed in parallel in the hardware implementation. A higher degree of parallelism can increase computing speed but increase hardware resource consumption; a lower degree of parallelism can reduce hardware resource consumption but reduce computing speed. In this embodiment, the data bit width can be set to a value range of [8, 16, 32] bits, and the parallelism factor (PF) can be set to a value range of [1, 2, 4, 8]. These value ranges can be adjusted based on the specific hardware platform and application requirements. For example, for resource-constrained edge devices, a smaller data bit width and lower degree of parallelism can be used to reduce hardware resource consumption; for applications requiring high performance, a larger data bit width and higher degree of parallelism can be used to increase computing speed.

[0061] By optimizing two key hardware configuration parameters, data width and parallelism, this paper effectively controls the hardware resource consumption and computing speed of the FPGA accelerator, achieving a balance between the two. Through collaborative optimization, the optimal data width and parallelism configuration can be found to maximize computing speed while satisfying hardware resource constraints, thereby achieving energy-efficient edge computing.

[0062] In some embodiments, the parallelism factor (PF) is a high-level synthesis (HLS) loop expansion factor applied to the core processing loop of the ensemble learning model. High-level synthesis (HLS) is a technology that converts high-level programming languages such as C / C++ into hardware description languages such as VHDL or Verilog. Through HLS, algorithm descriptions can be easily converted into hardware implementations without having to manually write complex hardware code. Loop expansion is a commonly used HLS optimization technique that can increase computing speed by copying the loop body multiple times and executing it in parallel. In the present invention, the parallelism factor (PF) is used as an HLS loop expansion factor and applied to the core processing loop of the ensemble learning model, such as the traversal loop of a decision tree. For example, if the parallelism factor (PF) is set to 4, the core processing loop will be expanded 4 times, allowing 4 data to be processed in parallel. This parallel processing method can significantly improve computing speed, but it also increases hardware resource consumption. By optimizing the parallelism factor as a hardware configuration parameter, the present invention can effectively control the hardware resource consumption and computing speed of the FPGA accelerator and achieve a balance between the two.

[0063] By combining the parallelism factor with HLS technology, this paper can easily accelerate ensemble learning models on FPGAs and effectively control hardware resource consumption and computing speed. This approach not only improves development efficiency but also enables energy-efficient edge computing.

[0064] In some embodiments, the expected FPGA hardware resource usage is configured to be calculated by a pre-established hardware resource usage prediction model (SHR). The SHR model receives algorithm hyperparameters as input and establishes a quantitative relationship between algorithm complexity and hardware resource consumption. The SHR model is a proxy model used to quickly predict hardware resource usage during algorithm hyperparameter optimization without the need for complete hardware synthesis every time. This can significantly improve optimization efficiency because hardware synthesis is a time-consuming process. In this embodiment, the SHR model can be established in a variety of ways, such as based on regression analysis, neural networks and other methods. The input of the SHR model can be algorithm hyperparameters, such as the number of trees, maximum depth, etc., and the output can be FPGA hardware resource usage, such as the number of LUTs, the number of FFs, etc. The functional form of the SHR model can be adjusted according to the specific application and hardware platform. For example, for the XGBoost model, the following form of SHR model can be used:

[0065] SHR=a*ne+b*(2^d)+c

[0066] Among them, ne is the number of trees, d is the maximum depth, and a, b, and c are model parameters that can be obtained by fitting experimental data.

[0067] By introducing the SHR model, the present invention can quickly predict hardware resource usage during algorithm hyperparameter optimization, thereby avoiding the time-consuming hardware synthesis process and significantly improving optimization efficiency. Furthermore, the SHR model can establish a quantitative relationship between algorithm complexity and hardware resource consumption, providing an important basis for collaborative optimization.

[0068] In some embodiments, the hardware resource occupancy prediction model (SHR) is pre-established by the following steps:

[0069] Obtain actual hardware resource usage data. First, it is necessary to obtain the actual hardware resource usage data of the ensemble learning model when deployed on the target FPGA hardware platform using various parameter combinations. This can be done by setting a grid-like algorithm hyperparameter configuration, converting the model into HLS code using the Conifer toolchain, and then using FPGA development tools such as Vivado for hardware synthesis to obtain the actual hardware resource usage data. For example, the number of trees (ne) can be set to [50, 100, 150, 200] and the maximum depth (d) to [3, 4, 5, 6]. Then, all possible parameter combinations (4x4 = 16) can be hardware synthesized separately, and hardware resource usage data such as the number of LUTs and the number of FFs can be recorded.

[0070] Function fitting. Then, based on the actual hardware resource usage data and the corresponding algorithm hyperparameters, a function fitting is performed to obtain the SHR model. Regression analysis methods such as nonlinear least squares can be used to fit the actual hardware resource usage data as the dependent variable and the algorithm hyperparameters as the independent variables to obtain the parameters of the SHR model. For example, the following SHR model can be used:

[0071] SHR=a*ne+b*(2^d)+c

[0072] Then, the nonlinear least squares method is used to fit the values of parameters a, b, and c based on the actual hardware resource occupancy data and the corresponding number of trees (ne) and maximum depth (d).

[0073] Through the above steps, an SHR model that can accurately predict hardware resource usage can be established in advance, providing an important basis for subsequent algorithm hyperparameter optimization.

[0074] In some embodiments, the functional form of the SHR model includes a term related to the number of trees (ne) and a term related to an exponential function (2^d) of the number of trees (ne) and the maximum depth (d). This particular functional form reflects the nonlinear growth relationship between the structural characteristics of the ensemble learning model (the number of trees and the depth) and the hardware resources (particularly the resources related to the decision logic). For example, the following SHR model can be used:

[0075] SHR=a*ne+b*(2^d)+c

[0076] The first term (a*ne) indicates that hardware resource usage is linearly proportional to the number of trees, while the second term (b*(2^d)) indicates that hardware resource usage is exponentially proportional to the maximum depth, reflecting that increasing the number of decision tree layers leads to an exponential increase in the complexity of the decision logic. The third term (c) represents a constant term, representing a fixed hardware resource usage. This specific function form enables more accurate predictions of hardware resource usage, providing a more reliable basis for optimizing algorithm hyperparameters.

[0077] By adopting this specific function form, the present invention can more accurately predict hardware resource usage, thereby providing a more reliable basis for optimizing algorithm hyperparameters. This helps find the optimal configuration that maximizes algorithm performance while meeting hardware resource constraints, thus achieving energy-efficient edge computing.

[0078] In some embodiments, the expected FPGA hardware resource occupancy or actual hardware resource occupancy data is configured as the occupancy of a lookup table (LUT). The lookup table (LUT) is a key resource for implementing combinational logic in the FPGA, and its occupancy directly reflects the mapping cost of the algorithm logic complexity on the hardware. LUT is a programmable logic unit that can implement various Boolean functions. In FPGA, complex combinational logic is usually composed of multiple LUTs. Therefore, the occupancy of the LUT can directly reflect the logic complexity of the algorithm and affect the resource utilization and performance of the FPGA. In this embodiment, the occupancy of the LUT is used as the output of the SHR model to predict the hardware resource consumption under different algorithm hyperparameter configurations. By optimizing the algorithm hyperparameters, the occupancy of the LUT can be reduced, thereby reducing the resource consumption of the FPGA and improving resource utilization.

[0079] By using LUT occupancy as a metric for hardware resource occupancy, the present invention can more accurately reflect the mapping cost of algorithm logic complexity on hardware, effectively reduce FPGA resource consumption, improve resource utilization, and achieve energy-efficient edge computing.

[0080] In some embodiments, the collaborative optimization objective function (Lcod) is determined by transforming the algorithm performance evaluation metric (AUC) (e.g., negating it) and performing a weighted summation with the normalized expected FPGA hardware resource usage (N(SHR)). This weighted summation explicitly unifies the two mutually constraining factors, algorithm performance and predicted hardware cost, into a single objective, guiding the optimization process to achieve a balance. AUC (Area Under Curve) is a commonly used algorithm performance evaluation metric used to measure the predictive ability of classification models. The AUC ranges from [0, 1], with a larger AUC indicating a stronger model's predictive ability. Since the optimization algorithm's goal is to minimize the objective function, the AUC needs to be transformed, for example, by taking a negative value, so that it aligns with the hardware resource usage in the same optimization direction. The expected FPGA hardware resource usage (SHR) represents the predicted hardware resource consumption, such as the number of LUTs. Since the hardware resource usage may have a different numerical range from the AUC, it needs to be normalized (N(SHR)) to a range of [0, 1] to eliminate dimensional differences. Weighted summation refers to the weighted combination of the transformed AUC and the normalized SHR, where the weight is used to adjust the balance between algorithm performance and hardware resources. For example, the following form of collaborative optimization objective function can be used:

[0081] Lcod=w1*(-AUC)+w2*N(SHR)

[0082] Where w1 and w2 are weights used to adjust the balance between algorithm performance and hardware resources. A larger w1 indicates a greater emphasis on algorithm performance, while a smaller w1 indicates a greater emphasis on hardware resource consumption.

[0083] By adopting this weighted summation approach, the present invention can unify the two mutually constraining factors of algorithm performance and predicted hardware cost into a single goal, thereby guiding the optimization process to achieve balance and find the optimal configuration that maximizes algorithm performance while satisfying hardware resource constraints.

[0084] In some embodiments, the automatic machine learning optimization algorithm adopts the Bayesian optimization super-band algorithm (BOHB) to efficiently explore and converge to the optimal configuration in the joint optimization space. Bayesian optimization is a model-based optimization method that can effectively predict the performance of different parameter combinations and guide subsequent parameter selection by establishing a probabilistic model of the objective function. The BOHB algorithm combines the advantages of Bayesian optimization and super-band algorithms, and can effectively explore and converge to the optimal configuration in high-dimensional space. Specifically, the BOHB algorithm first uses a random sampling method to initialize a part of parameter combinations, and then uses the Bayesian optimization method to establish a probabilistic model of the objective function, and selects the next parameter combination to be evaluated based on the model. In order to improve the search efficiency, the BOHB algorithm also adopts the super-band algorithm, that is, in each iteration, multiple parameter combinations are evaluated in parallel, and the search strategy is dynamically adjusted according to the evaluation results.

[0085] By adopting the BOHB algorithm, the present invention can efficiently explore and converge to the optimal configuration in the joint optimization space, thereby achieving a balance between algorithm performance and hardware resource consumption, and finding the optimal configuration that maximizes algorithm performance while satisfying hardware resource constraints.

[0086] In some embodiments, before step P100 of constructing the joint optimization space, a step of screening the input features of the ensemble learning model is also included to reduce the model complexity and potential resource requirements for subsequent hardware implementation. Feature screening is a commonly used data preprocessing technique that can select the most relevant features from the original features, thereby reducing the complexity of the model, improving the generalization ability of the model, and reducing the resource requirements for hardware implementation. In this embodiment, the feature screening step includes:

[0087] Methods based on model feature importance scores. This method uses a method based on model feature importance scores to select the first feature subset. You can use an ensemble learning model like XGBoost to train a base model. Then, based on the feature importance scores output by the model, select the most important features as the first feature subset. This method can quickly remove irrelevant features and reduce model complexity.

[0088] Recursive Feature Elimination (RFE) method. Using the recursive feature elimination (RFE) method, a second feature subset is selected from the first feature subset based on a preset performance metric evaluation criterion, where the number of features in the second feature subset is less than or equal to that in the first feature subset. RFE is an iterative feature selection method that removes the least important feature at each iteration and evaluates the performance of the model based on a performance metric. Through multiple iterations, the optimal feature subset can be selected. Performance metric evaluation criteria can include AUC, accuracy, etc., and the appropriate evaluation criterion can be selected based on the specific application.

[0089] By adopting this two-stage feature screening method, the present invention can effectively reduce the complexity of the model and the potential resource requirements of subsequent hardware implementation, and improve the generalization ability of the model.

[0090] In some embodiments, the hardware-aware automatic machine learning collaborative optimization method also includes a step of automatically generating a hardware description language code for configuring the target FPGA hardware platform based on the algorithm hyperparameters and hardware configuration parameters determined in the optimal configuration, thereby realizing the automated connection from optimized configuration to hardware implementation. The step of automatically generating the hardware description language code is completed using a high-level synthesis (HLS) tool. The HLS tool can convert a high-level programming language (such as C / C++) into a hardware description language (such as VHDL or Verilog), thereby conveniently converting the algorithm description into a hardware implementation without manually writing complex hardware code. In this embodiment, based on the algorithm hyperparameters and hardware configuration parameters determined in the optimal configuration, HLS code can be automatically generated, such as setting the loop expansion factor, data bit width, etc. Then, the HLS tool can be used to convert the HLS code into a hardware description language code for configuring the target FPGA hardware platform.

[0091] By adopting the HLS tool, the present invention can achieve automatic connection from optimized configuration to hardware implementation, thereby greatly improving development efficiency and reducing development costs.

[0092] In some embodiments, the hardware-aware automatic machine learning collaborative optimization method also includes the step of generating a bitstream file based on the hardware description language code to complete the final FPGA deployment. The bitstream file is a binary file used to configure the FPGA hardware platform, which contains the configuration information of the FPGA, such as the connection method of the logic unit, the initialization data of the storage unit, etc. In this embodiment, FPGA development tools such as Vivado can be used to perform steps such as synthesis, layout, and routing on the hardware description language code to finally generate a bitstream file. Then, the bitstream file can be loaded into the FPGA hardware platform using interfaces such as JTAG to complete the final FPGA deployment.

[0093] By generating a bitstream file and completing FPGA deployment, the present invention can deploy the integrated learning model that has undergone algorithm-hardware co-optimization to the edge device, achieving high-performance, low-power edge computing.

[0094] like Figure 2 As shown, an embodiment of the present invention further provides a hardware-aware automatic machine learning collaborative optimization system for implementing the above-mentioned hardware-aware automatic machine learning collaborative optimization method, the system comprising:

[0095] The parameter space management module M100 is used to store and manage the joint optimization space, which contains the algorithm hyperparameters of the integrated learning model and the target FPGA hardware configuration parameters.

[0096] The hardware resource prediction module M200 is configured to receive at least a portion of the algorithm hyperparameters as input, and output the corresponding expected FPGA hardware resource occupancy based on a pre-established hardware resource occupancy prediction model (SHR).

[0097] The objective function processing module M300 is connected to the hardware resource prediction module M200 and is configured to receive the expected FPGA hardware resource usage and calculate the value of the collaborative optimization objective function in combination with the algorithm performance evaluation index of the integrated learning model.

[0098] The automated machine learning optimization engine M400 is connected to the parameter space management module M100 and the objective function processing module M300 and is configured as follows:

[0099] Select candidate combinations of algorithm hyperparameters and hardware configuration parameters in the joint optimization space;

[0100] Providing the algorithm hyperparameters in the candidate combination to the hardware resource prediction module M200;

[0101] Receiving the collaborative optimization objective function value calculated by the objective function processing module M300;

[0102] The subsequent parameter combination selection is guided by the collaborative optimization objective function value until an optimal configuration is determined. The optimal configuration includes a set of algorithm hyperparameters and hardware configuration parameters.

[0103] The system can automatically perform algorithm-hardware co-optimization and improve the performance of the algorithm while meeting hardware resource constraints.

[0104] An embodiment of the present invention further provides a hardware-aware automated machine learning collaborative optimization program product, comprising computer instructions that, when executed by a processor, implement the steps of the hardware-aware automated machine learning collaborative optimization method described above. The program product can be stored on a computer-readable storage medium, such as a hard disk, optical disk, or USB flash drive, and can be executed by a computer or other processor to implement the technical solution of the present invention.

[0105] The present invention also provides another implementation of a hardware-aware automated machine learning collaborative optimization method and system. It should be noted that the following examples are based on the open source dataset FHWA NGSIM, and the FPGA development version is implemented on the PYNQ-Z2. An XGBoost accelerator running on the PYNQ-Z2 FPGA development board is designed. The selection of the dataset and FPGA development board model is only used to illustrate the technical solution of the present invention and does not constitute a limitation on the scope of protection of the present invention. This is achieved by the following steps:

[0106] S100, hardware-aware joint optimization space construction.

[0107] S110, algorithm-hardware hyperparameter space construction.

[0108] Construct algorithm-hardware hyperparameter joint optimization space S={O ml ,O hw},in:

[0109] Algorithm hyperparameter set O wl :Including ensemble learning algorithm type (XGBoost / EBM), number of trees n e ∈[10,200], maximum depth d∈[3,10], learning rate η∈[0.01,0.3], regularization coefficient λ.

[0110] S120: Hardware resource occupancy feedback modeling.

[0111] The Conifer tool chain is used to convert the algorithm into HLS code and implement FPGA deployment. The Surrogate Hardware Resource (SHR) model is introduced as a proxy for hardware resource occupancy. The SHR model is as follows: SHR = k0×n e +k1×n e ×2 d The model is calibrated using nonlinear least squares and an algorithm-hardware resource dataset (which is constructed by setting a grid-like algorithm hyperparameter configuration and obtaining actual hardware resource consumption using the Conifer toolchain).

[0112] In the subsequent algorithm hyperparameter optimization process, SHR will predict the hardware resource occupancy (such as LUT resources) after FPGA synthesis based on the parameter configuration sampled during the search process, thereby obtaining real-time hardware resource occupancy feedback.

[0113] S200 and BOHB-driven hardware and software collaborative optimization target setting.

[0114] Define the objective function of the BOHB optimization algorithm:

[0115] L cod =-w1·AUC+w2·N(SHR)

[0116] The default settings for weights w1 and w2 are w_1 = 0.9 and w_2 = 0.1, respectively, to prioritize algorithm performance while limiting resource consumption during the optimization process. This parameter can be adjusted dynamically based on user needs to achieve a balance between algorithm performance and hardware resources at varying levels.

[0117] The normalization function N(SHR) maps the hardware resource feedback value to the interval [0,1], thereby eliminating the dimension difference.

[0118] AUC is a commonly used indicator to evaluate the detection performance of an algorithm.

[0119] S300, algorithm-hardware collaborative optimal hyperparameter search.

[0120] S310: Feature screening and compressing feature dimensions.

[0121] Fast feature filtering: Select a basic XGBoost model as a feature selector and use sample data to drive model optimization. After the XGBoost model converges, filter the TopK1 features from the original features based on the feature importance scores output by XGBoost. The purpose of this step is to perform a coarse-grained feature selection.

[0122] Recursive Feature Elimination (RFE): Use RFE to gradually eliminate redundant features. Based on the trade-off between model accuracy and feature quantity, the top K2 key features are finally selected. The selection can be customized here. The default is the number of features obtained within a 0.1% drop in accuracy.

[0123] Note: Rapid feature selection is more efficient and can remove a large number of useless features at one time. Recursive feature elimination is less efficient than rapid feature selection, but it can screen features more accurately. The purpose of setting two solution segments is to improve the efficiency of overall feature selection.

[0124] S320 and AutoML obtain the best hyperparameter configuration.

[0125] Based on the algorithm-hardware coordination search space configured in step S100, the BOHB multi-objective optimization algorithm configured in step S200, and the key feature data processed in step S310, the AutoML optimization algorithm is started to search for the optimal algorithm-hardware hyperparameter configuration to obtain the optimal algorithm-hardware hyperparameter configuration.

[0126] S330, FPGA accelerator automatic generation.

[0127] Model compilation and conversion: Based on the optimal hyperparameters obtained in step S320, the integrated learning model configured with the optimal hyperparameters is converted into synthesizable HLS code through the Conifer tool.

[0128] IP core generation: Use the Vivado tool to synthesize the IP core, then generate a bitstream file based on the IP core and put it on the board to complete the deployment.

[0129] In this example, driven by the AutoML framework, the hyperparameter search problem is modeled as a black-box optimization problem using the BOHB algorithm. Compared to manually adjusting hyperparameters to achieve optimal model performance, this method significantly reduces labor costs and is more efficient. A two-stage feature selection (XGBoost + RFE) approach compresses feature dimensions while ensuring model performance, reducing FPGA data path bandwidth pressure.

[0130] like Figure 3 Figure 2 shows the specific implementation process of an embodiment of the present invention, which aims to automatically design an integrated learning FPGA accelerator for edge AI that strikes a balance between algorithm performance and hardware resource consumption. The framework consists of two core modules: the AutoML-driven hyperparameter optimization module on the left and the hardware automated generation and feedback module (Algo2HLS) on the right.

[0131] Hardware resource usage modeling: Before starting the collaborative optimization process, a hardware resource usage prediction model (SHR) needs to be established in advance. This step selects a series of representative algorithm hyperparameter combinations, such as different numbers of trees ne and maximum depth d, and uses the Algo2HLS process on the right, including model-to-HLS code conversion and FPGA synthesis, to obtain the actual hardware resource data consumed by each combination on the target FPGA platform, such as lookup tables LUTs, triggers FF, etc. Based on these "algorithm hyperparameter-actual resource usage" data pairs, the SHR model such as SHR = k0*ne+k1*ne*2^d+c is obtained through function fitting such as nonlinear least squares method. This SHR model can quickly predict the corresponding hardware resource usage based on the input algorithm hyperparameters (ne, d).

[0132] AutoML-driven collaborative optimization: In the left module, we first construct a joint optimization space based on application requirements such as specific domain models, integrated algorithm model types, and target hardware characteristics, including algorithm hyperparameters such as ne range [10, 200], d range [3, 10], and hardware configuration parameters such as bit width {8, 16, 32} and parallelism factor PF {1, 2, 4, 8}.

[0133] Next, a collaborative optimization objective function is defined. This function is injected into an AutoML optimization engine, such as the BOHB algorithm. This objective function combines two aspects: one is the algorithm performance indicator, such as the AUC value obtained through model evaluation; the other is the hardware resource utilization indicator, such as the predicted LUT value, predicted based on the current candidate algorithm hyperparameters using the SHR model established in the hardware resource utilization modeling. These two indicators are usually unified into a single target value through a weighted summation, such as Lcod = -w1*AUC + w2*N(SHR), where N(SHR) is the normalized resource utilization prediction value.

[0134] The AutoML optimization engine performs an iterative search in the joint optimization space, evaluates each candidate configuration (such as the combination of algorithm hyperparameters and hardware configuration parameters) based on the collaborative optimization objective function, and automatically searches for the configuration that optimizes the objective function, that is, achieves a preset balance between performance and resources, and finally outputs the optimal hyperparameter configuration.

[0135] Automated Hardware Generation: The module on the right receives the optimal hyperparameter configuration output by AutoML. First, this configuration is used to concretize the ensemble learning model. Next, automated tools within the Algo2HLS flow, such as the Conifer toolchain, are used to convert the configured model into high-level synthesis (HLS) code. Finally, FPGA development tools (such as Vivado) are used to synthesize and implement the HLS code, generating the final IP core and bitstream files, completing the design and deployment preparations for the FPGA accelerator.

[0136] like Figure 4 The following figure shows the typical convergence of the AutoML-driven hyperparameter optimization process in an embodiment of the present invention. The horizontal axis of the figure represents the number of iterations of the AutoML optimization algorithm (such as BOHB), and the vertical axis represents the value of the collaborative optimization objective function Lcod evaluated in each iteration.

[0137] As mentioned above, the collaborative optimization objective function Lcod = -w1*AUC + w2*N(SHR) is designed as the target to be minimized.

[0138] The predicted hardware resource usage term N(SHR) has a positive sign (+w2*N(SHR)). Therefore, minimizing Lcod essentially drives the optimization algorithm to find a balanced solution that maximizes algorithm performance (AUC) and minimizes hardware resource usage (SHR). The weight coefficients w1 and w2 (for example, by default w1 = 0.9 and w2 = 0.1) control the emphasis of this balance.

[0139] exist Figure 4 In the figure, each solid dot ("Search Point") represents a specific combination (algorithm hyperparameters, hardware configuration parameters) attempted by AutoML in a particular iteration and its corresponding Lcod evaluation value. As the number of iterations increases, the optimization algorithm tends to explore areas with lower Lcod values. The dotted line ("Cumulative Optimal") shows how the historical optimal Lcod value decreases and stabilizes as the iterations proceed. The figure may also include a regression fit line ("Regression Fit"), which roughly illustrates the overall trend of the search points.

[0140] Finally, the optimization process reaches the preset number of iterations, such as Figure 4 The figure shows about 200 iterations. The specific stopping condition can be set according to the actual situation, or it can be terminated after meeting other convergence criteria. The point marked with "X" ("Optimal Value") represents the hyperparameter configuration corresponding to the best Lcod value found in the entire optimization process. This optimal configuration is the integrated learning FPGA accelerator design scheme finally selected by this method, which achieves the expected balance between algorithm performance and hardware resource consumption and can be used for subsequent automated hardware generation steps. Figure 4 It is intuitively demonstrated that the collaborative optimization method adopted in the present invention can effectively converge in the joint parameter space and find a high-quality equilibrium solution.

[0141] like Figure 5 FIG2 is a schematic diagram showing an architecture of how the integrated learning FPGA accelerator IP core optimized and generated according to the method of the present invention is integrated into a typical Zynq SoC (system on chip) based hardware system, for example, implemented on a PYNQ-Z2 development board.

[0142] System Architecture: This system is centered around the ZYNQ7 Processing System, which includes an ARM processor core and memory controller. It connects various components via the Advanced eXtensible Interface (AXI) interconnect bus, such as axi_mem_intercon and processing_system_0_axi_periph in the figure. The Processor System Reset module provides a system reset signal.

[0143] Accelerator IP core integration: The internal logic structure, data bit width, parallelism factor (PF) of the My_prj_accelerator IP core, and the implemented ensemble learning model (defined by the optimal number of trees ne and depth d) are all determined by the aforementioned AutoML optimization process, aiming to achieve the best balance between algorithm performance and FPGA resource consumption.

[0144] The IP core interacts with the rest of the system through a standard AXI interface:

[0145] Control interface (s_axi_control): Typically an AXI-Lite slave interface connected to AXIInterconnect (processing_system_0_axi_periph). The Zynq processing system (PS) uses this interface to send control instructions (such as start, stop, and configuration parameters) to the accelerator and read status information.

[0146] Memory interface (m_axi_gmem0): Typically an AXI4 master interface connected to the high-bandwidth AXIInterconnect (axi_mem_intercon), which in turn connects to the high-performance (HP) port of the Zynq PS. The accelerator directly accesses the main system memory (DDR) through this interface, efficiently reading input data such as feature vectors to be processed and writing back computation results such as predicted categories or probabilities, achieving high-throughput data processing.

[0147] Clock and reset (ap_clk, ap_rst_n): respectively connected to the clock signal generated by the system clock management module such as FCLK_CLK0 of Zynq PS and the reset signal generated by the reset controller such as ProcessorSystemReset.

[0148] Interrupt signal: When the accelerator completes calculation or a specific event occurs, this output signal can be used to notify the Zynq PS and trigger the corresponding interrupt service routine.

[0149] Automated process manifestation: Figure 5The system integration process shown, especially

[0150] The generation of the My_prj_accelerator IP core and the standardization of its interface benefit from the automated hardware generation steps described in the technical solution of this invention. Using HLS tools and FPGA development tools, the IP core and its packaging can be automatically generated based on the optimized configuration, simplifying its integration into complex SoC designs.

[0151] In summary, Figure 5 The paper demonstrates how the optimized accelerator IP core generated by the method of the present invention is deployed in an actual hardware system, demonstrating that the method can generate an integrable, high-performance, resource-optimized hardware acceleration solution suitable for edge artificial intelligence applications.

[0152] By following these steps, we can obtain an XGBoost accelerator that runs on a PYNQ-Z2 FPGA development board and has undergone algorithm-hardware co-optimization. This accelerator can achieve high vehicle risk prediction accuracy while meeting hardware resource constraints.

[0153] In order to more clearly demonstrate the technical effects of the present invention, the following will be described in conjunction with experimental data. It should be noted that the following experimental data are only examples and do not constitute a limitation on the scope of protection of the present invention.

[0154] Table 1 Comparative experiments of single algorithm optimization and algorithm-hardware collaborative optimization

[0155] Lcod AUC SHR L(SHR) ne d / 0.9793 193445 12.1728 181 6 comparison 0.1778 0.9610 34143 10.4384 36 4

[0156] Using two-stage feature selection (XGBoost+RFE), we can compress feature dimensions and reduce FPGA data path bandwidth requirements while ensuring model performance. We obtain an empirical formula (SHR) for predicting LUT usage through nonlinear least squares fitting, where k0 =

[0157] 210.4, k1 = 11.7, c = 19830, the formula achieved a high coefficient of determination (R2 = 0.9820). SHR's modeling of hardware resource occupancy accurately predicts LUT consumption. As can be seen from Table 1, the optimal solution of collaborative optimization (ne = 30, d = 3) is compared with the single algorithm optimization. After adopting the algorithm-hardware collaborative optimization method of the present invention, LUT resources are reduced by 82.4%, and AUC = 0.9610 (loss ≤ 2%) is maintained. This shows that the present invention can significantly reduce hardware resource consumption while maintaining high algorithm performance, thereby achieving energy-efficient edge computing.

[0158] The experimental data above demonstrates that this invention effectively addresses the existing issue of balancing algorithm performance with hardware resource consumption, achieving algorithm-hardware collaborative optimization and maximizing algorithm performance while meeting hardware resource constraints. Furthermore, this invention automates the entire process, reducing development complexity and costs, providing an effective solution for the deployment of edge AI applications.

[0159] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and do not constitute a limitation on the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made by those skilled in the art within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0160] In summary, the present invention provides a hardware-aware automated machine learning collaborative optimization method, system, and program product. This method achieves a balance between algorithm performance and hardware resource consumption, while reducing development complexity and cost, by constructing a joint optimization space, defining a collaborative optimization objective function, and employing an automated machine learning optimization algorithm. The technical solution of the present invention is applicable to the design of integrated learning FPGA accelerators for edge artificial intelligence. It can effectively address the existing problems of independent optimization of algorithms and hardware, insufficient automation, and other issues, enabling energy-efficient edge computing.

[0161] The foregoing description is merely a specific embodiment of the present invention, and the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be readily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A hardware-aware automated machine learning collaborative optimization method for the design of integrated learning FPGA accelerators in edge artificial intelligence, characterized by: The following steps are involved: Construct a joint optimization space that includes the algorithm hyperparameters of the ensemble learning model and the target FPGA hardware configuration parameters; defining a collaborative optimization objective function, the objective function being determined based on an algorithm performance evaluation metric of the ensemble learning model and an expected FPGA hardware resource usage, wherein the expected FPGA hardware resource usage is predicted based on at least a portion of algorithm hyperparameters in the joint optimization space; An automatic machine learning optimization algorithm is used to search within the joint optimization space based on the collaborative optimization objective function. The objective function is used to guide the search process toward a configuration that takes into account both the algorithm performance evaluation indicators and the expected FPGA hardware resource usage, so as to determine an optimal configuration. The optimal configuration includes a set of algorithm hyperparameters and hardware configuration parameters.

2. The hardware-aware automatic machine learning collaborative optimization method according to claim 1, characterized in that: The algorithm hyperparameters include at least the number of trees and the maximum depth of the integrated learning model.

3. The hardware-aware automatic machine learning collaborative optimization method according to claim 1, characterized in that: The hardware configuration parameters include at least a data bit width and a parallelism factor, where the parallelism factor is a high-level integrated loop expansion factor applied to the core processing loop of the integrated learning model.

4. The hardware-aware automatic machine learning collaborative optimization method according to claim 1, characterized in that: The expected FPGA hardware resource occupancy is configured to be calculated by a pre-established hardware resource occupancy prediction model, which receives the algorithm hyperparameters as input and establishes a quantitative relationship between algorithm complexity and hardware resource consumption.

5. The hardware-aware automatic machine learning collaborative optimization method according to claim 4, characterized in that: The hardware resource occupancy prediction model is pre-established by the following steps: Obtaining actual hardware resource occupancy data of the ensemble learning model when deployed on a target FPGA hardware platform using each combination of parameters; Function fitting is performed based on the actual hardware resource occupancy data and the corresponding algorithm hyperparameters to obtain the hardware resource occupancy prediction model.

6. The hardware-aware automatic machine learning collaborative optimization method according to claim 1, characterized in that: The collaborative optimization objective function is determined by transforming the algorithm performance evaluation index and performing a weighted summation on the index with the normalized expected FPGA hardware resource usage. This weighted summation explicitly unifies the two mutually constraining factors of algorithm performance and predicted hardware cost into a single objective.

7. The hardware-aware automatic machine learning collaborative optimization method according to claim 1, characterized in that: Before constructing the joint optimization space, the step of screening the input features of the ensemble learning model is also included, including: Select the first feature subset using a method based on model feature importance scoring; A recursive feature elimination method is used to select a second feature subset from the first feature subset according to a preset performance indicator evaluation criterion, where the number of features in the second feature subset is less than or equal to that of the first feature subset.

8. The hardware-aware automatic machine learning collaborative optimization method according to claim 1, characterized in that: The method also includes a step of automatically generating hardware description language code for configuring the target FPGA hardware platform based on the algorithm hyperparameters and hardware configuration parameters determined in the optimal configuration, and the step of automatically generating hardware description language code is completed using a high-level synthesis tool.

9. A hardware-aware automatic machine learning collaborative optimization system, used to implement the hardware-aware automatic machine learning collaborative optimization method according to any one of claims 1 to 8, characterized in that: include: a parameter space management module configured to store and manage a joint optimization space, the joint optimization space comprising algorithm hyperparameters of the ensemble learning model and target FPGA hardware configuration parameters; a hardware resource prediction module configured to receive at least a portion of the algorithm hyperparameters as input and output a corresponding expected FPGA hardware resource usage based on a pre-established hardware resource usage prediction model (SHR); an objective function processing module, connected to the hardware resource prediction module, configured to receive the expected FPGA hardware resource usage and calculate a value of a collaborative optimization objective function in combination with the algorithm performance evaluation index of the ensemble learning model; An automated machine learning optimization engine is connected to the parameter space management module and the objective function processing unit to determine an optimal configuration, wherein the optimal configuration includes a set of algorithm hyperparameters and hardware configuration parameters.

10. A hardware-aware automatic machine learning collaborative optimization program product, the program product comprising computer instructions, which, when executed by a processor, implement the steps of the hardware-aware automatic machine learning collaborative optimization method described in any one of claims 1 to 8.

Images