Perception model self-recommendation method and device based on hyper-parameter intelligent optimization

By using Transformer encoder and target performance prediction model in perceptual deep learning model, combined with deep reinforcement learning, the problem of inefficient hyperparameter optimization is solved, and efficient hyperparameter search and performance improvement is achieved.

CN118917412APending Publication Date: 2024-11-08BEIJING JINGHANG COMPUTING & COMM RES INST
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202410894307.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-04
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The prior art has inefficient computation in the hyperparameter optimization of perceived deep learning models, making it difficult to cope with large-scale data and complex tasks, and traditional methods rely on complete training evaluation, which makes the optimization process time-consuming and labor-intensive.

Method used

The policy network and target performance prediction model based on Transformer encoder are adopted, combined with deep reinforcement learning, parallel inference and fast performance prediction are realized, and the training needs of full model are reduced.

Benefits of technology

It significantly improves the computing efficiency and throughput of hyperparameter optimization, reduces training overhead, supports large-scale hyperparameter search, and improves the performance and iteration speed of the perceived model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917412A_ABST
    Figure CN118917412A_ABST
Patent Text Reader

Abstract

According to the perception model self-recommendation method and device based on hyper-parameter intelligent optimization, the calculation efficiency and throughput in the hyper-parameter optimization process are greatly improved, the model is allowed to be rapidly iterated in a large-scale data set and a complex task, the dependence on calculation resources is reduced, and therefore the requirements of a modern AI system for high efficiency and rapid response are met; compared with the prior art, the method has the advantages that the hyper-parameter search process is obviously accelerated, the actual training overhead is reduced, the optimization process is more flexible and economical, and a wider hyper-parameter space can be explored under limited time and resources. The method comprises the following steps: (1) designing a strategy network based on a Transform encoder; (2) constructing a target performance prediction model; and (3) performing hyper-parameter optimization by adopting a near-end strategy optimization algorithm.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and in particular to a perception model self-recommendation method based on hyperparameter intelligent optimization, and a perception model self-recommendation device based on hyperparameter intelligent optimization. Background Art

[0002] In today's AI technology field, deep learning models have been widely used in image recognition, speech processing, natural language understanding and other aspects. Among them, perception models, as core components, play a vital role in tasks such as object detection, semantic segmentation, and scene understanding. With the growth of data scale and the diversification of application scenarios, how to efficiently optimize the hyperparameters of these models to achieve optimal performance has become a key issue that needs to be solved. Hyperparameter optimization is a resource-intensive process. Traditional methods such as grid search and random search are not only time-consuming, but also inefficient and difficult to cope with the needs of large-scale models and complex tasks.

[0003] In recent years, reinforcement learning, as an adaptive policy search method, has shown great potential in the field of hyperparameter optimization. In particular, deep reinforcement learning (DRL) technology can automatically explore the optimization space and find the optimal configuration by building a learning framework for the interaction between agents and the environment. As a representative of DRL, the Proximal Policy Optimization (PPO) algorithm is widely used in hyperparameter tuning tasks due to its robustness and efficiency. For example, Chen Senpeng et al. designed an LSTM policy network in "Hyperparameter Optimization Method Based on Reinforcement Learning" to achieve hyperparameter optimization of random forest and XGBoost algorithms; Liu Yunhao solved the problem of excessive hyperparameter configuration dimensions caused by the simultaneous existence of multiple machine learning algorithms in "Research on Hyperparameter Optimization of Machine Learning Pipeline Based on Reinforcement Learning" by modeling the hyperparameter optimization of the pipeline as a multi-agent collaboration problem to reduce the action dimension of each agent. However, these works mainly focus on machine learning algorithms with lower complexity, and there are still certain limitations in the optimization of hyperparameters specific to perceptual deep learning models, especially in automation and efficiency improvement in actual deployment. Most current studies tend to use recurrent neural networks (such as LSTM) to model strategies. Although LSTM can effectively process sequence data, its computational efficiency is low when dealing with large-scale parallel computing and long sequences, and it cannot be parallelized, which limits the speed and scale of hyperparameter optimization. In addition, most existing technologies rely on fully training the perception model until convergence to evaluate the effect of hyperparameter settings, which makes the optimization process extremely time-consuming. This approach is particularly inefficient when faced with the high dimensionality and complexity of the hyperparameter space.

[0004] Although deep reinforcement learning has made progress in hyperparameter optimization, efficient and automated hyperparameter optimization techniques for perception models still face challenges such as low computational efficiency of policy networks and high evaluation costs. Therefore, developing a new technical solution that can both improve optimization efficiency and maintain optimization quality is of great significance for promoting rapid iteration and performance improvement of perception-based deep learning models in practical applications. Summary of the invention

[0005] In order to overcome the defects of the prior art, the technical problem to be solved by the present invention is to provide a perceptual model self-recommendation method based on intelligent optimization of hyperparameters, which greatly improves the computational efficiency and throughput in the hyperparameter optimization process, allows the model to iterate quickly in large-scale data sets and complex tasks, and reduces dependence on computing resources, thereby meeting the needs of modern AI systems for high efficiency and rapid response, significantly accelerating the hyperparameter search process, reducing actual training overhead, making the optimization process more flexible and economical, and helping to explore a broader hyperparameter space with limited time and resources.

[0006] The technical solution of the present invention is: the perception model self-recommendation method based on hyperparameter intelligent optimization includes the following steps:

[0007] (1) Design a policy network based on the Transformer encoder. The Transformer encoder is composed of multiple encoder layers, each of which contains two sublayers: a multi-head self-attention mechanism and a feedforward neural network. In each encoder layer, the multi-head self-attention mechanism is used to capture the dependencies between different positions in the sequence, and the feedforward neural network is responsible for nonlinear transformation and feature extraction. Each sublayer is followed by a residual connection and layer normalization to ensure stable gradient transmission and efficient model training.

[0008] The policy network is designed based on the Transformer encoder structure. First, the candidate points in the hyperparameter space are encoded to form a serialized input representation. Each hyperparameter is represented as a vector. These vectors are processed by linear transformation and position encoding and then input into the Transformer encoder. Position encoding injects position information without introducing a loop or convolution structure, so that the model can recognize the relationship between each position in the sequence. The input sequence is processed by multiple encoder layers and converted into a high-dimensional feature representation. The multi-head self-attention mechanism ensures that the interdependence between hyperparameters is effectively captured, and the feedforward neural network further extracts and transforms features so that the final feature representation contains rich contextual information. Then, the output of the encoder passes through a fully connected layer and a Softmax activation function to generate a policy distribution.

[0009] (2) constructing a target performance prediction model, which adopts a fully connected neural network architecture;

[0010] (3) Proximal policy optimization (PPO) is used to optimize hyperparameters, including state representation, action generation, strategy update, reward calculation, and model prediction.

[0011] The present invention combines the advantages of deep reinforcement learning and Transformer encoders, bringing significant beneficial effects. By adopting a policy network based on Transformer encoders, the parallel reasoning ability of the policy network is realized compared with the traditional LSTM structure. This improvement greatly improves the computational efficiency and throughput in the hyperparameter optimization process. The Transformer encoder uses the self-attention mechanism and parallel computing architecture to simultaneously process the feature vectors of multiple hyperparameter configurations, thereby accelerating the reasoning speed of the policy network. This design not only meets the needs of modern AI systems for high efficiency and fast response, but also effectively supports the execution of large-scale hyperparameter search tasks and improves the overall optimization efficiency. The present invention solves the problem of long hyperparameter evaluation cycle by training a model that can accurately predict the optimization target value of the perception model. Traditional methods require complete model training and verification after each hyperparameter adjustment, which is time-consuming and labor-intensive, greatly limiting the efficiency of hyperparameter optimization. By introducing the target performance prediction model, the present invention can quickly predict the performance indicators of the perception model after each hyperparameter configuration is generated, thereby reducing the need for frequent full model training. This improvement not only accelerates the hyperparameter optimization process, but also reduces the actual training overhead, making the entire optimization process more flexible and economical. With limited time and resources, the optimization system can explore a wider hyperparameter space and discover better hyperparameter configurations, significantly improving the performance of the perception model.

[0012] A perception model self-recommendation device based on hyperparameter intelligent optimization is also provided, which includes:

[0013] The policy network design module is configured to design a policy network based on the Transformer encoder. The Transformer encoder is composed of multiple encoder layers, each of which contains two sublayers: a multi-head self-attention mechanism and a feedforward neural network. In each encoder layer, the multi-head self-attention mechanism is used to capture the dependencies between different positions in the sequence.

[0014] In the system, the feedforward neural network is responsible for nonlinear transformation and feature extraction; each sublayer is followed by residual connection and layer normalization to ensure stable transmission of gradients and efficient training of the model;

[0016] The policy network is designed based on the Transformer encoder structure. First, the candidate points in the hyperparameter space are encoded to form a serialized input representation. Each hyperparameter is represented as a vector. These vectors are processed by linear transformation and position encoding and then input into the Transformer encoder. Position encoding injects position information without introducing a loop or convolution structure, so that the model can recognize the relationship between each position in the sequence. The input sequence is processed by multiple encoder layers and converted into a high-dimensional feature representation. The multi-head self-attention mechanism ensures that the interdependence between hyperparameters is effectively captured. The feedforward neural network further extracts and transforms features so that the final feature representation contains rich contextual information.

[0017] Then, the output of the encoder passes through a fully connected layer and a Softmax activation function to generate a policy distribution;

[0018] A target performance prediction model building module, which is configured to build a target performance prediction model, and the target performance prediction model adopts a fully connected neural network architecture;

[0019] The hyperparameter optimization module is configured to use proximal policy optimization (PPO) for hyperparameter optimization, including: state representation, action generation, policy update, reward calculation, and model prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flow chart of the perceptual model self-recommendation method based on intelligent optimization of hyperparameters according to the present invention. DETAILED DESCRIPTION

[0022] like Figure 1 As shown in FIG. 1 , the perception model self-recommendation method based on hyperparameter intelligent optimization includes the following steps:

[0023] (1) Design a policy network based on the Transformer encoder. The Transformer encoder is composed of multiple encoder layers, each of which contains two sublayers: a multi-head self-attention mechanism and a feedforward neural network. In each encoder layer, the multi-head self-attention mechanism is used to capture the dependencies between different positions in the sequence, and the feedforward neural network is responsible for nonlinear transformation and feature extraction. Each sublayer is followed by a residual connection and layer normalization to ensure stable gradient transmission and efficient model training.

[0024] The policy network is designed based on the Transformer encoder structure. First, the candidate points in the hyperparameter space are encoded to form a serialized input representation. Each hyperparameter is represented as a vector. These vectors are processed by linear transformation and position encoding and then input into the Transformer encoder. Position encoding injects position information without introducing a loop or convolution structure, so that the model can recognize the relationship between each position in the sequence. The input sequence is processed by multiple encoder layers and converted into a high-dimensional feature representation. The multi-head self-attention mechanism ensures that the interdependence between hyperparameters is effectively captured, and the feedforward neural network further extracts and transforms features so that the final feature representation contains rich contextual information. Then, the output of the encoder passes through a fully connected layer and a Softmax activation function to generate a policy distribution.

[0025] (2) constructing a target performance prediction model, which adopts a fully connected neural network architecture;

[0026] (3) Proximal policy optimization (PPO) is used to optimize hyperparameters, including state representation, action generation, strategy update, reward calculation, and model prediction.

[0027] The present invention combines the advantages of deep reinforcement learning and Transformer encoders, bringing significant beneficial effects. By adopting a policy network based on Transformer encoders, the parallel reasoning ability of the policy network is realized compared with the traditional LSTM structure. This improvement greatly improves the computational efficiency and throughput in the hyperparameter optimization process. The Transformer encoder uses the self-attention mechanism and parallel computing architecture to simultaneously process the feature vectors of multiple hyperparameter configurations, thereby accelerating the reasoning speed of the policy network. This design not only meets the needs of modern AI systems for high efficiency and fast response, but also effectively supports the execution of large-scale hyperparameter search tasks and improves the overall optimization efficiency. The present invention solves the problem of long hyperparameter evaluation cycle by training a model that can accurately predict the optimization target value of the perception model. Traditional methods require complete model training and verification after each hyperparameter adjustment, which is time-consuming and labor-intensive, greatly limiting the efficiency of hyperparameter optimization. By introducing the target performance prediction model, the present invention can quickly predict the performance indicators of the perception model after each hyperparameter configuration is generated, thereby reducing the need for frequent full model training. This improvement not only accelerates the hyperparameter optimization process, but also reduces the actual training overhead, making the entire optimization process more flexible and economical. With limited time and resources, the optimization system can explore a wider hyperparameter space and discover better hyperparameter configurations, significantly improving the performance of the perception model.

[0028] Preferably, in step (1), the multi-head self-attention mechanism includes: first, the input sequence is linearly transformed to generate a query vector Query, a key vector Key and a value vector Value; then, the dot product of the query vector and the key vector is calculated, and normalized by a Softmax function to obtain an attention weight; finally, the attention weight is multiplied by the value vector to obtain a weighted output representation.

[0029] Preferably, in step (1), in each encoder layer, the output of the self-attention mechanism is processed by a position-independent feedforward neural network consisting of two linear transformations and a ReLU activation function.

[0030] Preferably, in step (1), the residual connection forms a short connection path by directly adding the sub-layer input to the output, and the layer normalization standardizes the output of each sub-layer.

[0031] Preferably, in step (2), in terms of input representation, a feature engineering-based encoding method is used to convert candidate points in the hyperparameter space into vector representations. For each hyperparameter, its value range and step size are first defined, and then the value is normalized and mapped to the interval [0,1]. For discrete hyperparameters, a one-hot encoding method is used to convert them into high-dimensional sparse vectors; for continuous hyperparameters, a linear transformation method is used to convert them into low-dimensional dense vectors.

[0032] Preferably, in the step (2), in terms of position coding, a position coding method based on sine and cosine functions is adopted to enable the model to recognize the relationship between each position in the sequence. For the i-th position in the sequence, the 2j-th dimension and the 2j+1-th dimension of the position coding vector are generated by the sine function and the cosine function respectively.

[0033] Preferably, in the step (2), in terms of strategy distribution generation, a method based on a fully connected layer and a Softmax activation function is adopted, where the output of the encoder first passes through a fully connected layer to map the high-dimensional features to a probability distribution in the hyperparameter space, and the weight matrix and bias vector of the fully connected layer are randomly initialized and continuously updated during the training process, and then, the output is converted into a probability distribution through a Softmax activation function.

[0034] Preferably, in step (2), the fully connected neural network includes:

[0035] The input layer accepts feature vectors as input. Feature vectors are combinations of hyperparameters. All input features are standardized to ensure that the value ranges of each dimension are similar.

[0036] Hidden layer, including multiple fully connected layers, each hidden layer is followed by a ReLU activation function and a Dropout layer to increase the nonlinear expression ability of the model and prevent overfitting. The specific structure and number of hidden layers are determined by cross-validation and grid search. It is assumed that the number of hidden layers is 2 and the number of nodes in each layer is 32;

[0037] Output layer, a single-node fully connected layer, is used to output the predicted target performance value;

[0038] The data set is continuously collected along with the process of optimizing the hyperparameters of the perception model. When a certain amount is reached, the prediction model is trained. The data is a hyperparameter combination, and the label is the target performance value of the perception model of the current combination. The Adam optimizer is used for training, and the mean square error (MSE) is selected as the loss function. The MSE and R2 scores on the validation set are monitored, and overfitting is prevented through the early stopping strategy.

[0039] Preferably, in step (3),

[0040] The state representation is the starting point of the hyperparameter optimization process. It describes the current environment and model state.

[0041] The state is the current hyperparameter configuration, which is represented by several feature vectors. Each vector represents the current value of a hyperparameter. To ensure the continuity and trainability of the state representation, these feature vectors are normalized to the same numerical range. The multi-head self-attention mechanism of the Transformer encoder captures the dependencies between different hyperparameters in the state representation.

[0042] Action generation is the process of the policy network generating the next hyperparameter configuration based on the current state. The policy network encodes the current state through the Transformer encoder to generate high-dimensional feature representations, which are then passed through the fully connected layer and the Softmax activation function to obtain the probability distribution of each hyperparameter. First, the current state is input into the Transformer encoder, and after being processed by multiple encoder layers, a set of high-dimensional feature vectors is generated. Then, these feature vectors are passed through the fully connected layer and mapped to the hyperparameter space. Finally, the Softmax activation function converts the mapped output into a probability distribution, which represents the probability of selecting each hyperparameter value in the current state. Based on these probability distributions, the next hyperparameter configuration is generated through sampling methods.

[0043] By updating the parameters of the policy network, the generated hyperparameter configuration can obtain better performance indicators in future training. Policy update depends on two key components: policy loss function and value network. The policy loss function is used to measure the performance of the policy network. It combines the difference between the current policy and the old policy to ensure that the magnitude of the policy update is within a reasonable range. The loss function of PPO is expressed as:

[0044]

[0045] Among them, r t (θ) represents the ratio of the current strategy to the old strategy, is the advantage function estimate, ∈ is a hyperparameter set to 0.2, the value network is used to estimate the value function of the current state, the training goal of the value network is to minimize the prediction error of the value function, and its loss function is the mean square error MSE;

[0046] Reward calculation is the driving factor for policy updates. It measures the degree to which the current hyperparameter configuration improves the performance of the perception model. The reward function is determined by the target performance indicator of the perception model on the validation set. The sliding window averaging method is used to smooth the performance indicator.

[0047] The input of the target performance prediction model is the feature vector of the current hyperparameter configuration, and the output is the predicted performance index. The training process of the prediction model is carried out synchronously with the training process of the perception model, and new training samples are continuously accumulated and model parameters are updated;

[0048] Hyperparameter optimization is iterated through a closed-loop system. First, the hyperparameter configuration and policy network are initialized to form an initial state. Then, a new hyperparameter configuration is generated through the policy network based on the current state and applied to the training of the perception model. After the training is completed, the performance indicators of the perception model on the validation set are calculated and fed back to the policy network as rewards. At the same time, the current hyperparameter configuration and the corresponding performance indicators are used as new training samples to update the target performance prediction model. According to the policy loss function and value function, the parameters of the policy network are updated using the gradient descent method. The whole process is repeated until the preset optimization goal or training round is reached. The training data of the policy network and the target performance prediction model are obtained and updated through online learning. After each hyperparameter configuration is generated and trained, new sample data is added to the training set, and the model is trained and evaluated periodically.

[0049] A person skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps of the above-mentioned embodiment method, and the storage medium can be: ROM / RAM, disk, CD, memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also includes a perceptual model self-recommendation device based on hyperparameter intelligent optimization, which is usually represented in the form of functional modules corresponding to the steps of the method. The device includes:

[0050] The policy network design module is configured to design a policy network based on the Transformer encoder. The Transformer encoder is composed of multiple encoder layers, each of which contains two sublayers: a multi-head self-attention mechanism and a feedforward neural network. In each encoder layer, the multi-head self-attention mechanism is used to capture the dependencies between different positions in the sequence, and the feedforward neural network is responsible for nonlinear transformation and feature extraction. Each sublayer is followed by residual connections and layer normalization to ensure stable gradient transmission and efficient model training.

[0053] The policy network is designed based on the Transformer encoder structure. First, the candidate points in the hyperparameter space are encoded to form a serialized input representation. Each hyperparameter is represented as a vector. These vectors are processed by linear transformation and position encoding and then input into the Transformer encoder. Position encoding injects position information without introducing a loop or convolution structure, so that the model can recognize the relationship between each position in the sequence. The input sequence is processed by multiple encoder layers and converted into a high-dimensional feature representation. The multi-head self-attention mechanism ensures that the interdependence between hyperparameters is effectively captured. The feedforward neural network further extracts and transforms features so that the final feature representation contains rich contextual information.

[0054] Then, the output of the encoder passes through a fully connected layer and a Softmax activation function to generate a policy distribution;

[0055] A target performance prediction model building module, which is configured to build a target performance prediction model, and the target performance prediction model adopts a fully connected neural network architecture;

[0056] The hyperparameter optimization module is configured to use proximal policy optimization (PPO) for hyperparameter optimization, including: state representation, action generation, policy update, reward calculation, and model prediction.

[0058] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the protection scope of the technical solution of the present invention.

Claims

1. A perception model self-recommendation method based on hyperparameter intelligent optimization, characterized by: The method comprises the following steps: (1) Design a policy network based on the Transformer encoder. The Transformer encoder is composed of multiple encoder layers, each of which contains two sublayers: a multi-head self-attention mechanism and a feedforward neural network. In each encoder layer, the multi-head self-attention mechanism is used to capture the dependencies between different positions in the sequence, and the feedforward neural network is responsible for nonlinear transformation and feature extraction. Each sublayer is followed by a residual connection and layer normalization to ensure stable gradient transmission and efficient model training. The policy network is designed based on the Transformer encoder structure. First, the candidate points in the hyperparameter space are encoded to form a serialized input representation. Each hyperparameter is represented as a vector. These vectors are processed by linear transformation and position encoding and then input into the Transformer encoder. Position encoding injects position information without introducing a loop or convolution structure, so that the model can recognize the relationship between each position in the sequence. The input sequence is processed by multiple encoder layers and converted into a high-dimensional feature representation. The multi-head self-attention mechanism ensures that the interdependence between hyperparameters is effectively captured, and the feedforward neural network further extracts and transforms features so that the final feature representation contains rich contextual information. Then, the output of the encoder passes through a fully connected layer and a Softmax activation function to generate a policy distribution. (2) constructing a target performance prediction model, which adopts a fully connected neural network architecture; (3) Proximal policy optimization (PPO) is used to optimize hyperparameters, including state representation, action generation, strategy update, reward calculation, and model prediction.

2. The perception model self-recommendation method based on hyperparameter intelligent optimization according to claim 1, characterized in that: In the step (1), the multi-head self-attention mechanism includes: first, the input sequence is linearly transformed to generate a query vector Query, a key vector Key and a value vector Value; then, the dot product of the query vector and the key vector is calculated and normalized by the Softmax function to obtain the attention weight; finally, the attention weight is multiplied by the value vector to obtain a weighted output representation.

3. The perception model self-recommendation method based on hyperparameter intelligent optimization according to claim 2 is characterized in that: In step (1), in each encoder layer, the output of the self-attention mechanism is processed by a position-independent feedforward neural network consisting of two linear transformations and a ReLU activation function.

4. The method for self-recommendation of a perception model based on intelligent optimization of hyperparameters according to claim 3, characterized in that: In step (1), the residual connection forms a short connection path by directly adding the sub-layer input to the output, and the layer normalization standardizes the output of each sub-layer.

5. The perception model self-recommendation method based on hyperparameter intelligent optimization according to claim 4 is characterized in that: In the step (2), in terms of input representation, a feature engineering-based encoding method is used to convert candidate points in the hyperparameter space into vector representation. For each hyperparameter, its value range and step size are first defined, and then the value is normalized and mapped to the interval [0,1]. For discrete hyperparameters, a one-hot encoding method is used to convert them into high-dimensional sparse vectors; for continuous hyperparameters, a linear transformation method is used to convert them into low-dimensional dense vectors.

6. The perception model self-recommendation method based on hyperparameter intelligent optimization according to claim 5 is characterized in that: In the step (2), in terms of position coding, a position coding method based on sine and cosine functions is adopted to enable the model to recognize the relationship between each position in the sequence. For the i-th position in the sequence, the 2j-th dimension and the 2j+1-th dimension of the position coding vector are generated by the sine function and the cosine function respectively.

7. The perception model self-recommendation method based on hyperparameter intelligent optimization according to claim 5, characterized in that: In the step (2), in terms of strategy distribution generation, a method based on a fully connected layer and a Softmax activation function is adopted. The output of the encoder first passes through a fully connected layer to map the high-dimensional features to the probability distribution of the hyperparameter space. The weight matrix and bias vector of the fully connected layer are randomly initialized and continuously updated during the training process. Then, the output is converted into a probability distribution through a Softmax activation function.

8. The perception model self-recommendation method based on hyperparameter intelligent optimization according to claim 5, characterized in that: In step (2), the fully connected neural network includes: The input layer accepts feature vectors as input. Feature vectors are combinations of hyperparameters. All input features are standardized to ensure that the value ranges of each dimension are similar. Hidden layer, including multiple fully connected layers, each hidden layer is followed by a ReLU activation function and a Dropout layer to increase the nonlinear expression ability of the model and prevent overfitting. The specific structure and number of hidden layers are determined by cross-validation and grid search. It is assumed that the number of hidden layers is 2 and the number of nodes in each layer is 32; Output layer, a single-node fully connected layer, is used to output the predicted target performance value; The data set is continuously collected along with the process of optimizing the hyperparameters of the perception model. When a certain amount is reached, the prediction model is trained. The data is a hyperparameter combination, and the label is the target performance value of the perception model of the current combination. The Adam optimizer is used for training, and the mean square error (MSE) is selected as the loss function. The MSE and R2 scores on the validation set are monitored, and overfitting is prevented through the early stopping strategy.

9. The method for self-recommendation of a perception model based on intelligent optimization of hyperparameters according to claim 5, characterized in that: In the step (3), The state representation is the starting point of the hyperparameter optimization process. It describes the current environment and model state. The state is the current hyperparameter configuration, which is represented by several feature vectors. Each vector represents the current value of a hyperparameter. To ensure the continuity and trainability of the state representation, these feature vectors are normalized to the same numerical range. The multi-head self-attention mechanism of the Transformer encoder captures the dependencies between different hyperparameters in the state representation. Action generation is the process of the policy network generating the next hyperparameter configuration based on the current state. The policy network encodes the current state through the Transformer encoder to generate high-dimensional feature representations, which are then passed through the fully connected layer and the Softmax activation function to obtain the probability distribution of each hyperparameter. First, the current state is input into the Transformer encoder, and after being processed by multiple encoder layers, a set of high-dimensional feature vectors is generated. Then, these feature vectors are passed through the fully connected layer and mapped to the hyperparameter space. Finally, the Softmax activation function converts the mapped output into a probability distribution, which represents the probability of selecting each hyperparameter value in the current state. Based on these probability distributions, the next hyperparameter configuration is generated through sampling methods. By updating the parameters of the policy network, the generated hyperparameter configuration can obtain better performance indicators in future training. Policy update depends on two key components: policy loss function and value network. The policy loss function is used to measure the performance of the policy network. It combines the difference between the current policy and the old policy to ensure that the magnitude of the policy update is within a reasonable range. The loss function of PPO is expressed as: Among them, r t (θ) represents the ratio of the current strategy to the old strategy, is the advantage function estimate, ∈ is a hyperparameter set to 0.2, the value network is used to estimate the value function of the current state, the training goal of the value network is to minimize the prediction error of the value function, and its loss function is the mean square error MSE; Reward calculation is the driving factor for policy updates. It measures the degree to which the current hyperparameter configuration improves the performance of the perception model. The reward function is determined by the target performance indicator of the perception model on the validation set. The sliding window averaging method is used to smooth the performance indicator. The input of the target performance prediction model is the feature vector of the current hyperparameter configuration, and the output is the predicted performance index. The training process of the prediction model is carried out synchronously with the training process of the perception model, and new training samples are continuously accumulated and model parameters are updated; Hyperparameter optimization is iterated through a closed-loop system. First, the hyperparameter configuration and policy network are initialized to form an initial state. Then, a new hyperparameter configuration is generated through the policy network based on the current state and applied to the training of the perception model. After the training is completed, the performance indicators of the perception model on the validation set are calculated and fed back to the policy network as rewards. At the same time, the current hyperparameter configuration and the corresponding performance indicators are used as new training samples to update the target performance prediction model. According to the policy loss function and value function, the parameters of the policy network are updated using the gradient descent method. The whole process is repeated until the preset optimization goal or training round is reached. The training data of the policy network and the target performance prediction model are obtained and updated through online learning. After each hyperparameter configuration is generated and trained, new sample data is added to the training set, and the model is trained and evaluated periodically.

10. A perception model self-recommendation device based on hyperparameter intelligent optimization, characterized in that: It includes: The policy network design module is configured to design a policy network based on the Transformer encoder. The Transformer encoder is composed of multiple encoder layers, each of which contains two sublayers: a multi-head self-attention mechanism and a feedforward neural network. In each encoder layer, the multi-head self-attention mechanism is used to capture the dependencies between different positions in the sequence, and the feedforward neural network is responsible for nonlinear transformation and feature extraction. Each sublayer is followed by residual connections and layer normalization to ensure stable gradient transmission and efficient model training. The policy network is designed based on the Transformer encoder structure. First, the candidate points in the hyperparameter space are encoded to form a serialized input representation. Each hyperparameter is represented as a vector. These vectors are processed by linear transformation and position encoding and then input into the Transformer encoder. Position encoding injects position information without introducing a loop or convolution structure, so that the model can recognize the relationship between each position in the sequence. The input sequence is processed by multiple encoder layers and converted into a high-dimensional feature representation. The multi-head self-attention mechanism ensures that the interdependencies between hyperparameters are effectively captured. The feedforward neural network further extracts and transforms features so that the final feature representation contains rich contextual information. Then, the output of the encoder passes through a fully connected layer and a Softmax activation function to generate a policy distribution. A target performance prediction model building module, which is configured to build a target performance prediction model, and the target performance prediction model adopts a fully connected neural network architecture; The hyperparameter optimization module is configured to use proximal policy optimization (PPO) for hyperparameter optimization, including: state representation, action generation, policy update, reward calculation, and model prediction.

Citation Information

Cited By

  • Self-attention-based reasoning model partitioning method for server-free application scene

    CN120181246A

  • Chip layout method and system based on reinforcement learning and back-end optimization

    CN120235107A

  • A chip layout method and system based on reinforcement learning and backend optimization

    CN120235107B