Scan Chain Wirelength Optimization via Q-Learning

The patent implements a Q-learning reinforcement learning algorithm that uses feedback mechanisms to iteratively optimize scan chain ordering. The algorithm learns from the wirelength outcomes of previous ordering decisions and adjusts future decisions to minimize total wirelength. This feedback-driven approach resolves the contradiction by enabling effective wirelength reduction through learned optimization patterns without requiring excessively complex manual design processes.

The Q-learning algorithm performs self-service optimization by automatically learning the optimal scan chain ordering strategy through iterative training and evaluation. The system serves itself by generating training data from random initial orderings, learning from the results, and progressively improving its ordering decisions without external intervention. This self-service capability enables effective wirelength reduction while keeping the optimization process autonomous and manageable.

The patent applies preliminary action by pre-training the Q-learning algorithm offline to establish an optimized scan chain ordering policy before actual circuit operation. The training phase generates a Q-table that stores optimal ordering decisions for various states, which can then be directly applied during production without requiring additional training time. This preliminary training resolves the contradiction by separating the time-consuming learning process from the operational phase, enabling low power consumption during actual use.

The patent utilizes parameter changes by adjusting the Q-learning hyperparameters (learning rate, discount factor, exploration rate) and training configuration to optimize the balance between training time and optimization effectiveness. By carefully tuning these parameters, the system achieves sufficient wirelength and power reduction while minimizing the computational time required for training, thus resolving the contradiction between energy savings and training time investment.

The patent applies local quality by incorporating design constraints directly into the Q-learning reward function and state representation. Different regions of the state space correspond to different constraint satisfaction levels, and the reward function provides localized guidance to steer the optimization toward feasible solutions. This approach resolves the contradiction by enabling chip area reduction through scan chain reordering while managing design constraints through localized reward signals that guide the learning process toward valid configurations.

The patent uses the Q-table as an intermediary that mediates between the optimization goals (wirelength reduction, area minimization) and the design constraints. The Q-table learns to map states to actions that simultaneously satisfy multiple objectives and constraints, acting as an intermediary layer that translates complex constraint requirements into actionable ordering decisions. This intermediary mechanism resolves the contradiction by enabling area reduction while automatically handling constraint management through the learned policy.