Semiconductor test control method and system based on big data
By deploying intelligent sensors and embedded devices on semiconductor testing equipment, combining big data and reinforcement learning algorithms, a test control decision model is built, which solves the problems of low efficiency and insufficient coverage of traditional semiconductor testing methods, and achieves more efficient and smarter test control.
Patent Information
- Application Number
- CN202410644526.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-23
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-05-23
AI Technical Summary
Traditional semiconductor testing methods have low testing efficiency, insufficient coverage and poor adaptability, making it difficult to meet the growing testing needs and changing process processes.
Using a semiconductor test control method based on big data, by deploying intelligent sensors and embedded devices on the test equipment, collecting and transmitting test data in real time, performing data preprocessing, extracting test features, and building state space and action space, combining reinforcement learning Algorithm and Dueling network structure, build a test control decision model and determine the optimal test control strategy.
It improves the intelligence level and practicality of the test, realizes adaptive testing process optimization, improves testing efficiency and quality, reduces the deviation of Q value estimation, speeds up the convergence speed, and improves sample utilization efficiency.
Smart Images

Figure CN118501648B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to testing technology, and in particular to a semiconductor testing control method and system based on big data. Background Art
[0002] In recent years, with the continuous development of semiconductor manufacturing technology and the continuous increase in the complexity of integrated circuit design, semiconductor testing has become a key link in ensuring product quality and reliability. Traditional semiconductor testing methods mainly rely on manual design of test plans and writing of test programs, which have problems such as low test efficiency, insufficient coverage, and poor adaptability, making it difficult to meet the growing test needs and changing process technology.
[0003] Therefore, there is an urgent need for a semiconductor test control method that can make full use of test data, adaptively optimize the test process, take into account both local and global test indicators, and improve the intelligence level and practicality of the test. Summary of the invention
[0004] The embodiments of the present invention provide a semiconductor test control method and system based on big data, which can solve the problems in the prior art.
[0005] According to a first aspect of the embodiments of the present invention,
[0006] Provide semiconductor test control methods based on big data, including:
[0007] Deploy smart sensors and embedded devices on the test equipment to collect and transmit test data in real time, perform data preprocessing on the test data, and extract test features corresponding to the test data after data preprocessing;
[0008] Construct a state space according to the test characteristics and the result data of the test items executed by the test device; abstract all optional test items corresponding to the test device into an action space; select any state in the state space as the current state, and determine the state transition probability function in combination with the action space; obtain the instant reward for executing any action in the action space under the current state, and construct a reward function; construct a test control decision model based on the state space, the action space, the state transition probability and the reward function;
[0009] According to the test control decision model, the test control decision model is solved by an improved decision control algorithm to determine the optimal test control strategy, wherein the improved decision control algorithm is constructed on the basis of a reinforcement learning algorithm, combined with priority experience replay and Dueling network structure.
[0010] In an optional embodiment,
[0011] The test features corresponding to the test data extracted after data preprocessing include:
[0012] Acquire a time domain signal of the test data, and divide the time domain signal into segments of the same size, determine a window and an overlap rate according to the segments, construct a data matrix according to the window and the overlap rate, and perform power spectrum density estimation according to the data matrix combined with a frequency domain autocorrelation function to obtain a first feature;
[0013] Calculating a time domain autocorrelation function according to the test data, selecting a delay point of the time domain autocorrelation function, and extracting a characteristic value of the time domain autocorrelation function according to the delay point, and using the characteristic value as a second feature;
[0014] According to the first feature and the second feature, a corresponding first weight is assigned to the first feature, and a corresponding second weight is assigned to the second feature. According to the first feature and the second feature, feature fusion is performed in combination with the first weight and the second weight to obtain a test feature.
[0015] In an optional embodiment,
[0016] Based on the state space, the action space, the state transition probability function and the reward function, constructing a test control decision model includes:
[0017] According to the test characteristics of the semiconductor device and the result data of the executed test items, the state space of the Markov decision process is constructed, and the test environment state corresponding to each time step is represented as a feature vector;
[0018] Abstract all optional test items supported by the test device into the action space of Markov decision process, where each test item corresponds to an action option;
[0019] Determine the state transition probability function of the Markov decision process and estimate the probability of transitioning to the next state after performing a certain action in the current state through a data-driven method;
[0020] Define a reward function for the Markov decision process to reflect the benefit or cost of each test decision. The reward value includes the weighted sum of at least one test indicator among the test pass rate, test efficiency, and test cost.
[0021] Based on the constructed state space, action space, state transition probability function and reward function, a test control decision model is constructed.
[0022] In an optional embodiment,
[0023] According to the test control decision model, solving the test control decision model by using an improved decision control algorithm to determine the optimal test control strategy includes:
[0024] Initializing the Q network and the target network of the test control decision model, wherein the Q network and the target network both include a shared feature layer, a state value estimator and an advantage function estimator, which are used to extract state features, set an estimated state value function and set an advantage function respectively;
[0025] Initialize the experience replay buffer and priority buffer, both used to store and prioritize historical transfer samples;
[0026] According to the current state, a test control action is selected through a greedy strategy, the action is executed and the environmental feedback is observed to obtain the immediate reward and the next state;
[0027] The transfer sample is stored in the experience replay buffer, and the transfer error and priority of the transfer sample are calculated and stored in the priority buffer;
[0028] Sample a transfer sample from the experience replay buffer according to the priority probability, and calculate the importance weight of each transfer sample;
[0029] The Dueling network structure is used to calculate the state-action value function of the transfer sample and the maximum Q value of the next state, where the state-action value function is expressed as the sum of the state value function and the advantage function;
[0030] Calculate a transfer target value for each transfer sample, where the transfer target value is used to indicate a discounted sum of an immediate reward and a maximum Q value of the next state;
[0031] The transfer error and priority of the transfer sample are updated, and the network parameters of the Q network are updated, the updated network parameters of the Q network are copied to the target network, and the output value of the target network is used as the optimal test control strategy.
[0032] In an optional embodiment,
[0033] Using the Dueling network structure to calculate the state-action value function of the transfer sample and the maximum Q value of the next state include:
[0034] Input the transfer sample into the Q network and extract the state features through the shared feature extraction layer;
[0035] Input the state features into the state value estimator and the advantage function estimator respectively, calculate the state value function and the advantage function, wherein the state value function represents the intrinsic value of the state, and the advantage function represents the advantage of taking a certain action in the state relative to the average action value;
[0036] Combining the state value function and the advantage function, we get the state-action value function, or Q function, where the advantage function needs to be subtracted from its average value over all actions to ensure the unbiasedness of the Q value estimation.
[0037] Input the features of the next state into the target network and calculate the state value function and advantage function of the next state;
[0038] The largest Q value is selected from all target actions in the next state as the target Q value, which is used to calculate the timing difference error;
[0039] The loss function is constructed using the temporal difference error, and the parameters of the Q network are updated by the gradient descent method so that the Q network approaches the optimal state-action value function.
[0040] In an optional embodiment,
[0041] Combining the state value function and the advantage function, the state-action value function is:
[0042] ;
[0043] ;
[0044] A(s , a)= f a (h(s) ; θ a )[a] ;
[0045] Among them, Q(s, a) represents the Q value of the state-action value function, s and a represent the current state and the current action corresponding to the current state respectively. represents the action space, Indicates the next action;
[0046] V(s) represents the state value of the state value function, f v () represents the state value estimator of the Q network, h(s) represents the shared feature extraction layer, θ v represents the parameters of the state value estimator of the Q network;
[0047] A(s, a) represents the function value of the advantage function, f a () represents the advantage function estimator of the Q network, θ a represents the parameters of the advantage function estimator of the Q network, and [a] represents the element corresponding to the selected action a in the advantage function output.
[0048] In an optional embodiment,
[0049] Calculate the transfer target value of each transfer sample, which is used to indicate the discount of the immediate reward and the maximum Q value of the next state and includes:
[0050] ;
[0051] ;
[0052] Among them, y represents the transfer target value, r represents the immediate reward, which means the immediate return obtained after taking action a in state s, and γ represents the discount factor, which is used to balance the importance of immediate reward and future reward;
[0053] Indicates the next action. Indicates the next state, Represents the parameters of the target network;
[0054] represents the state value function estimated by the target network, represents the advantage function estimated by the target network, represents the parameters of the state value estimator of the target network, represents the parameters of the advantage function estimator of the target network, Indicates the action corresponding to the maximum Q value.
[0055] According to a second aspect of the embodiments of the present invention,
[0056] Provide semiconductor test control system based on big data, including:
[0057] The first unit is used to deploy intelligent sensors and embedded devices on the test equipment, collect and transmit test data in real time, perform data preprocessing on the test data, and extract test features corresponding to the test data after data preprocessing;
[0058] The second unit is used to construct a state space according to the test characteristics and the result data of the test items executed by the test device; abstract all the optional test items corresponding to the test device into an action space; select any state in the state space as the current state, and determine the state transition probability function in combination with the action space; obtain the instant reward for executing any action in the action space under the current state, and construct a reward function; construct a test control decision model based on the state space, the action space, the state transition probability and the reward function;
[0059] The third unit is used to solve the test control decision model according to the test control decision model through an improved decision control algorithm to determine the optimal test control strategy, wherein the improved decision control algorithm is constructed on the basis of a reinforcement learning algorithm in combination with priority experience replay and Dueling network structure.
[0060] According to a third aspect of the embodiments of the present invention,
[0061] An electronic device is provided, comprising:
[0062] processor;
[0063] a memory for storing processor-executable instructions;
[0064] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0065] A fourth aspect of the embodiments of the present invention is:
[0066] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0067] This application provides a new and complete DQN improvement technical solution. By introducing the Dueling network structure and the priority experience replay mechanism, the algorithm can more effectively learn the state value function and advantage function, accelerate the convergence speed, improve the sample utilization efficiency, and thus obtain a better test control strategy. At the same time, the algorithm also inherits the advantages of Double DQN and reduces the deviation of Q value estimation. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] Figure 1 It is a flow chart of a semiconductor test control method based on big data according to an embodiment of the present invention;
[0069] Figure 2 It is a schematic diagram of the structure of a semiconductor test control system based on big data according to an embodiment of the present invention. DETAILED DESCRIPTION
[0070] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0071] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
[0072] Figure 1 FIG. 1 is a flow chart of a semiconductor test control method based on big data according to an embodiment of the present invention. Figure 1 As shown, the method includes:
[0073] S101 deploys smart sensors and embedded devices on the test equipment to collect and transmit test data in real time, and performs data preprocessing on the test data to extract test features corresponding to the test data after data preprocessing;
[0074] In an optional embodiment,
[0075] The test features corresponding to the test data extracted after data preprocessing include:
[0076] Acquire a time domain signal of the test data, and divide the time domain signal into segments of the same size, determine a window and an overlap rate according to the segments, construct a data matrix according to the window and the overlap rate, and perform power spectrum density estimation according to the data matrix combined with a frequency domain autocorrelation function to obtain a first feature;
[0077] Calculating a time domain autocorrelation function according to the test data, selecting a delay point of the time domain autocorrelation function, and extracting a characteristic value of the time domain autocorrelation function according to the delay point, and using the characteristic value as a second feature;
[0078] According to the first feature and the second feature, a corresponding first weight is assigned to the first feature, and a corresponding second weight is assigned to the second feature. According to the first feature and the second feature, feature fusion is performed in combination with the first weight and the second weight to obtain a test feature.
[0079] Exemplarily, the frequency domain autocorrelation function is a method to measure the repeatability of a signal in the frequency domain, the power spectral density estimation is used to describe the power distribution of a signal in the frequency domain, the delay point is a position in the autocorrelation function that represents a signal delay, and in the time domain autocorrelation, the delay point represents a time lag of the signal, and the data matrix is a structured data representation that is typically used to store and process multidimensional data. In the data matrix, data is organized into rows and columns, where each row typically represents a sample and each column represents a feature, and the feature extraction algorithm is the process of extracting useful information from raw data to create a more compact, efficient, and analyzable representation.
[0080] By deploying smart sensors and embedded devices on the test equipment, we can obtain information such as signal strength, signal-to-noise ratio, data transmission rate and latency to form test data, and then pre-process the test data by cleaning and removing noise to obtain standard data.
[0081] According to the standard data, in combination with the feature extraction algorithm, the time domain signal is extracted from the standard data through time domain transformation, and the extracted time domain signal is divided into segments of the same size by means of a sliding window, and the size of each window is ensured to be consistent, and the size and overlap rate of the window are determined according to specific needs and signal characteristics, wherein the size of the window affects the time domain resolution of each segment, and the overlap rate can be used to ensure that there is overlap between adjacent windows, and the segmented time domain signal is constructed into a data matrix according to the window and the overlap rate, each column represents the data in a window, and each column of the data matrix is Fourier transformed, and the amplitude spectrum of the Fourier transform is calculated, and then the autocorrelation calculation is performed to obtain the frequency domain autocorrelation function, and the frequency domain autocorrelation function is used to estimate the power spectrum density, and according to the power spectrum density estimation result, the information about the frequency domain characteristics of the signal is extracted, and finally the frequency domain characteristics of the standard data are obtained;
[0082] Acquire the standard data, calculate a time domain autocorrelation function according to the standard data, select a delay point of the time domain autocorrelation function according to task requirements, wherein the delay point represents the time offset of two time domain signals, extract the value of the time domain autocorrelation function at the selected delay point, and use the characteristic value of the extracted time domain autocorrelation function as the second feature, i.e., the time domain feature of the standard data, by evaluating the autocorrelation value at the selected specific delay point and the summary value obtained by processing the entire autocorrelation function;
[0083] According to the first feature and the second feature, a corresponding first weight is assigned to the first feature, a corresponding second weight is assigned to the second feature, and the first feature and the second feature are fused by weighted addition according to the first weight and the second weight, so as to finally obtain the component communication feature.
[0084] In this embodiment, by extracting the time domain signal and the time domain autocorrelation function and applying the frequency domain autocorrelation function to perform power spectral density estimation, rich test data features are obtained, which helps to capture different aspects of information, such as signal quality, transmission characteristics and time domain relationships. Preprocessing is performed by cleaning and removing noise to ensure the quality of standard data and help eliminate unnecessary interference factors.
[0085] S102. Construct a state space according to the test characteristics and the result data of the test items executed by the test device; abstract all optional test items corresponding to the test device into an action space; select any state in the state space as the current state, and determine the state transition probability function in combination with the action space; obtain the instant reward for executing any action in the action space in the current state, and construct a reward function; construct a test control decision model based on the state space, the action space, the state transition probability and the reward function;
[0086] In an optional embodiment,
[0087] Based on the state space, the action space, the state transition probability function and the reward function, constructing a test control decision model includes:
[0088] According to the test characteristics of the semiconductor device and the result data of the executed test items, the state space of the Markov decision process is constructed, and the test environment state corresponding to each time step is represented as a feature vector;
[0089] Abstract all optional test items supported by the test device into the action space of Markov decision process, where each test item corresponds to an action option;
[0090] Determine the state transition probability function of the Markov decision process and estimate the probability of transitioning to the next state after performing an action in the current state through a data-driven approach;
[0091] Define a reward function for the Markov decision process to reflect the benefit or cost of each test decision. The reward value includes the weighted sum of at least one test indicator among the test pass rate, test efficiency, and test cost.
[0092] Based on the constructed state space, action space, state transition probability function and reward function, a test control decision model is constructed.
[0093] For example, the test control decision problem can be modeled as a Markov decision process (MDP). MDP is a mathematical framework for describing sequential decision problems, which consists of four elements: state space, action space, transition probability and reward function. In the test control decision problem, the four elements of MDP can be defined as follows:
[0094] State space S: The state space S contains the characteristic information of the device under test and the result data of the executed test items. Specifically, the state can be represented as a feature vector s=(x_1, x_2, ..., x_n, r_1, r_2, ..., r_m), where x_i represents the i-th characteristic attribute of the device under test (such as device type, functional parameters, historical test data, etc.), and r_j represents the result of the j-th test item executed (such as pass / fail, performance indicators, etc.). The dimension of the state space depends on the number of selected features and the number of executed test items.
[0095] Action space A: The action space A contains all the optional test items. Each test item corresponds to a unique action a∈A. Test items can be different types such as functional test, performance test, safety test, etc., and can also target different functional modules or parameters of the device. The size of the action space depends on the number of candidate test items.
[0096] Transition probability P(s'|s,a): Transition probability describes the probability that the environment will transition to a new state s' after executing action a in state s. In the test control decision problem, the transition probability depends on the coverage of the test items and the actual condition of the device. Generally speaking, after executing a test item, the corresponding result component r_j in the state vector will change, while other components remain unchanged. The transition probability can be estimated by historical test data or expert experience.
[0097] Reward function R(s, a): The reward function represents the immediate reward obtained by executing action a in state s. In the test control decision problem, the reward function should comprehensively consider factors such as test efficiency, test cost, and test coverage. For example, the reward function can be defined as:
[0098] R(s, a) = w_1 * (test case pass rate) - w_2 * (test execution time) - w_3 * (test resource consumption) + w_4 * (test coverage improvement);
[0099] Among them, w_1, w_2, w_3, and w_4 are weight coefficients, which represent the importance of different factors. By setting the reward function reasonably, the agent can be guided to learn a test control strategy that takes into account multiple goals.
[0100] In the MDP framework, the goal of the test control decision problem is to find an optimal strategy π* that maximizes the expected cumulative reward under the strategy. In order to solve the optimal strategy π*, reinforcement learning algorithms can be used, such as Q-learning, SARSA, DQN, etc. These algorithms continuously interact with the environment, update the value function or policy function, and finally converge to the optimal strategy.
[0101] S103. According to the test control decision model, the test control decision model is solved by an improved decision control algorithm to determine the optimal test control strategy, wherein the improved decision control algorithm is constructed on the basis of a reinforcement learning algorithm in combination with priority experience replay and Dueling network structure.
[0102] In an optional embodiment,
[0103] According to the test control decision model, solving the test control decision model by using an improved decision control algorithm to determine the optimal test control strategy includes:
[0104] Initializing the Q network and the target network of the test control decision model, wherein the Q network and the target network both include a shared feature layer, a state value estimator and an advantage function estimator, which are used to extract state features, set an estimated state value function and set an advantage function respectively;
[0105] Initialize the experience replay buffer and priority buffer, both used to store and prioritize historical transfer samples;
[0106] According to the current state, a test control action is selected through a greedy strategy, the action is executed and the environmental feedback is observed to obtain the immediate reward and the next state;
[0107] The transfer sample is stored in the experience replay buffer, and the transfer error and priority of the transfer sample are calculated and stored in the priority buffer;
[0108] Sample a transfer sample from the experience replay buffer according to the priority probability, and calculate the importance weight of each transfer sample;
[0109] The Dueling network structure is used to calculate the state-action value function of the transfer sample and the maximum Q value of the next state, where the state-action value function is expressed as the sum of the state value function and the advantage function;
[0110] Calculate a transfer target value for each transfer sample, where the transfer target value is used to indicate a discounted sum of an immediate reward and a maximum Q value of the next state;
[0111] The transfer error and priority of the transfer sample are updated, and the network parameters of the Q network are updated, the updated network parameters of the Q network are copied to the target network, and the output value of the target network is used as the optimal test control strategy.
[0112] Exemplarily, combining the two improvements of Prioritized Experience Replay and DuelingDQN, this application provides a new and complete DQN improvement technical solution, called Dueling DDQN with Prioritized Experience Replay (Dueling DDQN-PER). By introducing the Dueling network structure and the priority experience replay mechanism, the algorithm can more effectively learn the state value function and advantage function, accelerate the convergence speed, improve the sample utilization efficiency, and thus obtain a better test control strategy. At the same time, the algorithm also inherits the advantages of Double DQN and reduces the deviation of Q value estimation.
[0113] The traditional DQN algorithm uses a single value function network to estimate the state-action value function Q(s, a). DuelingDQN splits the value function network into two parallel sub-networks: the state value function V(s) and the advantage function A(s, a). The final Q value is obtained by combining the outputs of the two sub-networks. The advantages of the Dueling network structure are:
[0114] The state value function V(s) evaluates the intrinsic value of the state, regardless of the specific action taken. This helps learn more stable and discriminative state representations. The advantage function A(s, a) evaluates the relative advantage of taking action a in state s, which helps learn more precise action selection strategies. Decoupling the state value function and the advantage function can make more effective use of sample data and accelerate training convergence.
[0115] The traditional experience replay mechanism extracts training samples from the buffer uniformly and randomly, without considering the importance of different samples to the training. The idea of priority experience replay is that for those samples with large TD error (i.e., the difference between the Q value target and the current estimate), a higher extraction priority should be given to speed up the training convergence. The priority experience replay mechanism of the Dueling DDQN-PER algorithm is implemented as follows:
[0116] Define the priority of samples: For each sample i, define its priority as ,in, is a small positive number used to avoid priority zero. represents the TD error corresponding to the i-th sample;
[0117] Define the priority probability distribution: According to the priority of the sample, define its probability of being extracted as:
[0118] , where α is a hyperparameter that controls the shape of the priority distribution, It indicates the priority of the i-th sample under the action of hyperparameters. Indicates the priority corresponding to the kth sample under the action of hyperparameters.
[0119] Define importance weights: In order to correct the bias introduced by priority sampling, an importance weight is introduced for each sample when calculating the loss function. , where N is the buffer size and β is a hyperparameter that controls the degree of weight scaling.
[0120] During the training process, the Dueling DDQN-PER algorithm extracts samples according to the priority probability distribution P(i), multiplies the corresponding importance weight w_i by the loss function, and performs gradient descent optimization. After each training, the priority of the sample is updated according to the new TD error.
[0121] In an optional embodiment,
[0122] Using the Dueling network structure to calculate the state-action value function of the transfer sample and the maximum Q value of the next state include:
[0123] Input the transfer sample into the Q network and extract the state features through the shared feature extraction layer;
[0124] Input the state features into the state value estimator and the advantage function estimator respectively, calculate the state value function and the advantage function, wherein the state value function represents the intrinsic value of the state, and the advantage function represents the advantage of taking a certain action in the state relative to the average action value;
[0125] Combining the state value function and the advantage function, we get the state-action value function, or Q function, where the advantage function needs to be subtracted from its average value over all actions to ensure the unbiasedness of the Q value estimation.
[0126] Input the features of the next state into the target network and calculate the state value function and advantage function of the next state;
[0127] The largest Q value is selected from all target actions in the next state as the target Q value, which is used to calculate the timing difference error;
[0128] The loss function is constructed using the temporal difference error, and the parameters of the Q network are updated by the gradient descent method so that the Q network approaches the optimal state-action value function.
[0129] Combining the Dueling network structure and the priority experience replay mechanism, the complete process of the Dueling DDQN-PER algorithm is as follows:
[0130] Create a Q network and a target network with the same network structure, including: Shared feature layer: used to extract feature representation of the state. State value estimator: used to estimate the intrinsic value of the state. Advantage function estimator: used to estimate the advantage of taking a specific action relative to the average action in a state. Randomly initialize the parameters of the Q network and the target network. Create an experience replay buffer to store the transition samples; create a priority buffer to store the priority of each transition sample.
[0131] Get the current state s. According to the ε-greedy strategy, randomly select action a with a probability of ε, otherwise select the optimal action output by the Q network; execute action a, observe the environmental feedback, obtain the immediate reward r and the next state; store the transfer sample in the experience replay buffer, calculate the transfer error of the transfer sample; calculate the priority of the transfer sample based on the transfer error; store the priority in the priority buffer.
[0132] Sample a batch of transition samples from the experience replay buffer according to the priority probability, and calculate the importance weight for each sampled transition sample. For each sampled transition sample, use the Q network to calculate the state-action value function; use the target network to calculate the maximum Q value of the next state. For each sampled transition sample, calculate its transition target value.
[0133] Update priority and Q network: Calculate the new transfer error for each transfer sample; Update the priority of the transfer sample; Update the parameters of the Q network using the weighted mean square error loss function; Minimize the loss function by gradient descent and update the parameters of the Q network. Output the trained Q network as the optimal test control strategy.
[0134] Through a series of operations such as initializing the dual network structure, maintaining the experience buffer, greedy exploration strategy, priority experience replay, calculating the state-action value function and target Q value, and updating network parameters, the algorithm can learn the optimal decision-making strategy in the test control task, adaptively adjust the test parameters and execution process, and improve the test efficiency and quality.
[0135] In an optional embodiment,
[0136] Combining the state value function and the advantage function, the state-action value function is:
[0137] ;
[0138] ;
[0139] A(s , a)= f a (h(s) ; θ a )[a] ;
[0140] Among them, Q(s, a) represents the Q value of the state-action value function, s and a represent the current state and the current action corresponding to the current state respectively. represents the action space, Indicates the next action;
[0141] V(s) represents the state value of the state value function, f v () represents the state value estimator of the Q network, h(s) represents the shared feature extraction layer, θ v represents the parameters of the state value estimator of the Q network;
[0142] A(s, a) represents the function value of the advantage function, f a () represents the advantage function estimator of the Q network, θ a represents the parameters of the advantage function estimator of the Q network, and [a] represents the element corresponding to the selected action a in the advantage function output.
[0143] In an optional embodiment,
[0144] Calculate the transfer target value of each transfer sample, which is used to indicate the discount of the immediate reward and the maximum Q value of the next state and includes:
[0145] ;
[0146] ;
[0147] Among them, y represents the transfer target value, r represents the immediate reward, which means the immediate return obtained after taking action a in state s, and γ represents the discount factor, which is used to balance the importance of immediate reward and future reward;
[0148] Indicates the next action. Indicates the next state, Represents the parameters of the target network;
[0149] represents the state value function estimated by the target network, represents the advantage function estimated by the target network, represents the parameters of the state value estimator of the target network, represents the parameters of the advantage function estimator of the target network, Indicates the action corresponding to the maximum Q value.
[0150] Figure 2FIG. 1 is a schematic diagram of the structure of a semiconductor test control system based on big data according to an embodiment of the present invention. Figure 2 As shown, the system comprises:
[0151] The first unit is used to deploy intelligent sensors and embedded devices on the test equipment, collect and transmit test data in real time, perform data preprocessing on the test data, and extract test features corresponding to the test data after data preprocessing;
[0152] The second unit is used to construct a state space according to the test characteristics and the result data of the test items executed by the test device; abstract all the optional test items corresponding to the test device into an action space; select any state in the state space as the current state, and determine the state transition probability function in combination with the action space; obtain the instant reward for executing any action in the action space under the current state, and construct a reward function; construct a test control decision model based on the state space, the action space, the state transition probability and the reward function;
[0153] The third unit is used to solve the test control decision model according to the test control decision model through an improved decision control algorithm to determine the optimal test control strategy, wherein the improved decision control algorithm is constructed on the basis of a reinforcement learning algorithm in combination with priority experience replay and Dueling network structure.
[0154] According to a third aspect of the embodiments of the present invention,
[0155] An electronic device is provided, comprising:
[0156] processor;
[0157] a memory for storing processor-executable instructions;
[0158] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.
[0159] A fourth aspect of the embodiments of the present invention is:
[0160] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.
[0161] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.
[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. Semiconductor test control method based on big data, It is characterized in that include: Deploy smart sensors and embedded devices on the test equipment to collect and transmit test data in real time, perform data preprocessing on the test data, and extract test features corresponding to the test data after data preprocessing; Constructing a state space according to the test characteristics and result data of the test items executed by the test device; Abstracting all optional test items corresponding to the test device into an action space; Select any state in the state space as the current state, and determine the state transition probability function in combination with the action space; obtain the instant reward for executing any action in the action space in the current state, and construct a reward function; construct a test control decision model based on the state space, the action space, the state transition probability and the reward function; According to the test control decision model, the test control decision model is solved by an improved decision control algorithm to determine the optimal test control strategy, wherein the improved decision control algorithm is constructed on the basis of a reinforcement learning algorithm, combined with priority experience replay and Dueling network structure.
2. The method according to claim 1, It is characterized in that The test features corresponding to the test data extracted after data preprocessing include: Acquire a time domain signal of the test data, and divide the time domain signal into segments of the same size, determine a window and an overlap rate according to the segments, construct a data matrix according to the window and the overlap rate, and perform power spectrum density estimation according to the data matrix combined with a frequency domain autocorrelation function to obtain a first feature; Calculating a time domain autocorrelation function according to the test data, selecting a delay point of the time domain autocorrelation function, and extracting a characteristic value of the time domain autocorrelation function according to the delay point, and using the characteristic value as a second feature; According to the first feature and the second feature, a corresponding first weight is assigned to the first feature, and a corresponding second weight is assigned to the second feature. According to the first feature and the second feature, feature fusion is performed in combination with the first weight and the second weight to obtain a test feature.
3. The method according to claim 1, It is characterized in that Based on the state space, the action space, the state transition probability function and the reward function, constructing a test control decision model includes: According to the test characteristics of the semiconductor device and the result data of the executed test items, the state space of the Markov decision process is constructed, and the test environment state corresponding to each time step is represented as a feature vector; Abstract all optional test items supported by the test device into the action space of Markov decision process, where each test item corresponds to an action option; Determine the state transition probability function of the Markov decision process and estimate the probability of transitioning to the next state after performing an action in the current state through a data-driven approach; Define a reward function for the Markov decision process to reflect the benefit or cost of each test decision. The reward value includes the weighted sum of at least one test indicator among the test pass rate, test efficiency, and test cost. Based on the constructed state space, action space, state transition probability function and reward function, a test control decision model is constructed.
4. The method according to claim 1, It is characterized in that According to the test control decision model, solving the test control decision model by using an improved decision control algorithm to determine the optimal test control strategy includes: Initializing the Q network and the target network of the test control decision model, wherein the Q network and the target network both include a shared feature layer, a state value estimator and an advantage function estimator, which are used to extract state features, set an estimated state value function and set an advantage function respectively; Initialize the experience replay buffer and priority buffer, both used to store and prioritize historical transfer samples; According to the current state, a test control action is selected through a greedy strategy, the action is executed and the environmental feedback is observed to obtain the immediate reward and the next state; The transfer sample is stored in the experience replay buffer, and the transfer error and priority of the transfer sample are calculated and stored in the priority buffer; Sample a transfer sample from the experience replay buffer according to the priority probability, and calculate the importance weight of each transfer sample; The Dueling network structure is used to calculate the state-action value function of the transfer sample and the maximum Q value of the next state, where the state-action value function is expressed as the sum of the state value function and the advantage function; Calculate a transfer target value for each transfer sample, where the transfer target value is used to indicate a discounted sum of an immediate reward and a maximum Q value of the next state; The transfer error and priority of the transfer sample are updated, and the network parameters of the Q network are updated, the updated network parameters of the Q network are copied to the target network, and the output value of the target network is used as the optimal test control strategy.
5. The method according to claim 4, It is characterized in that Using the Dueling network structure to calculate the state-action value function of the transfer sample and the maximum Q value of the next state include: Input the transfer sample into the Q network and extract the state features through the shared feature extraction layer; Input the state features into the state value estimator and the advantage function estimator respectively, calculate the state value function and the advantage function, wherein the state value function represents the intrinsic value of the state, and the advantage function represents the advantage of taking a certain action in the state relative to the average action value; Combining the state value function and the advantage function, we get the state-action value function, or Q function, where the advantage function needs to be subtracted from its average value over all actions to ensure the unbiasedness of the Q value estimation. Input the features of the next state into the target network and calculate the state value function and advantage function of the next state; The largest Q value is selected from all target actions in the next state as the target Q value, which is used to calculate the timing difference error; The loss function is constructed using the temporal difference error, and the parameters of the Q network are updated by the gradient descent method so that the Q network approaches the optimal state-action value function.
6. The method according to claim 5, It is characterized in that Combining the state value function and the advantage function, the state-action value function is: ; ; ; Among them, Q(s, a) represents the Q value of the state-action value function, s and a represent the current state and the current action corresponding to the current state respectively. represents the action space, Indicates the next action; V(s) represents the state value of the state value function, f v () represents the state value estimator of the Q network, h(s) represents the shared feature extraction layer, θ v represents the parameters of the state value estimator of the Q network; A(s, a) represents the function value of the advantage function, f a () represents the advantage function estimator of the Q network, θ a represents the parameters of the advantage function estimator of the Q network, and [a] represents the element corresponding to the selected action a in the advantage function output.
7. The method according to claim 6, It is characterized in that Calculate the transfer target value of each transfer sample, which is used to indicate the discount of the immediate reward and the maximum Q value of the next state and includes: ; ; Among them, y represents the transfer target value, r represents the immediate reward, which means the immediate return obtained after taking action a in state s, and γ represents the discount factor, which is used to balance the importance of immediate reward and future reward; Indicates the next action. Indicates the next state, Represents the parameters of the target network; represents the state value function estimated by the target network, represents the advantage function estimated by the target network, represents the parameters of the state value estimator of the target network, represents the parameters of the advantage function estimator of the target network, Indicates the action corresponding to the maximum Q value.
8. A semiconductor test control system based on big data, used to implement the method according to any one of claims 1 to 7, It is characterized in that include: The first unit is used to deploy intelligent sensors and embedded devices on the test equipment, collect and transmit test data in real time, perform data preprocessing on the test data, and extract test features corresponding to the test data after data preprocessing; A second unit is used to construct a state space according to the test characteristics and result data of the test items executed by the test device; Abstracting all optional test items corresponding to the test device into an action space; Select any state in the state space as the current state, and determine the state transition probability function in combination with the action space; obtain the instant reward for executing any action in the action space in the current state, and construct a reward function; Constructing a test control decision model based on the state space, the action space, the state transition probability and the reward function; The third unit is used to solve the test control decision model according to the test control decision model through an improved decision control algorithm to determine the optimal test control strategy, wherein the improved decision control algorithm is constructed on the basis of a reinforcement learning algorithm in combination with priority experience replay and Dueling network structure.
9. An electronic device, It is characterized in that include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.
10. A computer-readable storage medium having computer program instructions stored thereon, It is characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.
