Control of saving test data based on the execution of a pre-test program.

Abstract

An exemplary system includes a first memory containing a primary memory, a second memory containing a secondary memory, and a control system that anticipates the path taken through the test program during the planned execution of the test program and causes test data associated with the test program to be stored in the first or second memory based on the anticipated path.

Description

Technical Field

[0001] This specification is directed to controlling the storage of test data based on the execution of a pre-test program.

Background Art

[0002] A test system is configured to test the operation of an electronic device called a device under test (DUT). An exemplary test system may include test equipment for transmitting signals including commands and data to the DUT for testing. The commands and data are generated by executing a test program. Data for use by the test program may be stored in and obtained from memory on the test system. For example, the memory storing the test data may be located on each test device.

Summary of the Invention

Means for Solving the Problems

[0003] An exemplary system includes a first memory including a primary storage device, a second memory including a secondary storage device, and a control system for predicting a path through a test program taken during a planned execution of the test program and storing test data related to the test program in the first memory or the second memory based on the predicted path. An exemplary test system may include one or more of the following features alone or in combination.

[0004] Storing the test data may include storing the test data in the first memory. Storing the test data may include moving at least a portion of the test data from the first memory to the second memory before a planned execution and moving at least a portion of the test data back from the second memory to the first memory based on a planned execution time.

[0005] At least a portion of the test data may be moved from the first memory to the second memory based on the storage capacity of the first memory. At least a portion of the test data may be moved from the first memory to the second memory before the planned execution if the first memory is full. At least a portion of the test data may be moved from the first memory to the second memory if the first memory does not have sufficient storage capacity for both at least a portion of the test data and other test data. Other test data may be expected to be needed by the test program before at least a portion of the test data. At least a portion of the test data may be moved back from the second memory to the first memory before the planned execution time. The second memory may have a larger storage capacity than the first memory.

[0006] The control system may include a model for predicting the path through the test program. The model may include at least one of a probabilistic model, a Markov model, a Bayesian model, or a machine learning model. The model may include parameters obtained through a static analysis of the test program. The parameters may be weighted based on historical data. The historical data may be based on prior executions of the test program.

[0007] The control system may be configured to perform an operation that includes performing a static analysis of a test program, the static analysis of which includes defining a first path through the test program that can be taken during the execution of the assumed test program, acquiring historical data based on a second path through the test program taken during past executions of the test program, and weighting the first path based on the historical data to generate a model. Predicting the path through the test program may be done based on the model. Performing the static analysis may include defining branches in the test program. Branches may be divided into multiple paths that the test program may follow. The first path may include multiple paths. The test program may be packaged together with historical data and test data. Historical data may be acquired from the test program.

[0008] An exemplary method is to perform a static analysis of a test program, the static analysis comprising: defining a first path through the test program that can be taken during the execution of the test program under assumptions; obtaining historical data based on a second path through the test program taken during past executions of the test program; weighting the first path based on the historical data to generate a model; using the model to predict a third path through the test program that will be taken during a planned execution of the test program; and storing test data related to the test program in memory based on the third path. The method may include one or more of the following features individually or in combination:

[0009] The memory may include a first memory and a second memory. Storing test data may include storing test data in the first memory. At least a portion of the test data may be retrieved from the first memory during the execution of the test program. The method may include moving at least a portion of the test data from the first memory to the second memory before the planned execution, and moving at least a portion of the test data back from the second memory to the first memory based on the planned execution time. At least a portion of the test data may be moved from the first memory to the second memory when the first memory is full. At least a portion of the test data may be moved from the first memory to the second memory when the first memory does not have storage capacity for both at least a portion of the test data and other test data. Other test data may be expected to be needed by the test data before at least a portion of the test data. Moving at least a portion of the test data back from the second memory to the first memory may occur before the planned execution time.

[0010] The second memory may have a larger storage capacity than the first memory. Static analysis may include defining branches within the test program. Branches may be divided into multiple paths that the test program can follow. The first path may contain multiple paths. The model may include at least one of a probabilistic model, a Markov model, a Bayesian model, or a machine learning model. The test program may be packaged with historical data and test data. Historical data may be obtained from the test program.

[0011] An exemplary system includes a first memory containing a primary memory, a second memory containing a secondary memory storing a plurality of test programs, and a control system for determining which of the plurality of test programs to copy to the first memory based on one or more factors, and which of the test programs to copy from the second memory to the first memory based on one or more factors. The system may include one or more of the aforementioned features individually or in combination. One or more factors may be, or include, user input, identifiers of devices being tested by one or more test programs, and / or identifiers of manufacturers of devices being tested by one or more test programs.

[0012] The control system may be configured to receive information to authenticate the user before copying one or more of the test programs into the first memory. The test programs in the secondary memory may be encrypted.

[0013] Any two or more features described herein, including an overview of the present invention, may be combined to form implementations not specifically described herein.

[0014] At least some of the devices, systems, and processes described herein may be configured or controlled by executing instructions stored in one or more non-temporary machine-readable storage media on one or more processing devices. Examples of non-temporary machine-readable storage media include read-only memory, optical disk drives, memory disk drives, and random-access memory. At least some of the devices, systems, and processes described herein may be configured or controlled using a computing system consisting of one or more processing devices and memory storage instructions, which can be executed by one or more processing devices to perform various control operations. The devices, systems, and processes described herein may be configured, for example, through design, construction, composition, arrangement, installation, programming, operation, activation, deactivation, and / or control.

[0015] Details of one or more implementation configurations are described in the attached drawings and the following description. Other features and advantages will become apparent from the description and drawings, as well as from the claims. [Brief explanation of the drawing]

[0016] [Figure 1] This is a block diagram of exemplary components of a test system configured to store test data in primary or secondary memory on a tester. [Figure 2] This is a flowchart illustrating an exemplary process for storing test data in the primary or secondary memory of the tester. [Figure 3] This is a block diagram illustrating an exemplary test system. [Modes for carrying out the invention]

[0017] An exemplary test system includes primary memory for storing test data for use by a test program. However, primary memory has limited storage capacity. Therefore, the test system also includes secondary memory, which may be separate from or different from primary memory, and may have a larger storage capacity than primary memory. Test data can be stored in the secondary memory unit until it is needed, at which point all or part of the test data is transferred from secondary memory to primary memory. The exemplary process described herein uses historical data on the execution of the test program to anticipate when the test data or part thereof will be needed. Based on this prediction, the process moves the test data or part thereof from secondary memory to primary memory, and then retrieves the test data from primary memory. As a result of this prediction, delays in the execution of the test program can be reduced. For example, the test data may be available from secondary memory and do not need to be acquired from a remote source. Furthermore, the test data may be loaded from secondary memory to primary memory for use when it is anticipated that the test data will be needed.

[0018] Figure 1 is a block diagram of an exemplary component 100 of a test system configured to test a device under test (not shown). Component 100 may be part of an automated test apparatus (ATE) 300, an example of which is described below with respect to Figure 3. Component 100 includes a tester 106 and a control system 99.

[0019] The control system 99 may be a computing system of the type described herein, for example, with respect to Figure 3, which includes memory for storing a test program and data 102 and a prediction module 104. The test program and data 102 and the prediction module 104 include executable software and / or data.

[0020] The tester 106 may be a hardware device. For example, the tester 106 may be one test instrument 124 or more test instruments, or may include both. The test instrument may include a memory, as described below, for storing test data and one or more test programs, and one or more processing devices and / or circuit configurations 140 for executing one or more test programs using the test data to test one or more DUTs. Examples of processing devices are described herein.

[0021] In some implementations, the memory on the test device 124 includes a primary memory 126 and a secondary memory 128. The primary memory 126 may be a dynamic random access memory (DRAM) cache or any other suitable type of storage device on which test data can be stored. The secondary memory 126 may also be a DRAM cache or any other suitable type of storage device on which test data can be stored. In some implementations, the secondary memory 128 may have a larger storage capacity than the primary memory 126. For example, the secondary memory 128 may have a storage capacity greater than twice the storage capacity of the primary memory 126, the secondary memory 128 may have a storage capacity greater than three times the storage capacity of the primary memory 126, the secondary memory 128 may have a storage capacity greater than four times the storage capacity of the primary memory 126, and so on.

[0022] In some implementations, the primary memory 126 can be directly accessed by the control system 99. For example, the control system 99 may store a test program and / or test data in the primary memory 126 and retrieve them from the primary memory 126. This may be done independently of or without permission from the processing devices and / or circuit configurations 140. In some implementations, the secondary memory 128 cannot be directly accessed by the control system 99. Therefore, in order to access the secondary memory 128, the control system 99 must instruct the processing devices and / or circuit configurations 140 on the test equipment 124 to store a test program and / or test data in the secondary memory or move it there. For example, the control system instructs the processing devices and / or circuit configurations 140 on the test equipment 124 to store a test program and / or test data in the secondary memory or retrieve it there. The processing devices and / or circuit configurations 140 on the test equipment 124 access the primary and secondary memories to perform the instructed actions. In some implementations, both the primary memory 126 and the secondary memory 128 may be able to directly access the control system 99.

[0023] The test program and data 102 includes data and executable instructions that constitute one or more test programs for testing the DUT. In this example, the test program and data 102 also includes sheet 108, historical travel data 110, and pattern data 112.

[0024] Sheet 108 contains information used by the test system to perform tests on the DUT. In this regard, each test program may require several classes of data to describe the DUT, the test system, and the test settings. The sheet includes this information in, for example, a spreadsheet format. For example, the sheet specifies a set of data for the physical interconnection of the DUT logic pins and the test equipment. The sheet specifies the configuration of the test system, for example, the test equipment physically placed within the test system. The sheet also specifies each test performed on the DUT and the order in which those tests, for example, the test flow, are to be advanced. The sheet may contain a flow table referring to pattern data and executable instructions that are part of one or more test programs for testing the DUT.

[0025] The pattern database 112 stores test data such as pattern data for each test program. The pattern data includes an input (or test) data array that enables the test system to distinguish between correct DUT behavior and incorrect DUT behavior caused, for example, by a defect within the DUT when applied to the DUT by the test program.

[0026] The history run data 110 includes data collected from past executions of the test programs defined by the test programs and data 102. For example, the data may be collected from dozens, hundreds, thousands, or more past executions of the test programs on the same type of DUT. The same type of DUT may include, for example, microprocessors by different or the same manufacturers. In this regard, the test programs are typically designed by the DUT manufacturer for a specific DUT, but this need not always be the case. For a test program designed by the DUT manufacturer for a specific DUT, the DUT may consist of the same type and the same manufacturer.

[0027] The historical run data 110 may include, for example, one or more consecutive operations that the test program takes when executing a test. For example, the sheet may define a branch in the test flow of the test program. A branch indicates what operation is performed under specific conditions at the start of the branch. For example, the sheet may define a loop in the flow of the test program. A loop indicates how many times a set of operations is performed based on specific conditions associated with the loop. The historical run data may also show, for example, loops, branches, etc., and their frequencies taken before the execution of the test program.

[0028] The historical travel data 110 may be stored as a binary file and may be used as training data when predicting future flow executions. For example, the historical travel data may be used to train a machine learning model as described herein.

[0029] The prediction module 104 is a programming module consisting of executable instructions configured to predict which operations will be performed when testing a DUT based on the test program and data 102, and in particular, historical run data. In some implementations, the prediction module 104 is configured to use pattern execution predictions based on a probabilistic model to predict pattern load arrays that will achieve repeatable test times and thereby improve the cost of testing (COT). The cost of testing may include, for example, the amount of money required to test each DUT, the time required to test each DUT, and / or other metrics that affect test performance.

[0030] In some implementations, the prediction module 104 includes a prediction model 114, a prediction database 116, and a digital device driver 118. The prediction model 114 may be a computer program consisting of executable instructions that define a machine learning model, the digital device driver 118 may also be a computer program consisting of executable instructions, and the prediction pattern database 116 may be a collection of data and / or executable instructions.

[0031] The prediction model 114 is configured to receive a sheet 108 related to the test program, analyze the flow of the test program based on the sheet, and generate and update a prediction table based on the probability of each bend (e.g., branch, loop, etc.) point in the test program. For example, the prediction model 114 is configured to perform a static analysis of the test program based on the sheet 108. The prediction model 114 tracks each path through the test program based on the sheet 108, including cases where the program separates into multiple branches or returns to a loop. The prediction model 114 may thus define all possible paths through the test program and generate a record of all possible paths through the test program.

[0032] When generating records, the prediction model 114 uses historical travel data 110 to define the most likely routes through test programs and assign weights to individual routes based on the likelihood of each route being taken. The likelihood of those routes being taken is based on historical data. For example, routes through test programs with less historical data are less likely to be taken now or in the future. Conversely, routes through test programs with more historical data are more likely to be taken now or in the future. Accordingly, routes with a lower likelihood of being taken are assigned a lower weight than routes with a higher likelihood of being taken. Conversely, routes with a higher likelihood of being taken are assigned a higher weight than routes with a lower likelihood of being taken.

[0033] In some implementations, the prediction model 114 may store and train a machine learning model based on historical data. For example, a record of all possible paths through a test program weighted based on historical travel data may be applied to train a machine learning model, for example, to generate a weighted model (also called the “trained model”). The weighted model can be used to predict, based on the weighted paths, which path is most likely to be taken through the test program now or in the future. For example, the machine learning model may be trained based on data collected from past runs of the test program described above and may include a Markov model that relies on a stochastic process to predict future runs for current pattern data. For example, the machine learning model may include a Bayesian model from which the resulting values ​​can be simulated from a posterior predictive distribution, which is a distribution of unobserved (future) pattern data given to the historical data. For example, the prediction database 116 may include a probabilistic model that can be trained based on data collected from past runs of the test program described above. The probabilistic model may be any suitable one or more machine learning or trainable models, or may include them.

[0034] The prediction database 116 may be a relational database or a multidimensional database, or may include both. The prediction database 116 may store, for example, a machine learning model trained by the prediction model 114. In some implementations, the prediction database may store prediction data obtained by applying the machine learning model to the current pattern data.

[0035] In this example, the digital device driver 118 is software that includes a pattern loader 120 and a pattern load thread 122.

[0036] The pattern loader 120 is configured to receive test program pattern data 112 from memory. The pattern loader 120 is configured to send the pattern data to a prediction model 114 or prediction database 116 so that a weighted machine learning model can be applied to the pattern data to predict which pattern data will be needed during the execution of the test program. Predictions of the pattern data needed for the execution of the test program may be obtained, for example, from the prediction model 114 and / or prediction database 116 and sent back to the pattern loader.

[0037] The pattern load thread 122 may be a background thread that manages the movement of test patterns and program instructions between two or more memories 126, 128 within the tester 106. This thread attempts to keep the pattern data that the test program is most likely to need for its next processing in primary memory. To this end, the pattern load thread 122 is configured to load pattern data for the test program into primary memory 126 or secondary memory 128 based on the likelihood and timing of when the pattern data will be needed. As described, the control system 99 may instruct the processing devices and circuit configurations 140 on the tester 106 to store in secondary memory 128 any instructions that are executable if they are their current destination.

[0038] Figure 2 is a flowchart of an exemplary process 200 for storing test data (e.g., pattern data) related to a test program in primary memory 126 or secondary memory 128 based on the expected path through the test program. In some implementations, the process 200 may be performed by a control system 99 in combination with memory on the tester 106 and processing devices and circuit configuration 140.

[0039] Process 200 includes a prediction module 104 that receives a sheet 108 for the test program and historical run data for the test program (202). The prediction model 114 performs a static analysis of the test program based on the sheet and the executable instructions of the test program mentioned in the sheet (204). Static program analysis is an analysis of a computer program performed without executing the program. In some implementations, the static analysis may define all paths that can be taken through the test program during the execution of the assumed test program.

[0040] The prediction model 114 generates a record that defines the paths that can be taken through the test program (206). As described above, the record weights the paths through the test program based on historical driving data, with less likely paths being assigned lower weights than more likely paths, and more likely paths being assigned higher weights than less likely paths. In some implementations, when performing static analysis, the prediction model 114 may use a probabilistic model that predicts the optimal pattern load sequence to reach a repeatable test time.

[0041] The records generated by the prediction model 114 are used to train a prediction model, such as a machine learning model (208), to predict future executions of test program instructions (e.g., for the same or similar DUT) based on historical data. An example of a machine learning model is given above. The trained model may be stored in a prediction database 116 in some implementations.

[0042] Process 200 includes using a trained model to predict the path taken through the test program during the planned execution of the test program using given pattern data (210). The prediction model may generate prediction data for each branch encountered. Pattern loader 120 obtains the prediction data for the path (212). For example, pattern loader 120 may request the prediction model 114 to retrieve the prediction data for the path from the prediction database 116.

[0043] Process 200 includes storing test data, such as pattern data related to the test program, in primary memory 126 or secondary memory 128 based on expected data (214). For example, the pattern load thread 122 may load test data that may be required by the execution of the test program into primary memory 126, based on, for example, the probability that the test program is likely to take a path that requires test data. For example, the pattern load thread 122 may load test data that is less likely to be required by the execution of the test program into second memory 126, based on, for example, the probability that the test program is unlikely to take a path that requires test data. Specifically, the pattern load thread 122 may instruct the processing device and / or circuit configuration in the tester 106 to store the test data in secondary memory.

[0044] The pattern loader thread 122 may, for example, continuously evaluate the status of executed pattern data in primary memory 126. This may be done, for example, by monitoring, based on the execution of a test program running on the test equipment and / or control system. Once it determines which pattern data stored in the secondary memory cache 128 needs to be executed, the pattern loader thread 122 may begin moving the pattern data to primary memory 126 for execution, for example by instructing the processing devices and / or circuit configurations in the tester 106. The pattern loader thread 122 may, for example, move at least a portion of the test data from secondary memory to primary memory (or back) based on the planned execution time. This may be done, for example, when it is expected that the data from secondary memory will be needed for the current execution of the test program. That is, since the test program accesses that data from primary memory, it may be necessary to move the test data from secondary memory to primary memory if the test data from secondary memory is needed for the current execution. The pattern loader thread 122 may also move at least a portion of the test data from primary memory to secondary memory before the planned execution. This may be done, for example, when data from primary memory is not needed for the current execution of the test program and the storage space in primary memory needs to be freed. In this regard, at least some of the test data may be moved from primary memory to secondary memory when primary memory does not have sufficient storage capacity.

[0045] In some implementations, the pattern loader thread 122 may be evaluated when the test program attempts to execute pattern data that is not loaded into either the primary memory 126 or the secondary memory 128. This is considered a pattern miss. The pattern loader thread 122 may communicate the pattern miss back to the pattern of the prediction model 114 via the pattern loader 120. This may include process 200 updating the prediction model (216) using data from recent executions of the test program. The prediction model 114 may update the data in the prediction database 116 based on the history from recent executions of the test program and update any pattern misses to improve predictions in subsequent runs. Furthermore, the prediction model 114 may update the history run 110 data to reflect changes in the current run. The prediction model may use the updated information included in its training to fine-tune its prediction probabilities.

[0046] In some implementations, the secondary memory 128 may store one or more test programs (referred to herein as a library of test programs) that can be loaded into primary memory for execution when needed. For example, after a test program has been loaded into secondary memory 128 as described herein, the test program may be copied to primary memory 126 for execution, and the copy of the test program may remain / persist in secondary memory 128 for later use. This may apply to multiple test programs, such that secondary memory 128 stores a library of test programs that can be copied / loaded into primary memory 126 when needed for execution. As a result, when a new device is being tested, it may be tested using an existing test program in secondary memory 128, as it is not necessary to load a new test program into the tester for execution, thereby potentially reducing test time. The existing test program may be selected by the control system to test a new device based on, for example, user input, an identifier for the device being tested, an identifier for the device manufacturer, the tests to be performed, and / or other appropriate factors.

[0047] In some implementations, access to one or more test programs in secondary memory 128 may be restricted based on an external access function. Access may require a user, such as a test technician or manufacturer, to provide authentication information associated with one or more test programs before granting access to those test programs and copying them to primary memory. For example, before accessing a test program in secondary memory 128, the user may be prompted by the tester, for example, via an associated or attached computing system, to enter one or more identifiers, such as a username and password. In some implementations, only after the username and password have been authenticated can the user execute a test program from a library associated with that username and password (for example, copy the test program to primary memory for execution).

[0048] In some implementations, test programs in secondary memory 128 may be encrypted. For example, test programs may be encrypted using a public key. Only holders of the private key capable of decrypting the encrypted test programs may be able to execute these test programs (e.g., copy the test programs to primary memory for execution). For additional security, encryption may be combined with a username and password as described above. In another example, encryption of a secret symmetric key may be used to encrypt test programs in secondary memory 128. The encryption system follows the NIST (National Institute of Standards and Technology) guidelines for key management systems.

[0049] Figure 3 is a block diagram showing the components of an exemplary ATE300, including a test device (referred to herein as a “tester”) 301 and a control system 302. Component 100 in Figure 1 may also be part of the ATE300. Tester 301 may be an implementation of tester 106, and control system 302 may be an implementation of control system 99.

[0050] The tester 301 includes a test head 303 and a device interface board (DIB) 304 physically and electrically connected to the test head 303. In this example, the DIB 304 includes a circuit board with mechanical and electrical interfaces at site 305. One or more DUTs, such as DUT 308, are connected to each of their sites for testing by the ATE. The DIB 304 may include, among other things, connectors, conductive traces, conductive layers, and circuit configurations to determine the routing of signals between the test equipment in the test head 303 and the DUTs connected to the DIB sites and other circuit configurations in the ATE. Power, including voltage and current, may flow through one or more layers in the DIB to the DUTs connected to the DIB.

[0051] The test head 303 includes multiple test instruments 311a to 311n, each of which may be configured to perform tests and / or other functions as needed. Although only four test instruments are shown, the ATE 300 may include any appropriate number of test instruments, including one or more located outside the test head 315.

[0052] The test equipment may be a hardware device, which may include one or more processing devices and / or other circuit configurations. The test equipment may be configured, for example, programmed, to output commands to test the DUT held on the DIB. The commands to test the DUT may be, or include, instructions, signals, data, parameters, variables, test patterns, and / or any other information designed to elicit a response from the DUT. One or more, for example all, of the test equipment may be configured to receive from the DUT in response to commands sent from the ATE to the DUT. The response is in the form of response data. The test equipment may be configured to analyze the response data to determine whether the DUT passed or failed the test. The test equipment may be configured to send the response data to the control system 302 for analysis according to process 200.

[0053] In some implementations, all or part of process 200 may be performed using one or more test devices. To give a representative example of all test devices, test device 311n is given, and each test device (or a subset thereof) may include one or more processing devices and / or other circuit configurations 350, a primary memory 351, and a secondary memory 352, as shown in Figure 1. In some implementations, functions attributable to the control system 99 may also be implemented on the test device, or instead implemented on the test device, for example, the test device may include test and program data 102 and a prediction module 104 operating as described herein.

[0054] Test channel 315 is configured between the test head and the DIB to enable communication between the DUT and the test equipment. Although only four test channels are shown in Figure 3, any number of test channels may be included, for example, one or more test channels per DUT.

[0055] The control system 302 is configured and programmed, for example, to communicate with the test equipment 311a-311n to instruct and / or control the testing of the DUT. In some implementations, the link for this communication 320 may be via a direct connection, such as a high-speed serial bus of the type described herein. In some implementations, the communication link may be via a network. In some implementations, the communication link may be considered as part of one or more test channels. In some implementations, the communication link may not be considered as part of one or more test channels.

[0056] The control system 302 may be configured to provide test programs, commands, and / or test data (e.g., test patterns) to test instructions 311a to 311n in the test head, and its test equipment is used to test the DUT. The control system 302 may also be configured to receive response data from the test equipment and to analyze the response data to determine whether the DUT passed or failed the test. The control system 302 may also be configured to perform all or part of the operation process 200 shown in Figure 2 above. Accordingly, the control system 302 may include a memory 332 for storing the expected model 353 and test and program data 354, as shown in Figure 1. In some implementations, the functions attributed to the control system 302 may be implemented in each test equipment or distributed among two or more test equipment.

[0057] Although process 200 describes storing data between two memory locations, process 200 may be used to store data between three or more memory locations. For example, process 200 may be used to move data between two secondary memory locations and one primary memory location, thereby increasing the storage capacity on the tester.

[0058] All or part of the test systems and processes described herein, and various modifications thereof, may be configured or controlled, at least in part, by one or more computers, such as control systems 99 or 302, using one or more computer programs explicitly embodied on one or more information carriers, such as one or more non-temporary machine-readable storage media. The computer programs may be written in any form of programming language, including edited or translated languages, and may be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. The computer programs may be deployed to be executed on one or more computers, distributed at one or across multiple sites, and interconnected and run via a network.

[0059] The actions relating to the configuration or control of the test systems and processes described herein may be performed by one or more programmable processors executing one or more computer programs to control or perform all or part of the operations described herein. All or part of the test systems and processes may be configured or controlled by dedicated logic circuit configurations such as FPGAs (Field Programmable Gate Arrays) and / or ASICs (Application-Specific Integrated Circuits), or by embedded microprocessors localized to the equipment hardware.

[0060] Processors suitable for executing computer programs include, for example, both general-purpose and specific-purpose microprocessors, as well as any one or more processors of any type of digital computer. Generally, processors receive instructions and data from read-only storage, random-access storage, or both. The elements of a computer include one or more processors for executing instructions, and one or more storage devices for storing instructions and data. Generally, a computer also includes one or more machine-readable storage media for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or is operablely coupled to receive data from them, transmit data to them, or both. Non-temporary machine-readable storage media suitable for realizing computer program instructions and data include all forms of non-volatile storage areas, such as semiconductor storage devices including EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electronically Erasable Programmable Read-Only Memory), and flash memory devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, CD-ROM (Compact Disc Read-Only Memory), and DVD-ROM (Digital Multipurpose Disc Read-Only Memory).

[0061] Elements of the different implementations described may be combined to form other implementations not specifically described previously. Elements may be excluded from the previously described systems without adversely affecting the operation of the system or the system as a whole. Furthermore, various distinct elements may be combined into one or more individual elements to perform the functions described herein.

[0062] Other implementations not specifically described herein are also within the scope of the following claims.

Claims

1. It is a system, A first memory including primary memory, A second memory including a secondary storage device, Control system and Includes, The control system predicts the path taken through the test program during the planned execution of the test program, and stores test data related to the test program in the first memory or the second memory based on the predicted path.

2. The system according to claim 1, wherein storing the test data includes storing the test data in the first memory.

3. The aforementioned test data is stored, Moving at least a portion of the test data from the first memory to the second memory before the planned execution, To move at least a portion of the test data from the second memory to the first memory and back based on the planned execution time. The system according to claim 2, including the above.

4. The system according to claim 3, wherein at least a portion of the test data is moved from the first memory to the second memory based on the storage capacity of the first memory.

5. The system according to claim 4, wherein at least a portion of the test data is moved from the first memory to the second memory before the planned execution when the first memory is full.

6. At least a portion of the test data is moved from the first memory to the second memory when the first memory does not have sufficient storage capacity for both the at least portion of the test data and the other test data. The system according to claim 4, wherein the other test data is expected to be required by the test program prior to at least a portion of the test data.

7. The system according to claim 3, wherein at least a portion of the test data is moved from the second memory to the first memory and back at a time prior to the planned execution time.

8. The system according to claim 2, wherein the second memory has a larger storage capacity than the first memory.

9. The control system according to claim 1, wherein the control system includes a model for predicting the path through the test program.

10. The system according to claim 9, wherein the model includes at least one of a probabilistic model, a Markov model, a Bayesian model, or a machine learning model.

11. The system according to claim 9, wherein the model includes parameters obtained through static analysis of the test program.

12. The aforementioned parameters are weighted based on historical data, The system according to claim 11, wherein the historical data is based on the prior execution of the test program.

13. The control system is The static analysis of the test program is performed, wherein the static analysis defines a first path that can be taken through the test program during the execution of the assumed test program. Obtaining historical data based on a second path taken through the test program during past executions of the test program, To generate a model, the first path is weighted based on the historical data. It is configured to perform operations including, The system according to claim 1, wherein predicting the aforementioned path through a test program is performed based on the aforementioned model.

14. Performing the static analysis includes defining the branches within the test program. The aforementioned branch is divided into multiple paths that the test program can follow. The system according to claim 13, wherein the first path includes the plurality of paths.

15. The test program is packaged together with the historical data and the test data. The system according to claim 13, wherein the historical data is obtained from the test program.

16. It is a method, The static analysis of the test program is performed, and the static analysis is performed to define a first path that can be taken through the test program during the execution of the assumed test program. Obtaining historical data based on a second path taken through the test program during past executions of the test program, To generate a model, the first path is weighted based on the historical data, The model is used to predict a third path taken through the test program during the planned execution of the test program, Based on the third path, test data related to the test program is stored in memory. Methods that include...

17. The memory includes a first memory and a second memory, Storing the test data includes storing the test data in the first memory, The method according to claim 16, wherein at least a portion of the test data is retrieved from the first memory during the execution of the test program.

18. Before the planned execution, move at least a portion of the test data from the first memory to the second memory, Based on the planned execution time, move at least a portion of the test data from the second memory to the first memory and back. The method according to claim 17, further comprising:

19. The method according to claim 18, wherein at least a portion of the test data is moved from the first memory to the second memory when the first memory is full.

20. At least a portion of the test data is moved from the first memory to the second memory when the first memory does not have storage capacity for both the at least portion of the test data and the other test data. The method of claim 19, wherein the other test data is expected to be required by the test program prior to at least a portion of the test data.

21. The method according to claim 18, wherein moving at least a portion of the test data from the second memory to the first memory and back occurs before the scheduled execution time.

22. The method according to claim 17, wherein the second memory has a larger storage capacity than the first memory.

23. Performing the static analysis includes defining the branches within the test program. The aforementioned branch is divided into multiple paths that the test program may follow. The method according to claim 16, wherein the first route includes the plurality of routes.

24. The method according to claim 16, wherein the model includes at least one of a probabilistic model, a Markov model, a Bayesian model, or a machine learning model.

25. The test program is packaged together with the historical data and the test data. The historical data is obtained from the test program, according to the method of claim 16.

26. It is a system A first memory including primary memory, A second memory including a secondary storage device, wherein the second memory stores a plurality of test programs, Control system and Includes, The control system is a system that determines which of the plurality of test programs to copy to the first memory based on one or more factors, and which of the plurality of test programs to copy from the second memory to the first memory based on one or more factors.

27. The system according to claim 26, wherein the one or more of the factors include user input.

28. The system according to claim 26, wherein the one or more factors include an identifier for a device being tested by the one or more test programs.

29. The system according to claim 26, wherein the one or more factors include an identifier of the manufacturer of the device being tested by the one or more test programs.

30. The system according to claim 26, wherein the control system is configured to receive information to authenticate the user before copying one or more of the plurality of test programs to the first memory.

31. The system according to claim 26, wherein the plurality of test programs in the secondary memory are encrypted.

Images