299 results about "Test suite" patented technology

A method for automatically maintaining a database that stores coverage data for testing software code under development is provided. The method includes providing modified instrumented software and using test cases of a test suite to execute the modified instrumented software. Modified coverage data that includes a call tree defining a path of each test case is created. The method also includes replacing the coverage data with the modified coverage data at scheduled intervals.

A generic testing framework to automatically allocate, install and verify a given version of a system under test, to exercise the system against a series of tests in a “hands-off” objective manner, and then to export information about the tests to one or more developer repositories (such as a query-able database, an email list, a developer web server, a source code version control system, a defect tracking system, or the like). The framework does not “care” or concern itself with the particular implementation language of the test as long as the test can issue directives via a command line or configuration file. During the automated testing of a given test suite having multiple tests, and after a particular test is run, the framework preferably generates an “image” of the system under test and makes that information available to developers, even while additional tests in the suite are being carried out. In this manner, the framework preserves the system “state” to facilitate concurrent or after-the-fact debugging. The framework also will re-install and verify a given version of the system between tests, which may be necessary in the event a given test is destructive or otherwise places the system in an unacceptable condition.

A system for automatic generation of test programs employs test generation foci linked to a finite state machine behavioral model of a software application under test to produce abstract test suites that are executed by an execution engine. The foci include directive expressions that tag coverage variables of the behavioral model. The execution engine operates until the tagged coverage variables have assumed all their possible values. Both the behavioral models and the foci are archivable and can be retrieved and reused independently.

A method of automatically performing a component test at any number of locations in a distributed environment is disclosed. In general, in order to assure compatibility of the various components in an enterprise computing system, a service test is created as part of a compatibility test suite and passed to a test application server having a test application program. The test application program includes a generic vehicle class that includes a plurality of vehicle class invokers each of which is used to implement each of the object types that are run in each of a plurality of containers. During the build process of the CTS, each service test is automatically packaged with each of the appropriate vehicle classes so that each can be deployed into and run within the associated container.

A generic testing framework to automatically allocate, install and verify a given version of a system under test, to exercise the system against a series of tests in a “hands-off” objective manner, and then to export information about the tests to one or more developer repositories (such as a query-able database, an email list, a developer web server, a source code version control system, a defect tracking system, or the like). The framework does not “care” or concern itself with the particular implementation language of the test as long as the test can issue directives via a command line or configuration file. During the automated testing of a given test suite having multiple tests, and after a particular test is run, the framework preferably generates an “image” of the system under test and makes that information available to developers, even while additional tests in the suite are being carried out. In this manner, the framework preserves the system “state” to facilitate concurrent or after-the-fact debugging. The framework also will re-install and verify a given version of the system between tests, which may be necessary in the event a given test is destructive or otherwise places the system in an unacceptable condition.

A pre-designed system-on-chip architecture and method includes several standard library devices, HDL source code, simulation environment and regression, synthesis scripts, software header files, software libraries, ASIC verification test suites, and makefiles. The standard library devices comprise an integrated CPU, a shared memory controller, a peripheral controller, system peripherals, a DMA controller, embedded memory, and general system control. CPU bridges are used to accommodate a variety of processor types and to insulate users from the complexities of interfacing to different kinds of processors. Such CPU bridges further allow the latest processors to be rapidly integrated into existing integration platforms and designs.

A test manager software program includes an interactive test graphical-user-interface (GUI) for testing network devices using high-level networking commands. The test GUI allows the test engineer user to enter high-level commands such as Simple Networking Management Protocol (SNMP) commands that read values in a management information database in a network device under test. The high-level commands can be sent from the test manager using a command-line interface (CLI) in a telnet session opened to the network device during testing. The user specifies high-level test, analyze, and restore commands in test cases that are collected into test suites. Rules for logging on to the network device under test are stored that include expected prompts from the network device and user responses such as passwords. Addresses of the network device under test can be re-mapped for testing other devices so the test suites can be reused.

An execution engine synchronizes test suite execution to run multiple independent operations concurrently or in parallel. The execution engine operates by stepping through the test suite. At each step it runs the required number of independent operations and waits for their completion. It then checks the results, and continues to the next step of the test suite. If at any step, the actual results are different from the expected results, then the exact sequence that triggered the fault is known. The sequence can be repeated when a correction is provided for the fault. Furthermore, the execution engine allows the user to interactively step through a test case when debugging a fault. A synchronization pattern generator may be incorporated in the system to generate minimal numbers of repetitions of test cases to guarantee varying degrees of interaction coverage.

Coverage metrics are expressed with an intuitive graphical interface based upon data flow. Coverage analysis and presentation objects are integrated to produce coverage results which enable device functionality in a device under test to be modeled as objects, subject to event occurrence. Event objects are introspected at run-time, allowing the user to determine the event object's attributes with specification of coverage metrics subject to a selected combination of the event object's attributes. The event objects are serialized into permanent storage, allowing the user to specify and execute new coverage metrics at any time after simulation. Operations used to describe coverage metrics are modeled as analysis objects. Such analysis objects accept event objects as inputs, using a predetermined, well-defined interface. The combination of event objects and analysis objects allows coverage metrics to be specified in a simple data flow manner. With such a coverage metric, the user attaches or wires (metaphorically) the analysis objects together in a visual builder environment. Using the analysis objects, the user specifies desired coverage metrics, such as coverage of sequences of events and / or coverage of events that occur during the same time window of a simulation. The display functionality of the coverage tool is expandable because the presentation objects use the same event object interface as the analysis operator objects. Coverage metrics are subject to specification either before or after event occurrence. The user specifies coverage metrics using an intuitive graphical interface based upon data flow, without any specific programming language skills being necessary. Functional events in the device under test are treated as event objects. Each event object may be passed to selected analysis tools chosen by the user, such as analyzers, logic gates, and coincidence counters.

In an arrangement for testing multiple low-end computing devices simultaneously, different tests are executed simultaneously on different platforms using a single instance of a test harness to which multiple devices are connected. A platform-specific API is provided for independent components of the tests and platform-specific components are implemented for each test according to the respective platform-specific API. At run-time the test harness deploys each test together with a platform-specific execution agent, configured according to the components of the test. The agents execute the test suites, and return test results to the test harness.

Systems and techniques relating to evaluating performance characteristics are provided. One or more fitness devices, for example, a ball and / or a mat, may have sensors for measuring a user's actions during an activity protocol, such as a testing drill. Sensor configured to detect if the fitness device meets operational criteria are disclosed. In one embodiment, the fitness device may comprise a ball and the operational criteria may relate to the weight, dimensions, and / or internal pressure of the ball. In one embodiment, data from fitness device(s) may be used to calculate a score. In other embodiments, a score may be calculated; however, it may not be associated with credentials or certified if an operational criteria is not met.

Graphical user interface testing is provided. User interface (UI) build data and text data are transformed into a testable data format, such as XML, by a UI parser. The transformed UI data may be stored to a backend server where stored procedures and functions may be utilized to analyze the UI data against build differencing procedures, command mapping procedures, comparison to previous or subsequent user interface builds, etc. Additional stored procedures may allow UI testers to query data, create test suites and record testing information for a given UI. A front end testing module may provide a testing user an interface to query the backend database for information on various UI components and to review results for tests conducted on UI data. The front end testing module may also provide an interface for allowing testers to generate and execute new tests for a given user interface.

A verification system for a procedure interface is generated by using formal specifications of the procedure interface and generating test suites. The test suites are generated from the formal specifications and templates or skeletons which are used to generate an element of a verification system. The skeletons are generated based on decomposition of test suits.

A method and system for testing a software application. A description of a test suite for testing the software application being tested (ABT) is inserted into a test database. The ABT is invokes multiple external software applications during execution of a test script of the test suite. Each external application invoked by the ABT is replaced by a corresponding simulator during execution of the test script. Output data to be returned to the ABT by each invoked simulator is inserted into the test database, after which each test script of the test suite is executed. The executing includes: sending a request, by the ABT to each simulator invoked in each test script, for requested data; and receiving, by the ABT, the requested data from each simulator invoked in each test script. The received requested data includes the output data that had been inserted into the test database.

A test coverage tool provides output that identifies differences between the actual coverage provided by a test suite run on a program under test and the coverage criteria (e.g., the coverage criteria required by the test / development team management). The output from the test coverage tool is generated in the same language that was used to write the coverage criteria that are input to an automated test generator to create the test cases which form the test suite. As a result, the output from the coverage tool can be input back into the automated test generator to cause the generator to revise the test cases to correct the inadequacies. This allows iterative refinement of the test suite automatically, enabling automated test generation to be more effectively and efficiently used with more complex software and more complex test generation inputs.In preferred embodiments, test coverage analysis results of several different test suites, some manually generated and others automatically generated, are used to produce a streamlined automatically-generated test suite and / or to add missing elements to an automatically generated test-suite.

This invention is comprised of a system, protocol, method and apparatus for the assessment of properties that may have been, or are being, subject to clandestine drug manufacturing and / or processing activities. The invention includes a comprehensive home test kit to be used in or upon a suspect premises to detect, identify, and delineate toxic chemical hazards that may have originated from an illegal drug making operation. The test kit is designed to be conveniently equipped with all-inclusive content consisting of an assortment of user-selected sampling equipment, media, containers, materials, documentation, instruction manual, as well as an audio-visual instructional media pack. This Kit is designed to enable a person of average intelligence to conduct the sampling activities and the Kit to a designated analytical laboratory for processing and reporting of results in order to determine the risk presented by a property and damages it may have sustained.

A method and a system evaluate an efficacy of a test suite in a model-based diagnostic testing system to determine a revision of the test suite. The evaluation comprises suggesting a test to be added to the test suite based on probabilities of a correct diagnosis and an incorrect diagnosis. The evaluation either alternatively or further comprises identifying a test to be deleted from the test suite based on probabilities of a correct diagnosis for the test suite and for a modified test suite that does not include a test. An efficacy value of each test in the test suite is computed. The system has a computer program comprising instructions that implement an evaluation of diagnostic efficacy of the test suite. The system either is a stand-alone system or is incorporated into the testing system.

A system provides for fuzzy classification in comparisons of scanner responses. A web application test suite performs tests against a web application by sending client requests from a testing computer to the server running the web application and checking how the web application responds. A thorough web application security scan might involve thousands of checks and responses. As a result, some sort of programmatic analysis is needed. One such evaluation involves comparing one response against another. Response matching that compares two HTTP responses might use fuzzy classification processes.

A method and system for generating an optimized suite of test cases includes deriving a set of use case constraints and generating a test suite based upon the set of use case constraints.

An intelligent test system for testing software code is provided. The intelligent system includes a profiled software component, a test executor, a database, a database updater, and a data analyzer / optimizer. The test executor creates a coverage data by using the profiled software component to execute a plurality of test cases of a test suite. The database stores the coverage data and the database updater is configured to store the coverage data into the database. The data analyzer / optimizer analyzes the coverage data contained in the database to find test cases affected by a modification to the software code. The data analyzer / optimizer also optimizes the test cases affected by the modification by selecting fewer test cases that satisfy a criteria. The test executor uses the test cases that satisfy the criteria to test the modification to the software code.

A method of testing an embedded example in a graphical user interface documentation includes creating an extractable embedded example by tagging the embedded example, extracting the extractable embedded example from the graphical user interface documentation to generate an extracted example, selecting a tagged entity from the extracted example, interpreting the tagged entity to generate an interpreted tagged entity, creating a test suite using the interpreted tagged entity, selecting a graphical tool against which to execute the test suite, executing the test suite against the graphical tool to generate an output response, and verifying the output response.

A system stored on a computer-readable storage medium for testing computer software, including: a plurality of heterogeneous test tools; a testing application that provides an interface that enables a user to collectively access each of said plurality of heterogeneous test tools; and a plurality of modules that interact with the testing application for creating and modifying, via said interface, a test that is implemented with said plurality of test tools, including: a tool manager module for selecting one or more tools, from among said plurality of test tools, to be employed in a test; a case manager module for selecting test cases, each comprising an elemental executable test, that are performed by selected tools, for inclusion in the test; and a suite manager module for creating, modifying and selecting test suites, each comprising at least one group of selected test cases, to be executed during the test.

A method and apparatus is disclosed herein for determining whether a test case is to be included in a test suite. In one embodiment, the method comprises executing an application with test inputs of a test case during performance of software application testing and using a Finite State Machine (FSM) model of observable events generated from execution of other test cases in the test suite to determine whether to include the test case in the test suite for use in future testing of the application.

The invention discloses a regression test automation method which comprises the steps that: a test plan framework is generated and submitted to a test plan library; a regression test environment is constructed; test environment configuration and a topological mapping framework are generated and submitted to a regression test library; a regression test management module extracts the test plan framework and generates a test suite construction framework; a test suite management module constructs a test suite construction strategy according to the test suite construction framework, extracts test cases, generates a regression test suite and submits the regression test suite to the regression test library; the regression test management module detects the test plan and starts a test operating dispatcher; the test operating dispatcher informs a test environment management module to apply for test resources; the test operating dispatcher dispatches the regression test suite and starts a test operating engine; the test operating engine executes the regression test suite and generates test executing results in an operating result library. The invention also discloses a regression test automation system. The method and the system of the invention can reduce human intervention and realize regression test automation.

A mechanism for testing execution of applets with plug-ins and applications is described. In one embodiment, a data structure that specifies at least two applets of a plurality of applets is accessed. Based on the data structure that specifies the two applets, a plug-in that executes with an application is caused to attempt to execute each of the two applets. After causing the plug-in to attempt to execute the two applets, results are generated based on the attempt to execute each of the two applets. The mechanism may be implemented to test multiple test suites that each comprise one or more test cases that each include an applet, and reports may be generated for individual test cases, test suites, or a set of test suites. This embodiment of the present invention minimizes the time required for testing and allows for testing the effects of applets that are not visible to a user.

A system for testing a software build is presented. A current software build is compared to a reference software build, typically a known, previous build. The comparison identifies those areas in the current software build that have changed with regard to the reference software build. The identified areas are used by a coverage analysis process to determine a focused test suite to test the modified areas of the current build. The test coverage analysis uses information in a master test suite to determine the focused test suite. The focused test suite is used by a test process to test the modified areas of the current software build. The coverage analysis process may also identify those areas of the current software build that cannot be tested using the tests in the master test suite. A report is generated identifying those areas that are not covered by the focused test suite.

The invention provides automated test suite for compliance testing of cloud storage server to a Cloud Data Management Interface (CDMI) by performing functional testing of CRUD (Create, Read, Update, and Delete) operations. It offers a solution containing test scripts for validating the response from CRUD operations performed on CDMI objects and checks for the cloud storage to be CDMI compliant.

A system and method is disclosed to estimate both, the time and number of resources required to execute a test suite or a subset of test suite in parallel, with the objective of providing a balanced workload distribution. The present invention partitions test suite for parallelization, given the dependencies that exists between test cases and test execution time.