System and method for computer based creation of tests formatted to facilitate computer based testing

Abstract

A computer-based test creation system and method for generating test items. The test creation process includes creating items in electronic form and allowing access by a plurality of reviewers via a network. The reviewers may review the item for content, fairness and format, and collaborate with each other before indicating that the item is acceptable for inclusion in a test. After the administration of the items to test takers, item performance may be evaluated on-line by reviewing electronic feedback from the administrations and creating electronic evaluation reports. Since the items are created in an electronic form, the items may further include new forms of stimuli such as graphics and multimedia.

Description

CROSS REFERENCE TO RELATED APPLICATIONS [0001] This is a continuation-in-part of application Ser. No. 09 / 777,836, filed Feb. 5, 2001, which is, in turn, a continuation of application Ser. No. 08 / 826,417, filed Mar. 27, 1997.FIELD OF THE INVENTION [0002] This invention generally relates to the field of computer-based test creation systems, and more particularly, to the on-line creation of items for tests using a computer-based test creation system. BACKGROUND OF THE INVENTION [0003] For many years, standardized tests have been administered to examinees for various reasons such as for educational testing or for evaluating particular skills. For instance, academic skills tests, e.g., SATs, LSATs, GMATs, etc., are typically administered to a large number of students. Results of these tests are used by colleges, universities and other educational institutions as a factor in determining whether an examinee should be admitted to study at that particular institution. Other standardized test...

AI Technical Summary

Benefits of technology

[0017] An item performance evaluation subsystem is provided for evaluating the performance of the items after the administration of the items with the use of electronic evaluation reports. The item performance evaluation process includes the steps of electronically compiling the feedback from the administration of the items to test takers, electronically reviewing the feedback, identifying any problems in the test items, and resolving any problems in the test items.

[0018] The preferred embodiment of the invention includes an analysis feedback review subsystem for identifying items that do not meet predetermined performance criteria after the administration of the items to test takers and initiating evaluation reports for these items. The preferred embodiment of the invention also includes an item migration and review subsystem for converting items from the prior art TD / DC system into the electronic form of the test creation system. The preferred embodiment of the present invention further includes a metrics and management subsystem for checking the status and monitoring the progress of items throughout the item creation process.

[0019] An additional feature of the invention, that of statistical flags, is provided to automatically initiate evaluation reports for those items that do not meet the predetermined performance criteria. Another feature of the invention, present in the item creation and review subsystem, is a means for creating a plurality of multi-media item types. An additional feature of the invention, that of a pre-administration alert function, provides an alert notification if a problem item, i.e., one that was just subjected to a change in status through the item performance evaluation process, appears in any currently assembled tests.

Problems solved by technology

In recent years, the methods for creating, delivering, administering, and scoring tests have been determined to be inadequate.

Only a few attempts have been made, however, to automate only portions of the testing process.

Furthermore, these attempts are limited in their ability to generate a variety of item types.

For example, a prior art test creation system, “TD / DC,” is disclosed in U.S. Pat. No. 5,565,316, entitled “System and Method for Computer Basted Testing” and owned by Educational Testing Service, Princeton, N.J. The TD / DC system, however, has drawbacks in that the creation stage is not automated.

Since it is not automated, the entire test creation process with the TD / DC system involves many steps and “hand-offs” of the item / work folder.

Because of the hand-offs, much time is wasted waiting for a user to pass the work folder off to the next user.

When using the TD / DC system, it is not feasible for people to review an item at the same time to collaborate their thoughts.

Another drawback of the TD / DC system is that throughout the item creation process, everything is done on paper.

This makes it very difficult for a user to view the current state of the item.

Moreover, a user or reviewer never sees what the item will look like until it is viewed in a completed test, either in a computer based test or a test booklet.

Another drawback to the TD / DC system is that it cannot handle multimedia item types.

That is, because every step of the creation process is done on paper, multimedia items such as audio, video or animation items cannot be made with the TD / DC system.

Another drawback to the TD / DC system is that evaluation of the performance of items is not efficient.

The prior art system, i.e., the TD / DC system, lacks efficiency in that much of it is still paper based, it involves numerous steps and hand-offs, it lacks automatic or automated steps, and the item is never viewed as it is intended until it is present in its final form, i.e., in the computer based test or in a test booklet.

Images