Method and apparatus for monitoring and logging the operation of a distributed processing system

Abstract

Method and apparatus for monitoring and logging the operation of a distributed processing system. A method for monitoring the operation of a distributed transaction system that is operable to process one or more transactions, each of which is comprised of a plurality of discrete processes, and which transaction as a whole is operable to perform a transaction on data when transferring data from a first location on a network to a second location on the network and the transaction comprised of operating on the data at intermediate nodes in the system with one or more of the plurality of processes during the transaction. First, a determination is made as to when a transaction has been initiated from the first location and has been transferred to the one of the intermediate nodes in the network. The initiation of the transaction is then logged at a central location on the network. A determination is then made as to when the initiated transaction has been completed by transfer of the processed data to the second location on the network from the last of the intermediate nodes in the network that has operated on the data. Completion of the transaction is then logged at the central location on the network.

Description

CROSS REFERENCE TO RELATED APPLICATIONS [0001] This application is a Continuation of U.S. patent application Ser. No. 09 / 971,999, filed Oct. 5, 2001, entitled “Method and Apparatus for Monitoring and Logging the Operation of a Distributed Processing System,” which issues as U.S. Pat. No. 6,975,494 on Dec. 13, 2005, which is a Continuation of U.S. Pat. No. 6,778,648, which issued on Sep. 7, 2004, entitled “Method And Apparatus For Load Balancing a Distributed Processing System,” which is a Continuation-in-Part of U.S. patent application Ser. No. 09 / 887,494, filed Jun. 22, 2001, entitled “Method And Apparatus For Converting Data Between Two Dissimilar Systems” (Atty Dkt No. ATTA-25514), which is a Continuation-in-Part of U.S. patent application Ser. No. 09 / 879,571, filed Jun. 12, 2001, entitled “Method and Apparatus for Generating Unique Id Packets in a Distributed Processing System” (Atty Dkt No. ATTA-25,515), which is a Continuation-in-Part of U.S. patent application Ser. No. 09 / 841...

AI Technical Summary

Benefits of technology

The system enables efficient data classification, rapid search capabilities, and effective data transmission between dissimilar systems while minimizing CPU load and system errors, improving overall data management and processing efficiency.

Problems solved by technology

The classification and management of data is one of the most difficult tasks faced by corporations, government entities, and other large users of information.

Data exchanges face a bigger challenge in that they must work with multiple companies and develop a comprehensive classification system for their buyers.

However, no existing classification database is comprehensive enough to address all the issues associated with building a classification system.

These issues include: uniform numbers for products that cross multiple industries, restricting products from inclusion in classification, and non-usage of slang or industry standard language to access or classify products.

The classification databases frequently do not address all the products, thus resulting in inconsistencies even when companies use the same classification system.

Additionally, many of the various classification systems conflict with each other.

This program requires companies to pay multiple fees and go through a lengthy administrative process.

Even then it may not cover all products in an industry.

These efforts can be costly, and for this reason, compliance is generally not high.

However, under current practices, most exchanges offer barely adequate search capabilities for their buyers.

In many instances the buyer will fail to find the product that they seek.

These existing processes could therefore be characterized as cumbersome, time consuming, frustrating and ineffective.

Another challenging data management task is the transmission of data between dissimilar systems.

Transmitting data between such systems can be a time-consuming and expensive task.

Thus, in adapter-based systems, CPU load considerations may affect when and how often data pulls can be scheduled.

It is well known that when large amounts of data are being transmitted between systems, a system error (i.e., stoppage) and / or data loss (i.e., dropout) may occur.

With conventional adapter-based system architectures, debugging a system stoppage can be very challenging because of the large number of conversion processes involved, and because most systems do not have an integrated way to indicate the point at which processing stopped, relying instead upon error logs.

For example, if the host system is used for OLTP (on line transaction processing) during the day, pulling bulk data from the host system in order to replace data lost in a previous data transfer may be delayed until the late night hours.

Of course, the delay in processing the data can have an adverse impact of its own.

Images