A system, apparatus, and methods are provided for managing on-demand profiling of one or more instances of a software application executing on a plurality of machines within one or more data centers. During operation, the system executes the one or more instances of the software application on the plurality of machines. Next, the system publishes, to a command channel, a command message that comprises a profiling request, wherein the profiling request specifies a subset of the machines. The system then receives, via a data channel, one or more data messages from the subset of the machines, wherein the data messages comprise data gathered by the subset of the machines in response to receiving the command message. Next, the system then evaluates the performance of the software application by aggregating and processing the data messages. Responsive to detecting an anomaly in the performance, the system then executes one or more remedies.