Provision and surfacing of metrics for visualization

Abstract

Embodiments are directed to generating metrics based on visualizations. A dashboard, which may be associated with a source visualization, displays current values ​​of metrics from a source visualization model. A classifier may automatically use features from the source visualization to determine metrics for the source visualization. The source visualization model may be sampled to provide values ​​of metrics over time at a sampling rate determined by a metric profile. The sampled values ​​may be stored with time values ​​in a metric data store such that the time values ​​may correspond to when the values ​​were sampled from the visualization. A metric visualization may be generated based on the value and time values ​​such that the metric visualization displays previously sampled values ​​of the metric.

Description

Technical Field

[0001] The present invention generally relates to data analysis, and more specifically, but not limited to, generating metrics based on visualization.

Background Art

[0002] Organizations are generating and collecting increasing amounts of data. This data can be associated with different parts of an organization, such as consumer activities, manufacturing activities, customer service, server logs, etc. In some cases, an organization may develop various different data sources or data models to represent information that it may be interested in analyzing. In some cases, an organization may employ computer-based applications or tools to generate user interfaces, such as dashboards, that can provide visualizations, such as visualizations, and that can help enable improved inferences about some or all of that data. In some cases, a dashboard may provide a current or real-time view of the data. A dashboard can help a user determine the status of selected key performance indicators, but a dashboard may omit visualizations that provide historical context, trends, etc. Accordingly, the present invention has been made in view of these and other considerations.

Brief Description of the Drawings

[0003] Non-limiting and non-exhaustive embodiments of the present invention are described with reference to the following drawings. In the drawings, like reference numerals refer to like parts throughout the various figures unless otherwise specified. For a better understanding of the innovations described, refer to the following "Mode for Carrying Out the Invention" which should be read in conjunction with the accompanying drawings.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

MODE FOR CARRYING OUT THE INVENTION

[0004] Next, with reference to the accompanying drawings that form a part of this specification and illustrate specific exemplary embodiments in which the invention may be practiced, various embodiments will be described more fully below. However, the embodiments may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the embodiments to those skilled in the art. In particular, the various embodiments may be methods, systems, media, or devices. Thus, the various embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Accordingly, the following detailed description should not be construed in a limiting sense.

[0005] Throughout this specification and the claims, the following terms, unless the context clearly dictates otherwise, take the meanings explicitly associated with them herein. The phrase "in one embodiment" as used herein does not necessarily refer to the same embodiment, but may. Further, the phrase "in another embodiment" as used herein does not necessarily refer to a different embodiment, but may. Thus, as described below, various embodiments may be readily combined without departing from the scope or spirit of the invention.

[0006] In addition, as used herein, the term "or" is the inclusive "or" operator and is equivalent to the term "and / or" unless the context clearly dictates otherwise. The term "based on" is not exclusive and allows for being based on additional factors not recited unless the context clearly dictates otherwise. Also, throughout this specification, the meanings of "a", "an", and "the" include plural references. The meaning of "in" includes "in" and "on".

[0007] With respect to exemplary embodiments, the following terms are also used herein according to their corresponding meanings unless the context clearly dictates otherwise.

[0008] As used herein, the term "engine" refers to logic embodied in hardware or software instructions that can be written in a programming language such as C, C++, Objective-C, COBOL, Java®, PHP, Perl, JavaScript®, Ruby, VBScript, C#, or other Microsoft.NET® languages. An engine can be compiled into an executable program or written in an interpreted programming language. A software engine can be called from other engines or from itself. The engines described herein refer to one or more logical modules that can merge with other engines or applications or can be divided into sub-engines. An engine is stored on a non-transitory computer-readable medium or computer storage device, stored on and thereby executed on one or more general-purpose computers, and thus can create a dedicated computer configured to provide the engine.

[0009] As used herein, the term "data source" refers to the underlying source of information that is being modeled or otherwise analyzed. A data source can include information from or provided by databases (e.g., relational, graph-based, no-sql, etc.), file systems, unstructured data, streams, etc. Data sources are typically arranged to model, record, or store various operations or activities associated with an organization. In some cases, data sources are arranged to provide or facilitate various data-intensive actions such as efficient storage, querying, indexing, data exchange, searching, updating, etc. Generally, a data source may be arranged to provide features related to data manipulation or data management rather than an easily understandable presentation or visualization of the data.

[0010] As used herein, the term "data model" refers to one or more data structures that provide a representation of the underlying data source. In some cases, a data model may provide a view of the data source for a particular application. A data model can be considered a view or interface to the underlying data source. In some cases, a data model may map directly to the data source (e.g., effectively a logical pass-through). Also, in some cases, a data model may be provided by the data source. In some situations, a data model can be considered an interface to the data source. A data model enables an organization to organize or present information from the data source in a more convenient, more meaningful (e.g., easier to reason about), more secure way, etc.

[0011] As used herein, the term "data model field" refers to a named or nameable property or characteristic of a data model. A data model field is similar to a column in a database table, a node in a graph, a Java™ class attribute, and the like. For example, a data model corresponding to an employee database table may have data model fields such as name, email address, phone number, employee ID, and the like.

[0012] As used herein, the term "data object" refers to one or more entities or data structures that include a data model. In some cases, a data object may be considered part of a data model. A data object may represent an individual instance of an item or class or type of item.

[0013] As used herein, the term "data field" refers to a named or nameable property or attribute of a data object. In some cases, a data field may be considered similar to a class member of an object in object-oriented programming.

[0014] As used herein, the term "visualization model" refers to one or more data structures that a visualization engine may employ to generate visualizations for display on one or more hardware displays. A visualization model may define various features or objects that a visualization engine may render on a displayed visualization, including styling or user interface features that may be made available to non-authoring users.

[0015] As used herein, the term "metric" refers to various quantifiable or measurable values derived from visualizations. In some cases, the type of metrics available may depend on the visualization being analyzed or monitored.

[0016] As used herein, the term "source visualization" refers to a visualization that is analyzed or monitored to provide one or more metric values. Otherwise, the source visualization may be considered a normal / regular visualization.

[0017] As used herein, the term "metric visualization" refers to a visualization that displays metric information or metric values derived from a source visualization. Otherwise, the metric visualization may be considered a visualization.

[0018] As used herein, the term "metric visualization model" refers to one or more data structures that a visualization engine may employ to generate a metric visualization for display on one or more hardware displays. Otherwise, the metric visualization model may be considered a visualization model.

[0019] As used herein, the term "metric data model" refers to one or more data structures that provide the underlying data representation used for metric visualizations. Otherwise, the metric data model may be considered a data model.

[0020] As used herein, the term "source visualization model" refers to one or more data structures that a visualization engine may employ to generate source visualizations for display on one or more hardware displays. Otherwise, the source visualization model may be considered a visualization model.

[0021] As used herein, the term "source data model" refers to one or more data structures that provide a representation of the underlying data used in source visualization. Otherwise, the source data model may be considered a data model.

[0022] As used herein, the term "visualization specification" or "visualization specification information" refers to computer-readable information that a visualization engine may employ to generate a visualization model, including a source visualization model or a metric visualization model. For example, the visualization specification may be a JSON file or an XML file that defines one or more characteristics of the visualization. In some cases, the visualization specification may function as a visualization model. In other cases, the visualization specification or visualization specification information may be determined from the visualization model using introspection, reflection, decompilation, etc.

[0023] As used herein, the term "panel" refers to an area within a graphical user interface (GUI) that has a defined geometric shape (e.g., in the x, y, z order) within the GUI. A panel can be arranged to display information to a user or to host one or more interactive controls. The geometry or style associated with a panel can be defined using configuration information that includes dynamic rules. Also, in some cases, a user may be enabled to perform actions such as moving, displaying, hiding, resizing, reordering, etc. on one or more panels.

[0024] As used herein, the term "configuration information" refers to information that can include rule-based policies, pattern matching, scripts (e.g., computer-readable instructions), etc., provided from various sources such as configuration files, databases, user input, built-in defaults, etc., or combinations thereof.

[0025] The following provides a brief description of embodiments of the present invention in order to offer a basic understanding of some aspects of the present invention. This brief description is not intended to be an extensive overview. It is not intended to identify key or critical elements or to delineate or otherwise narrow the scope. Its purpose is simply to present some concepts in a simplified form as a prelude to a more detailed description that will be presented later.

[0026] Briefly stated, various embodiments are directed to generating metrics based on visualization using one or more processors that execute one or more instructions for performing as described herein.

[0027] In one or more of various embodiments, a dashboard that can be associated with one or more source visualizations that each display a current value of one or more metrics from one or more source visualization models can be provided such that each source visualization conforms to the specification.

[0028] In one or more of various embodiments, each specification is evaluated to determine one or more characteristics of each source visualization such that the one or more source visualizations can be classified based on one or more classifiers and one or more characteristics.

[0029] In one or more of various embodiments, the step of evaluating each specification can include repeating one or more classifiers to determine a class of visualizations that can correspond to one or more source visualizations, performing one or more actions to determine one or more characteristics of each source visualization based on its corresponding class, excluding each of one or more source visualizations that remain unclassified, and the like.

[0030] In one or more of various embodiments, one or more metrics for each classified source visualization can be determined based on one or more classifiers.

[0031] In one or more of various embodiments, the step of determining one or more metrics can include determining one or more of one or more single-value metrics or one or more multi-value metrics, where each of the one or more multi-value metrics is a single metric that is divided into two or more categories.

[0032] In one or more of various embodiments, one or more metric profiles corresponding to one or more metrics can be generated based on one or more classifiers.

[0033] In one or more of various embodiments, one or more source visualization models may be sampled to provide one or more values of one or more metrics such that a sampling rate may be based on one or more metric profiles. In one or more of various embodiments, the step of sampling one or more source visualization models may include the step of sampling one or more source visualization models while one or more dashboards or one or more source visualizations may be inactive so that the step of sampling may be omitted since one or more inactive dashboards or one or more inactive source visualizations are being displayed.

[0034] In one or more of various embodiments, one or more sampled values may be stored in a metric data store along with one or more time values such that one or more time values may correspond when one or more values are sampled.

[0035] In one or more of various embodiments, one or more metric visualizations may be generated based on one or more values and one or more time values such that one or more metric visualizations display one or more previously sampled values of one or more metrics. In some embodiments, the step of displaying one or more metric visualizations may include the step of displaying one or more metric visualizations on a dashboard or another user interface.

[0036] In one or more of various embodiments, one or more anomaly detectors may be provided that are arranged to identify one or more statistical anomalies present in one or more values of one or more metrics.

[0037] In one or more of various embodiments, in response to determining one or more statistical anomalies based on one or more anomaly detectors, one or more alerts including one or more of one or more notifications, one or more alerts, or one or more reports are provided, and further actions may be performed including communicating the one or more alerts to one or more responsible parties or one or more of one or more services.

[0038] The Illustrated Operating Environment FIG. 1 shows the components of one embodiment of an environment in which embodiments of the present invention may be implemented. Not all of the components are required to implement the present invention, and changes in the arrangement and type of components may be made without departing from the spirit or scope of the present invention. As shown, the system 100 of FIG. 1 includes a local area network (LAN) / wide area network (WAN)-(network) 110, a wireless network 108, client computers 102-105, a visualization server computer 116, and the like.

[0039] At least one embodiment of client computers 102-105 will be described in more detail below in connection with FIG. 2. In one embodiment, at least some of client computers 102-105 may operate via one or more wired or wireless networks such as network 108 or 110. Generally, client computers 102-105 can include substantially any computer capable of sending and receiving information and communicating via a network to perform various online activities, offline actions, and the like. In one embodiment, one or more of client computers 102-105 can be configured to operate within a business or other entity to perform various services for the business or other entity. For example, client computers 102-105 can be configured to operate as web servers, firewalls, client applications, media players, mobile phones, game consoles, desktop computers, and the like. However, client computers 102-105 are not limited to these services and can be employed, for example, with respect to end-user computing in other embodiments. More or fewer client computers (such as those shown in FIG. 1) can be included within a system as described herein, and thus it should be appreciated that embodiments are not limited by the number or type of client computers employed.

[0040] Computers that can operate as client computer 102 can include computers typically connected using wired or wireless communication media, such as personal computers, multiprocessor systems, microprocessor-based or programmable electronic devices, network PCs, and the like. In some embodiments, client computers 102 - 105 can include substantially any portable computer that can connect to another computer to receive information, such as laptop computer 103, mobile computer 104, tablet computer 105, and the like. However, portable computers are not so limited and can also include other portable computers such as cellular phones, display pagers, radio frequency (RF) devices, infrared (IR) devices, personal digital assistants (PDAs), handheld computers, wearable computers, integrated devices combining one or more of the aforementioned computers, and the like. Thus, client computers 102 - 105 typically vary widely with respect to functionality and features. Further, client computers 102 - 105 can access a variety of computing applications, including browsers or other web-based applications.

[0041] The web-enabled client computer may include a browser application configured to send requests and receive responses over the web. The browser application may be configured to receive and display graphics, text, multimedia, etc., employing substantially any web-based language. In one embodiment, the browser application may employ JavaScript®, Hypertext Markup Language (HTML), Extensible Markup Language (XML), JavaScript Object Notation (JSON), Cascading Style Sheets (CSS), etc., or combinations thereof, to display and send messages. In one embodiment, a user of the client computer may employ the browser application to perform various activities over the network (online). However, another application may also be used to perform various online activities.

[0042] Client computers 102-105 may include at least one other client application configured to receive or send content between other computers. The client application may include the ability to send or receive content, etc. The client application may further provide information identifying itself, including type, capabilities, name, etc. In one embodiment, client computers 102-105 may uniquely identify themselves via any of a variety of mechanisms, including Internet Protocol (IP) addresses, telephone numbers, Mobile Identification Numbers (MINs), Electronic Serial Numbers (ESNs), client certificates, or other device identifiers. Such information may be provided in one or more network packets transmitted between other client computers, visualization server computers 116, or other computers.

[0043] Client computers 102-105 may be further configured to include a client application that enables an end user to log in to an end user account that may be managed by another computer, such as visualization server computer 116. Such an end user account, in one non-limiting example, may be configured to enable the end user to manage one or more online activities, including, in one non-limiting example, project management, software development, system administration, configuration management, search activities, social networking activities, browse various websites, communicate with other users, etc. Further, the client computer may be arranged to enable the user to display reports, an interactive user interface, or results provided by visualization server computer 116, etc.

[0044] Wireless network 108 is configured to couple client computers 103-105 and their components to network 110. Wireless network 108 may include any of a variety of wireless sub-networks that may further overlay a stand-alone ad hoc network, etc., to provide an infrastructure-oriented connection for client computers 103-105. Such sub-networks may include a mesh network, a wireless LAN (WLAN) network, a cellular network, etc. In one embodiment, the system may include two or more wireless networks.

[0045] Wireless network 108 may further include autonomous systems such as terminals, gateways, routers, etc., connected by a wireless radio link, etc. These connectors may be configured to move freely and randomly and arbitrarily organize themselves so that the topology of wireless network 108 can change rapidly.

[0046] Wireless network 108 may further employ multiple access technologies including second-generation (2G), third-generation (3G), fourth-generation (4G), fifth-generation (5G) wireless access for cellular systems, WLANs, wireless router (WR) meshes, and the like. Access technologies such as 2G, 3G, 4G, 5G, and future access networks may enable wide-area coverage for mobile computers such as client computers 103-105 having various degrees of mobility. In a non-limiting example, wireless network 108 may enable wireless connections via wireless network access such as GSM (Global System for Mobile communication), General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wideband Code Division Multiple Access (WCDMA), High-Speed Downlink Packet Access (HSDPA), Long Term Evolution (LTE), and the like. In essence, wireless network 108 may include substantially any wireless communication mechanism through which information may move between client computers 103-105 and another computer, network, cloud-based network, cloud instance, or the like.

[0047] Network 110 is configured to couple network computers with other computers including visualization server computer 116, client computers 102, and client computers 103 - 105 via wireless network 108. Network 110 can employ any form of computer-readable medium for communicating information from one electronic device to another. Also, network 110 can include the Internet, in addition to a local area network (LAN), a wide area network (WAN), for example, a direct connection via a Universal Serial Bus (USB) port, an Ethernet® port, other forms of computer-readable media, or any combination thereof. On an interconnected set of LANs including those based on different architectures and protocols, a router functions as a link between the LANs, enabling messages to be sent from one to the other. Additionally, communication links within a LAN typically include twisted pair wires or coaxial cables, while communication links between networks can utilize other carrier mechanisms including analog telephone lines, T1, T2, T3, and T4 full or partial dedicated digital lines, or wireless links including, for example, E-carrier, Integrated Services Digital Network (ISDN), Digital Subscriber Line (DSL), satellite links, or other communication links known to those skilled in the art. Further, communication links can further employ any of a variety of digital signaling techniques including, but not limited to, for example, DS-0, DS-1, DS-2, DS-3, DS-4, OC-3, OC-12, OC-48, etc. Additionally, remote computers and other related electronic devices can be remotely connected to either a LAN or a WAN via a modem and a temporary telephone line. In one embodiment, network 110 can be configured to transport Internet Protocol (IP) information.

[0048] Additionally, a communication medium typically embodies computer-readable instructions, data structures, program modules, or other transport mechanisms and includes any non-transitory or transitory information delivery medium. By way of example, a communication medium includes wired media such as twisted pair, coaxial cable, fiber optic, waveguides, and other wired media, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0049] Also, one embodiment of the visualization server computer 116 will be described in more detail below in connection with FIG. 3. Although FIG. 1 shows the visualization server computer 116 and the like as a single computer, the invention or embodiments are not so limited. For example, one or more functions such as the visualization server computer 116 can be distributed across one or more separate network computers. Further, in one or more embodiments, the visualization server computer 116 can be implemented using a plurality of network computers. Still further, in one or more of the various embodiments, the visualization server computer 116 and the like can be implemented using one or more cloud instances within one or more cloud networks. Accordingly, these innovations and embodiments should not be construed as limited to a single environment, and other configurations and other architectures are also contemplated.

[0050] Exemplary Client Computer FIG. 2 shows an embodiment of a client computer 200 that may include more or fewer components than those illustrated. The client computer 200 may represent, for example, one or more embodiments of the mobile computer or client computer shown in FIG. 1.

[0051] The client computer 200 may include a processor 202 that communicates with a memory 204 via a bus 228. The client computer 200 may also include a power supply 230, a network interface 232, an audio interface 256, a display 250, a keypad 252, an illuminator 254, a video interface 242, an input / output interface 238, a tactile interface 264, a global positioning system (GPS) receiver 258, an open air gesture interface 260, a temperature interface 262, one or more cameras 240, a projector 246, a pointing device interface 266, a processor-readable fixed storage device 234, and a processor-readable removable storage device 236. The client computer 200 may optionally communicate with a base station (not shown) or directly with another computer. And in one embodiment, although not shown, a gyroscope may be employed within the client computer 200 to measure or maintain the orientation of the client computer 200.

[0052] The power supply 230 may supply power to the client computer 200. A rechargeable or non-rechargeable battery may be used to provide power. The power may also be provided by an external power source such as an AC adapter or a powered docking cradle that complements or recharges the battery.

[0053] The network interface 232 includes circuitry for coupling the client computer 200 to one or more networks and is constructed for use with one or more communication protocols and technologies including, but not limited to, any portion of the OSI model for mobile communication (GSM (registered trademark)), CDMA, Time Division Multiple Access (TDMA), UDP, TCP / IP, SMS, MMS, GPRS, WAP, UWB, WiMax, SIP / RTP, GPRS, EDGE, WCDMA (registered trademark), LTE, UMTS, OFDM, CDMA2000, EV-DO, HSDPA, or any of a variety of other wireless communication protocols. The network interface 232 may also be known as a transceiver, a transceiver device, or a network interface card (NIC).

[0054] The audio interface 256 may be arranged to generate and receive audio signals such as the sound of a human voice. For example, the audio interface 256 may be coupled to a speaker and a microphone (not shown) to enable electrical communication with others or to generate an audio confirmation response for some action. The microphone within the audio interface 256 can also be used for input to or control of the client computer 200, such as detecting a touch based on sound, for example, using voice recognition.

[0055] The display 250 can be a liquid crystal display (LCD), gas plasma, electronic ink, light emitting diode (LED), organic LED (OLED), or any other type of light-reflective or light-transmissive display that can be used with a computer. The display 250 may also include a touch interface 244 arranged to receive input from objects such as a stylus or a human finger and may use resistive, capacitive, surface acoustic wave (SAW), infrared, radar, or other technologies to sense touches or gestures.

[0056] The projector 246 can be an integrated projector that can project an image onto a remote handheld projector or any other reflective object such as a remote wall or remote screen.

[0057] The video interface 242 can be arranged to capture video images such as still photos, video segments, infrared video, etc. For example, the video interface 242 can be coupled to a digital video camera, a web camera, etc. The video interface 242 can include a lens, an image sensor, and other electronic devices. The image sensor can include a complementary metal-oxide-semiconductor (CMOS) integrated circuit, a charge-coupled device (CCD), or any other integrated circuit for sensing light.

[0058] The keypad 252 can include any input device arranged to receive input from a user. For example, the keypad 252 can include a push-button type numeric dial or a keyboard. The keypad 252 can also include command buttons associated with the selection and transmission of images.

[0059] The illuminator 254 can provide a status indication or provide light. The illuminator 254 can remain active for a specific period or in response to an event message. For example, when active, the illuminator 254 can backlight the buttons on the keypad 252 and can remain on while the client computer is powered. Also, the illuminator 254 can backlight these buttons in various patterns when a specific action such as dialing another client computer is performed. The illuminator 254 can also illuminate a light source placed within the transparent or translucent case of the client computer in response to an action.

[0060] Furthermore, the client computer 200 may also include a hardware security module (HSM) 268 for providing additional anti-tampering prevention means for generating, storing, or using security / cryptographic information such as keys, digital certificates, passwords, passphrases, two-factor authentication information, etc. In some embodiments, the hardware security module may be employed to support one or more standard public key infrastructures (PKIs), and may be employed to generate, manage, or store key pairs, etc. In some embodiments, the HSM 268 may be a stand-alone computer, and in other cases, the HSM 268 may be arranged as a hardware card that can be added to the client computer.

[0061] The client computer 200 may also include an input / output interface 238 for communicating with other computers such as external peripheral devices or other client computers and network computers. Peripheral devices may include, for example, audio headsets, virtual reality headsets, display screen glasses, remote speaker systems, remote speakers and microphone systems, etc. The input / output interface 238 can utilize one or more technologies such as Universal Serial Bus (USB (R)), infrared, WiFi, WiMax, Bluetooth (R), etc.

[0062] The input / output interface 238 may also include one or more sensors for determining geolocation information (e.g., GPS), one or more sensors for monitoring power status (e.g., voltage sensors, current sensors, frequency sensors, etc.), one or more sensors for monitoring weather (e.g., thermostats, barometers, anemometers, humidity detectors, precipitation gauges, etc.), etc. The sensors may be one or more hardware sensors that collect or measure data external to the client computer 200.

[0063] The tactile interface 264 can be arranged to provide tactile feedback to a user of the client computer. For example, the tactile interface 264 can be employed to vibrate the client computer 200 in a specific manner when another user of the computer is on the phone. The temperature interface 262 can be used to provide a temperature measurement input or a temperature change output to the user of the client computer 200. The open air gesture interface 260 can sense the physical gestures of the user of the client computer 200, for example, by using a single or stereo video camera, radar, a gyro sensor within a computer held or worn by the user, and the like. The camera 240 can be used to track the physical eye movements of the user of the client computer 200.

[0064] The GPS transceiver 258 can determine the physical coordinates of the client computer 200 on the surface of the earth and typically outputs the position as values of latitude and longitude. The GPS transceiver 258 can also employ other geolocation mechanisms including, but not limited to, triangulation, assisted GPS (AGPS), enhanced observed time difference (E-OTD), cell identifier (CI), service area identifier (SAI), enhanced timing advance (ETA), base station subsystem (BSS), etc. to further determine the physical location of the client computer 200 on the surface of the earth. It should be understood that under different conditions, the GPS transceiver 258 can determine the physical location of the client computer 200. However, in one or more embodiments, the client computer 200 can provide other information that can be employed to determine the physical location of the client computer, for example, via other components, including, but not limited to, a media access control (MAC) address, an IP address, and the like.

[0065] In at least one of the various embodiments, applications such as operating system 206, visualization client 222, other client applications 224, web browser 226, etc. are arranged to employ geolocation information to select one or more localization features such as time zone, language, currency, calendar format, etc. The localization features can be used in display objects, data models, data objects, user interfaces, reports, and internal processes or databases. In at least one of the various embodiments, the geolocation information used to select the localization information can be provided by GPS 258. Also, in some embodiments, the geolocation information can include information provided using one or more geolocation protocols via a network such as wireless network 108 or network 111.

[0066] The human interface components can be peripheral devices physically separated from the client computer 200 and enable remote input or output to the client computer 200. For example, the information routed as described herein through human interface components such as the display 250 or the keyboard 252 can instead be routed through the network interface 232 to an appropriate human interface component located remotely. Examples of human interface peripheral components that can be remote include, but are not limited to, audio devices, pointing devices, keypads, displays, cameras, projectors, and the like. These peripheral components can communicate via piconetworks such as Bluetooth (registered trademark), Zigbee (registered trademark). One non-limiting example of a client computer with such peripheral human interface components is a wearable computer that includes one or more cameras that communicate remotely with a client computer located separately to sense a user's gesture with respect to a portion of an image projected by a picoprojector onto a reflective surface such as a wall or the user's hand, along with a remote picoprojector.

[0067] The client computer can include a web browser application 226 configured to send and receive web pages, web-based messages, graphics, text, multimedia, and the like. The browser application of the client computer can employ substantially any programming language, including, for example, Wireless Application Protocol messages (WAP). In one or more embodiments, the browser application can employ Handheld Device Markup Language (HDML), Wireless Markup Language (WML), WMLScript, JavaScript (registered trademark), Standard Generalized Markup Language (SGML), Hypertext Markup Language (HTML), Extensible Markup Language (XML), HTML5, and the like.

[0068] Memory 204 may include RAM, ROM, or other types of memory. Memory 204 represents an example of a computer-readable storage medium (device) for storing information such as computer-readable instructions, data structures, program modules, or other data. Memory 204 may store BIOS 208 for controlling the low-level operation of client computer 200. The memory may also store operating system 206 for controlling the operation of client computer 200. It will be appreciated that this component may include a general-purpose operating system such as a version of UNIX® or LINUX®, or a dedicated client computer communication operating system such as Windows Phone® or Symbian operating system. The operating system may include or interface with a Java® virtual machine module that enables control of the hardware components or the operation of the operating system via Java® application programs.

[0069] Memory 204 may further include one or more data storages 210 available for use by client computer 200 to store, among other things, application 220 or other data. For example, data storage 210 may also be employed to store information describing various functions of client computer 200. The information can then be provided to another device or computer based on any of a variety of methods, including being sent as part of a header during communication, being sent in response to a request, etc. Data storage 210 may also be employed to store social networking information, including an address book, buddy list, aliases, user profile information, and the like. Data storage 210 may further include program code, data, algorithms, etc. for use by a processor, such as processor 202, to execute and implement actions. In one embodiment, at least a portion of data storage 210 may be stored on another component of client computer 200, including, but not limited to, non-transitory processor-readable removable storage device 236, processor-readable fixed storage device 234, and even external to the client computer.

[0070] When executed by client computer 200, application 220 may include computer-executable instructions that send, receive, or otherwise process instructions and data. Application 220 may include, for example, visualization client 222, other client applications 224, web browser 226, and the like. The client computer may be arranged to communicate with one or more servers.

[0071] Other examples of application programs include calendars, search programs, email client applications, IM applications, SMS applications, Voice over Internet Protocol (VOIP) applications, contact managers, task managers, transcoders, database programs, word processing programs, security applications, spreadsheet programs, games, search programs, visualization applications, and the like.

[0072] Additionally, in one or more embodiments (not shown), client computer 200 may include an embedded logic hardware device such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable array logic (PAL), or the like, or a combination thereof, instead of a CPU. The embedded logic hardware device may directly execute its embedded logic to perform actions. Also, in one or more embodiments (not shown), client computer 200 may include one or more hardware microcontrollers instead of a CPU. In one or more embodiments, the one or more microcontrollers may directly execute their own embedded logic to perform actions and may access their own internal memory and their own external input and output interfaces (e.g., hardware pins or wireless transceivers) to perform actions such as a system on a chip (SOC).

[0073] Exemplary Network Computer Figure 3 shows an embodiment of a network computer 300 that may be included in a system implementing one or more of the various embodiments. The network computer 300 may include more or fewer components than those shown in Figure 3. However, the components shown are sufficient to disclose exemplary embodiments for implementing these innovations. The network computer 300 may represent, for example, an embodiment of at least one of the event analysis server computers 116 of Figure 1.

[0074] A network computer, such as network computer 300, may include a processor 302 that may communicate with a memory 304 via a bus 328. In some embodiments, the processor 302 may be composed of one or more hardware processors or one or more processor cores. In some cases, one or more of the one or more processors may be dedicated processors designed to perform one or more dedicated actions, such as the actions described herein. The network computer 300 also includes a power supply 330, a network interface 332, an audio interface 356, a display 350, a keyboard 352, an input / output interface 338, a processor-readable fixed storage device 334, and a processor-readable removable storage device 336. The power supply 330 supplies power to the network computer 300.

[0075] Network interface 332 includes circuitry for coupling network computer 300 to one or more networks and includes one or more communication protocols and technologies for use with, without limitation, any portion of the Open Systems Interconnection Model (OSI model), GSM® (Global System for Mobile communication), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), User Datagram Protocol (UDP), Transmission Control Protocol / Internet Protocol (TCP / IP), Short Message Service (SMS), Multimedia Messaging Service (MMS), General Packet Radio Service (GPRS), WAP, Ultra Wide Band (UWB), IEEE 802.16 WiMax (Worldwide Interoperability for Microwave Access), Session Initiation Protocol / Real Time Transport Protocol (SIP / RTP), or various other wired and wireless communication protocols. Network interface 332 may also be known as a transceiver, a transmit / receive device, or a network interface card (NIC). Network computer 300 may optionally communicate with a base station (not shown) or directly with another computer.

[0076] Audio interface 356 is arranged to generate and receive audio signals such as the sound of a human voice. For example, audio interface 356 may be coupled to a speaker and a microphone (not shown) to enable telecommunication with others or to generate an audio confirmation response to some action. The microphone within audio interface 356 may also be used, for example, to provide input to or control of network computer 300 using speech recognition.

[0077] The display 350 can be a liquid crystal display (LCD), gas plasma, electronic ink, light emitting diode (LED), organic LED (OLED), or any other type of light-reflective or light-transmissive display that can be used with a computer. In some embodiments, the display 350 can be a handheld projector or pico projector capable of projecting an image onto a wall or other object.

[0078] The network computer 300 can also include an input / output interface 338 for communicating with external devices or computers not shown in FIG. 3. The input / output interface 338 can utilize one or more wired or wireless communication technologies such as USB (registered trademark), Firewire (registered trademark), WiFi, WiMax, Thunderbolt (registered trademark), infrared, Bluetooth (registered trademark), Zigbee (registered trademark), serial port, parallel port, etc.

[0079] In addition, the input / output interface 338 may also include one or more sensors for determining geolocation information (e.g., GPS), one or more sensors for monitoring power status (e.g., voltage sensor, current sensor, frequency sensor, etc.), one or more sensors for monitoring weather (e.g., thermostat, barometer, anemometer, humidity detector, precipitation gauge, etc.), and the like. The sensors can be one or more hardware sensors that collect or measure data external to the network computer 300. The human interface components can be physically separated from the network computer 300 and enable remote input or output to the network computer 300. For example, information routed as described herein through a human interface component such as a display 350 or a keyboard 352 can instead be routed through the network interface 332 to an appropriate human interface component located elsewhere on the network. The human interface components include any components that enable a computer to receive input from or send output to a human user of the computer. Thus, pointing devices such as a mouse, stylus, trackball, etc. can communicate via a pointing device interface 358 to receive user input.

[0080] The GPS transceiver 340 can determine the physical coordinates of the network computer 300 on the surface of the earth and typically outputs the location as values of latitude and longitude. The GPS transceiver 340 can also employ other geolocation mechanisms including, but not limited to, triangulation, assisted GPS (AGPS), enhanced observed time difference (E-OTD), cell identifier (CI), service area identifier (SAI), enhanced timing advance (ETA), base station subsystem (BSS), etc. to further determine the physical location of the network computer 300 on the surface of the earth. It should be understood that under different conditions, the GPS transceiver 340 can determine the physical location of the network computer 300. However, in one or more embodiments, the network computer 300 can provide other information that can be employed to determine the physical location of a client computer via other components, such as, for example, media access control (MAC) address, IP address, etc.

[0081] In at least one of the various embodiments, applications such as the operating system 306, metric engine 322, visualization engine 324, other applications 329, etc. are arranged to employ geolocation information to select one or more localization features such as time zone, language, calendar format, etc. The localization features can be used in the user interface, dashboard, visualization, report, and internal processes or databases. In at least one of the various embodiments, the geolocation information used to select the localization information can be provided by the GPS 340. Also, in some embodiments, the geolocation information can include information provided using one or more geolocation protocols via a network such as the wireless network 108 or network 111.

[0082] Memory 304 may include random access memory (RAM), read-only memory (ROM), or other types of memory. Memory 304 represents an example of a computer-readable storage medium (device) for storing information such as computer-readable instructions, data structures, program modules, or other data. Memory 304 stores a basic input / output system (BIOS) 308 for controlling the low-level operation of network computer 300. The memory also stores an operating system 306 for controlling the operation of network computer 300. It will be appreciated that this component may include a general-purpose operating system such as a version of UNIX® or Linux®, or a proprietary operating system such as the Windows® operating system of Microsoft Corporation or the macOS® operating system of Apple Corporation. The operating system may include or interface with one or more virtual machine modules such as a Java® virtual machine module that enables control of hardware components or operating system operations via Java® application programs. Similarly, other runtime environments may be included.

[0083] Memory 304 may further include one or more data storages 310 available for use by network computer 300 to store, among other things, application 320 or other data. For example, data storage 310 may also be employed to store information describing various functions of network computer 300. The information may then be provided to another device or computer based on any of a variety of methods, including being transmitted as part of a header during communication, being transmitted in response to a request, etc. Data storage 310 may also be employed to store social networking information, including address books, buddy lists, aliases, user profile information, and the like. Data storage 310 may further include program code, data, algorithms, etc. for use by a processor, such as processor 302, to perform and implement actions such as those described below. In one embodiment, at least a portion of data storage 310 may be stored on another component of network computer 300, including, but not limited to, processor-readable removable storage device 336, processor-readable fixed storage device 334, or any other computer-readable storage device within or external to network computer 300, including non-transitory media. Data storage 310 may include, for example, data source 314, data model 316, metric model 318, and the like.

[0084] When executed by network computer 300, application 320 can include computer-executable instructions that send, receive, or otherwise process messages (such as SMS, Multimedia Messaging Service (MMS), Instant Message (IM), email, or other messages), audio, and video, enabling telecommunications with another user of another mobile computer. Other examples of application programs include calendars, search programs, email client applications, IM applications, SMS applications, Voice over Internet Protocol (VOIP) applications, contact managers, task managers, transcoders, database programs, word processing programs, security applications, spreadsheet programs, games, search programs, and the like. Application 320 can include a metric engine 322, a visualization engine 324, other applications 329, etc., which can be arranged to perform actions for the embodiments described below. In one or more of various embodiments, one or more of the applications can be implemented as a module or component of another application. Further, in one or more of various embodiments, the application can be implemented as an operating system extension, module, plugin, or the like.

[0085] Furthermore, in one or more of the various embodiments, the metric engine 322, the visualization engine 324, other applications 329, etc. may operate in a cloud-based computing environment. In one or more of the various embodiments, these applications including the management platform and other applications may be running within a virtual machine or virtual server that can be managed in a cloud-based computing environment. In one or more of the various embodiments, in this context, an application may flow from one physical network computer to another physical network computer within a cloud-based environment in accordance with performance and scaling considerations that are automatically managed by the cloud computing environment. Similarly, in one or more of the various embodiments, a virtual machine or virtual server dedicated to the metric engine 322, the visualization engine 324, other applications 329, etc. may be automatically provisioned and de-commissioned.

[0086] Also, in one or more of the various embodiments, the metric engine 322, the visualization engine 324, other applications 329, etc. may be located within a virtual server that is launched within a cloud-based computing environment rather than being tied to one or more specific physical network computers.

[0087] Furthermore, network computer 300 may also include a hardware security module (HSM) 360 for providing additional anti-tampering prevention means for generating, storing, or using security / cryptographic information such as keys, digital certificates, passwords, passphrases, two-factor authentication information, etc. In some embodiments, the hardware security module may be employed to support one or more standard public key infrastructures (PKIs), and may be employed to generate, manage, or store key pairs, etc. In some embodiments, HSM 360 may be a stand-alone network computer, and in other cases, HSM 360 may be arranged as a hardware card installable within the network computer.

[0088] Additionally, in one or more embodiments (not shown), network computer 300 may include embedded logic hardware devices such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable array logic (PALs), etc., or combinations thereof, instead of a CPU. The embedded logic hardware device may directly execute its embedded logic to perform actions. Also, in one or more embodiments (not shown), the network computer may include one or more hardware microcontrollers instead of a CPU. In one or more embodiments, the one or more microcontrollers may directly execute their own embedded logic to perform actions, and may access their own internal memory and their own external input and output interfaces (e.g., hardware pins or wireless transceivers) to perform actions such as system-on-chip (SOC).

[0089] Exemplary Logical System Architecture Figure 4 shows the logical architecture of a system 400 for providing and surfacing metrics for visualization according to one or more of various embodiments. In one or more of various embodiments, the system 400 may be a visualization platform arranged to include various components such as a metric engine 402, a visualization engine 404, a source visualization 406A, a metric visualization 406B, a visualization model / specification 408A, a metric visualization model / specification 408B, a data model 410A, a metric data model 410B, a data source 412, a visualization classifier 414, a metric value 416, a dashboard 418, an anomaly detector 420, and the like.

[0090] In one or more of various embodiments, the data source 412 represents a source of raw data, records, data items, or the like that the metric engine 402 may employ to enable a user to generate or modify a data model such as the data model 410.

[0091] In one or more of the various embodiments, a data model, such as data model 410A, can be a data structure that provides one or more logical representations of information stored in one or more data sources, such as data source 412. In some embodiments, the data model can include data objects corresponding to one or more portions of tables, views, or files within the data source. For example, in some embodiments, where data source 412 is a CSV file or a database, a data model, such as data model 412, can be composed of one or more data objects that can correspond to record fields within data source 412. Similarly, in some embodiments, the data model can include fields corresponding to fields or attributes within the data source. For example, in some embodiments, where data source 412 is a relational database management system (RDBMS), the data model included in data model 410A can be composed of one or more data model fields corresponding to one or more columns or one or more tables included in data source 412.

[0092] In some embodiments, a visualization engine, such as visualization engine 404, can be employed to convert or map a part or all of data source 412 to data model 410A. In some embodiments, the visualization engine can be arranged to employ or execute computer-readable instructions provided by configuration information to determine a part or all of the steps for converting values within the data source to the data model.

[0093] In some embodiments, the visualization engine 404 may be employed to convert or map some or all of the data source 412, or some or all of the metric values 416, to the metric data model 410B. In some embodiments, the visualization engine may be arranged to employ or execute computer-readable instructions provided by the configuration information to determine some or all of the steps for converting objects or values from the data source or metric values to the data model. Note that in some embodiments, since the metric data model may be stored in the same data store as other data models, the metric data model 410B is shown using a dashed line.

[0094] In one or more of the various embodiments, a visualization engine, such as the visualization engine 404, may be arranged to employ a visualization model, such as the source visualization model 408A, to determine the layout, styling, interactivity, etc. for a source visualization, such as the source visualization 406A, that may be presented to the user. Also, in some embodiments, the visualization engine may be arranged to employ the data item values provided via the data source 412 to populate the source visualization with values based on the source data model.

[0095] Similarly, in some embodiments, a visualization engine, such as visualization engine 404, may be arranged to employ a metric visualization model 408B to determine layout, styling, interactivity, etc. for a metric visualization, such as metric visualization 406B, that can be presented to a user. Also, in some embodiments, the visualization engine may be arranged to employ data item values or metric values 416 provided via data source 412 to populate the metric visualization with values based on the metric data model. In some embodiments, note that the metric visualization model 410B is shown using a dashed line because the metric visualization model or metric visualization specification may be stored in the same data store as other visualization models / specifications.

[0096] In one or more of various embodiments, a visualization model may be defined using one or more visualization specifications. In some embodiments, a visualization specification may include computer-readable instructions, such as formal or semi-formal rules, that may correspond to the visualization model. In some embodiments, a visualization specification may be used to represent or define one or more visualization models. In some embodiments, a source visualization specification may be employed to generate a source visualization model or source visualization. Similarly, in some embodiments, a metric visualization specification may be employed to generate a metric visualization model or metric visualization.

[0097] In one or more of various embodiments, the visualization engine may be arranged to generate a visualization or visualization model based on a visualization specification. In one or more of various embodiments, the visualization engine or metric engine may be arranged to support one or more different types of visualization specifications. Thus, in some embodiments, the metric engine or visualization engine may be arranged to employ rules, grammars, etc. provided via configuration information to interpret a given visualization specification.

[0098] In some embodiments, a dashboard user interface, such as dashboard 418, can be a common form of visual analytics often employed in business intelligence applications, informatics, or industrial monitoring, and many other domains or analysis tasks. In some embodiments, these visual representations can take many forms and styles based on the data acquired and the information needs of the viewer or analyst. In some cases, due to this diversity, there may be a lack of a consistent agreed-upon definition of what constitutes a high-quality dashboard visualization. Thus, conventionally, a broad view can be taken of what is considered a dashboard, including infographics, narrative elements, etc. As used herein, for some embodiments, a dashboard user interface can be a user interface that is arranged to include at least a visual representation of important information that may be required to achieve one or more objectives and that is integrated and arranged on a single screen such that the information can be monitored at a glance.

[0099] In some cases, dashboards can be designed to display current statuses or KPIs of interest to the user or organization. In some cases, data sources or dashboards can be designed to show historical data, and the history of changes can be apparent in how the data is stored or visualized. However, it is often the case that the historical information is not represented anywhere. In this case, since the history of the data needs to be separately captured and stored, additional IT resources may often be required, which can disadvantage the reporting or visualization of such information.

[0100] Accordingly, in some embodiments, a metric engine, such as metric engine 402, can be arranged to automatically determine one or more metrics in a visualization, including the visualizations included in a dashboard. In some embodiments, the metric engine can be arranged to interpret a source visualization model or source visualization specification associated with a source visualization included in the dashboard to identify one or more metrics to be captured or analyzed. In some embodiments, the captured or determined metrics can be stored in a data store, such as metric value 416.

[0101] In one or more of various embodiments, the metric engine can be arranged to provide one or more metric visualization specifications or metric visualization models that enable a visualization engine to display metric values in a visualization.

[0102] In some embodiments, the metric engine may be arranged to automatically determine one or more metrics from the monitored source visualization. In some embodiments, the metric engine may be arranged to periodically sample the values of one or more metrics and store them in the metric value 416. In some embodiments, the metric may be associated with a metric profile that includes additional information such as a sample rate, data type information, a metric visualization specification (or reference thereto) for displaying the metric visualization (or reference thereto).

[0103] In one or more of various embodiments, the metric engine may be arranged to employ one or more visualization classifiers to classify source visualizations included in a dashboard to determine how metrics can be identified or extracted. In some embodiments, the visualization classifier may be configured to recognize different classes of source visualizations and determine whether key metrics can be identified. Thus, in some embodiments, if the metric engine can classify the source visualization, the metric engine may take an action based on the classification to determine a metric or extract a metric value associated with the source visualization. Also, in one or more of various embodiments, the classification enables the metric engine to identify one or more source visualizations that are not suitable for providing and surfacing metrics. For example, in some cases, the source visualization may be designed to report or visualize historical data. Thus, in this example, the metric engine may not need to attempt to replicate the effort. Also, for example, some source visualizations may have a source visualization specification that has features that impede automatic metric determination. For example, a source visualization specification that defines a non-conventional source visualization may be determined to not be suitable for automatic metric determination.

[0104] In some embodiments, the visualization classifier may be associated with a metric visualization specification or a metric visualization specification template that can be automatically employed to generate a metric visualization. Alternatively, in some embodiments, visualization specification information for specific metrics may be stored in their corresponding metric profiles.

[0105] In one or more of various embodiments, the metric engine may be arranged to collect a history record of changes associated with metric values. Thus, in some embodiments, metric visualization may be employed to display these history records, and a user can observe how the KPIs within the dashboard vary over time.

[0106] Furthermore, in some embodiments, the metric engine may be arranged to employ one or more anomaly detectors to identify abnormal metric values. In one or more of various embodiments, the anomaly detector may be composed of a data structure or computer-readable instructions that may be arranged to identify one or more statistical anomalies in the metric values associated with the source visualization. In one or more of various embodiments, the metric engine may be arranged to employ different anomaly detectors that may be targeted at identifying different types of anomalies. Also, in some embodiments, a user or organization may be enabled to provide preference information or the like that the metric engine may employ to determine which anomaly detector should be used. Also, in some embodiments, a user or organization may be enabled to provide one or more parameters, thresholds, time ranges / windows, etc., that may be employed to define one or more abnormal states / events. Furthermore, in some embodiments, the metric engine may be arranged to enable a user or organization to provide a custom or configured anomaly detector for their application needs. Thus, in some embodiments, the metric engine may be arranged to determine one or more anomaly detectors from the configuration information to account for local requirements or local situations.

[0107] In one or more of various embodiments, when an anomaly can be detected, the metric engine may be arranged to generate one or more notifications, alerts, events, reports, etc. In some embodiments, notifications or the like may be provided to an external service or system that may manage the investigation of the reported anomaly.

[0108] FIG. 5 shows a portion of a user interface 500 that can be considered a dashboard according to one or more of various embodiments. In this example, the user interface 500 includes several different visualizations including a dashboard that includes source visualizations 502, source visualization 504, source visualization 506, source visualization 508, source visualization 510, source visualization 512, and the like. These source visualizations can be considered to represent visualizations of various KPIs, statuses, and the like. In some cases, the visualizations included in a dashboard such as dashboard 500 can include one or more source visualizations or one or more metric visualizations.

[0109] In one or more of various embodiments, the user interface 500 can be displayed on one or more hardware displays such as a client computer display, a mobile device display, and the like. In some embodiments, the user interface 500 can be provided via a native application or as a web application hosted within a web browser or other similar application. Those skilled in the art will understand that many details common to commercial / production user interfaces have been omitted from the user interface 500, at least for clarity or brevity. Similarly, in some embodiments, the user interface can be arranged differently than shown depending on local circumstances or local requirements such as display type, display resolution, user preferences, and the like. However, those skilled in the art will understand that the disclosure / description of the user interface 500 is at least sufficient to disclose the innovation included herein.

[0110] FIG. 6 shows a partial logical representation of a metric visualization user interface 600 for providing and surfacing metrics for visualization according to one or more of the various embodiments. As described above, in some embodiments, the metric engine may be arranged to monitor or record metrics associated with the source visualization. Thus, in some embodiments, the metric engine may be arranged to generate a metric visualization, such as metric visualization 602, to display historical information regarding one or more metrics for one or more of the source visualizations.

[0111] In this example, metric visualization 602 may be considered to be based on source visualization 502 shown in FIG. 5. In this example, for some embodiments, metric visualization 602 may be a line plot representing the number of cases over time (from source visualization 502 of FIG. 5). In this example, the current view of source visualization 502 showing the values of 43 cases is represented by point 604 ("B"). Thus, the source visualization 502 within user interface 500 shows the current number of cases, but does not provide context or trend information that may be of interest to the user. In this example, point 606 ("A") represents a previous time when the number of cases was greater than the current value. In this example, if the user views the dashboard (user interface 500) at point (time) 608 ("C") and then at point (time) 604 ("B"), the user may have the impression that the value of source visualization 502 was flat and smooth, even if the actual value represented by source visualization 502 was fluctuating wildly.

[0112] Accordingly, in some embodiments, the metric engine may be arranged to automatically collect historical metric values that can be presented to a user using metric visualization that provides the user with an improved understanding of how the user's system or organization may be functioning.

[0113] Also, in one or more of various embodiments, the metric engine may be arranged to monitor metrics to identify one or more anomalies based on the metric values. In this example, point 610 (“D”) represents a low value that may be considered an anomaly. Accordingly, in this example, the metric engine may employ one or more anomaly detectors that can identify the anomaly value even when the user is not looking at the dashboard. As described above, the metric engine may be arranged to generate one or more notifications, events, alerts, etc. when an anomaly can be detected.

[0114] FIG. 7 shows an exemplary visualization specification for providing and surfacing metrics for visualization according to one or more of the various embodiments. As described above, a visualization (source visualization or metric visualization) may be defined using a visualization specification or a visualization model (based on the visualization specification). The visualization specification may take many forms. In this example, visualization specification 702 and visualization specification 704 may be considered non-limiting examples that are intentionally simplified herein for brevity and clarity. Those skilled in the art will understand that the visualization specification may include many more options, attributes, etc. than shown herein. Also, visualization specification 702 and visualization specification 704 are represented herein using a format / syntax such as JSON, but those skilled in the art will understand that other formats or syntaxes are available. However, those skilled in the art will understand that these simplified visualization specifications are sufficient to at least disclose the novelty disclosed herein.

[0115] In one or more of the various embodiments, the visualization specification defines a visualization model that a visualization engine may adopt to generate a visualization for display to a user or provides a formal or canonical syntax corresponding thereto. In this example, visualization specification 702 may be considered to correspond to source visualization 502 of FIG. 5. For example, visualization specification 702 defines one value with a numeric mark. Also, in this example, visualization specification 702 defines a data source, type information, color, etc. In this example, the "field" attribute indicates that the value shown in the visualization is an integer representing the total of "cases" from "events.db" and should be colored "red".

[0116] Also, in this example, the visualization specification 704 can be considered a visualization specification for a bar graph having two bars (for the "Seattle" case and for the "Tacoma" case).

[0117] In one or more of various embodiments, the metric engine can be arranged to parse or interpret visualization specifications from different sources that may employ different syntaxes, definitions, etc. In one or more of various embodiments, the metric engine can be arranged to employ a parser, grammar, etc., provided via configuration information to process the visualization specification. Thus, in some embodiments, the metric engine can be arranged to operate with different visualization platforms that may employ different visualization specification definitions.

[0118] In one or more of various embodiments, the metric engine can be arranged to employ one or more visualization classifiers to identify which specification formats may be applicable. Also, in some embodiments, the visualization specification can include one or more attributes that store metadata such as specification type, version, vendor, author, authoring tool, etc. Thus, in some embodiments, the metric engine can be arranged to employ the available metadata to determine the type of the visualization specification.

[0119] In one or more of various embodiments, the metric engine can be arranged to employ a visualization classifier to identify the relevant metrics to be determined for each source visualization. Also, in one or more of various embodiments, the visualization classifier can include one or more heuristics for determining whether the metric engine should capture historical metric values for the source visualization.

[0120] In some embodiments, if the metric engine can classify source visualization specifications, the metric engine can determine one or more metrics of interest. Thus, in one or more of various embodiments, if the source visualization specification or source visualization can be classified such that its key metrics can be extracted, historical metric information can be collected. Otherwise, in some embodiments, the historical metric information may not be collected.

[0121] Generalized Operations Figures 8-12 depict generalized operations for providing and surfacing metrics for visualization, according to one or more of the various embodiments. In one or more of the various embodiments, the processes 800, 900, 1000, 1100, and 1200 described in connection with Figures 8-12 may be implemented or executed by one or more processors on a single network computer, such as the network computer 300 of Figure 3. In other embodiments, these processes or portions thereof may be implemented or executed by multiple network computers, such as the network computer 300 of Figure 3. In still other embodiments, these processes or portions thereof may be implemented or executed by one or more virtualized computers, such as those within a cloud-based environment. However, the embodiments are not so limited, and various combinations of network computers, client computers, etc. may be utilized. Further, in one or more of the various embodiments, the processes described in connection with Figures 8-12 may be used to provide and surface metrics for visualization according to at least one of the various embodiments or architectures, such as those described in connection with Figures 4-7. Additionally, in one or more of the various embodiments, some or all of the actions performed by the processes 800, 900, 1000, 1100, and 1200 may be partially executed by a metric engine 322, a visualization engine 324, etc., running on one or more processors of one or more network computers.

[0122] FIG. 8 shows a general flowchart of a process 800 for providing and surfacing metrics for visualization according to one or more of various embodiments. After the START block, in start block 802, in one or more of various embodiments, a dashboard user interface associated with one or more source visualizations may be provided to a metric engine. In one or more of various embodiments, a user may be enabled to select a dashboard or other source visualization for providing and surfacing metrics. In some embodiments, the metric engine may be arranged to automatically process the source visualization associated with the dashboard. In some embodiments, a user or organization may be enabled to set filters, preferences, etc. to determine whether a source visualization or dashboard may be processed to provide and surface metrics for display in a metric visualization.

[0123] In block 804, in one or more of various embodiments, the metric engine may be arranged to employ one or more visualization classifiers to classify one or more source visualizations included in the dashboard user interface. In one or more of various embodiments, the visualization classifier may be arranged to determine whether the source visualization is suitable for providing and surfacing metrics for display in a metric visualization. Also, in one or more of various embodiments, the visualization classification enables the metric engine to employ the correct syntax analysis strategy to identify metrics of interest from the source visualization specification.

[0124] In block 806, in one or more of various embodiments, the metric engine can be arranged to determine one or more metrics for one or more source visualizations based on those classifications. In one or more of various embodiments, the classification process can include identifying metrics for the source visualization.

[0125] In block 808, in one or more of various embodiments, the metric engine can be arranged to sample values for metrics associated with the source visualization. In some embodiments, the sampled metric values can be stored in a metric data store. In one or more of various embodiments, the metric can be associated with a metric profile that can be registered with the metric engine. In some embodiments, the metric profile can include sampling information that determines how often the metric engine samples a given metric.

[0126] In some embodiments, the metric engine can be arranged to sample metric values directly from the source visualization or source visualization model. Thus, in some embodiments, the metric engine can ensure that the sampled metric values match the values displayed in the visualization. In some embodiments, by collecting metric values directly from the source visualization model rather than from the data source, the metric engine is freed from having to perform actions such as filtering, aggregating, averaging, grouping, sorting, etc., that may be required to generate a given metric value. In some embodiments, the visualization engine can initiate one or more actions for retrieving data from a data source that can be used to generate the sampled metric, such as connecting to the data source, providing a query formula, filtering, formatting, etc.

[0127] In block 810, in one or more of various embodiments, the metric engine may be arranged to provide metric visualization based on one or more metric values. In one or more of various embodiments, the metric visualization may be designed to display historical values of the metric. Certain metric visualizations may vary according to the metric being sampled. Also, in some embodiments, the metric engine may be arranged to enable a metric visualization specification to be associated with the metric visualization. Thus, in some embodiments, a user or organization may be enabled to customize those metric visualizations.

[0128] Next, in one or more of various embodiments, control may be returned to the calling process.

[0129] FIG. 9 shows a flowchart of a process 900 for providing and surfacing metrics for visualization, according to one or more of various embodiments. After the START block, in start block 902, in one or more of various embodiments, a source visualization specification associated with a source visualization may be provided to the metric engine. As described above, the dashboard user interface may include or be associated with one or more source visualizations. When the source visualization is processed, the metric engine may be arranged to determine a source visualization specification for some or all of the source visualization. In some cases, the source visualization specification or a reference thereto may be provided directly to the metric engine. In some cases, an identifier or reference to a source visualization specification associated with the source visualization being processed may be provided.

[0130] Alternatively, in one or more of the various embodiments, the metric engine may be arranged to decompile or inspect a source visualization or source visualization model to generate a source visualization specification or source visualization specification information. For example, in some embodiments, if the metric engine is provided with a reference or identifier corresponding to a source visualization model, the metric engine may use conventional or custom reflection techniques to inspect some or all of the metadata, components, objects, data, etc. included in the source visualization model to determine or generate source visualization specification information. Thus, in some embodiments, the metric engine may provide an API or interface that enables the calling process to provide information such as a visualization identifier that enables the metric engine to look up or retrieve the source visualization specification.

[0131] In block 904, in one or more of the various embodiments, the metric engine may be arranged to provide one or more visualization classifiers. In one or more of the various embodiments, the metric engine may be arranged to have access to one or more visualization classifiers. In some embodiments, the metric engine may be arranged to maintain a pool of visualization classifiers. In some embodiments, the metric engine may be arranged to enable a user or organization to provide one or more visualization classifiers or to define which visualization classifier(s) should be used. Thus, in some embodiments, the metric engine may be arranged to determine some or all of the visualization classifiers based on configuration information to account for local environments or local requirements.

[0132] In block 906, in one or more of various embodiments, the metric engine may be arranged to classify source visualizations associated with a source visualization specification based on the source visualization specification and a visualization classifier.

[0133] In one or more of various embodiments, the metric engine may be arranged to execute one or more visualization classifiers against a source visualization specification to classify the source visualization. In one or more of various embodiments, the visualization classifier may define criteria for determining whether the source visualization matches the classifier. In some embodiments, the metric engine may be arranged to execute one or more tests provided or referenced in the visualization classifier. In some embodiments, the visualization classifier may include or reference a grammar used to parse the source visualization specification to identify features necessary to match the classifier. For example, the visualization classifier may be looking for specific features such as specific field definitions, data types, row definitions, axis definitions, etc.

[0134] In decision block 908, in one or more of various embodiments, if a metric in the source visualization can be determined, control may proceed to block 910; otherwise, control may return to the calling process. In one or more of various embodiments, if the source visualization can be classified, it may be assumed to match a visualization classifier that enables identification of key metrics in the source visualization.

[0135] In one or more of the various embodiments, if the source visualization remains unclassified, it may not be suitable for providing and surfacing metrics for display in the metric visualization.

[0136] In block 910, in one or more of the various embodiments, the metric engine may be arranged to determine one or more metrics based on the classified source visualization. In one or more of the various embodiments, the classification may include determining the name, label, etc. of the key metric. In some embodiments, the previous classification step (block 908) may include determining metric information simultaneously with the source visualization being classified.

[0137] Next, in one or more of the various embodiments, control may be returned to the calling process.

[0138] FIG. 10 shows a flowchart of a process 1000 for providing and surfacing metrics for visualization according to one or more of the various embodiments. After the START block, in start block 1002, in one or more of the various embodiments, the metric engine may be arranged to provide the classified source visualization. As described above, the precondition for surfacing metrics in the visualization includes the source visualization being classified as suitable for determining metrics. Thus, in some embodiments, if the source visualization is classified into one or more categories that support metric surfacing, the source visualization or a reference thereto may be provided to the metric engine.

[0139] In block 1004, in one or more of various embodiments, the metric engine may be arranged to determine a metric profile associated with a source visualization. In one or more of various embodiments, the metric engine may be arranged to perform one or more actions to determine metric information from a source visualization or its source visualization specification. In one or more of various embodiments, a particular action may be determined based on a category or class of the source visualization. In some embodiments, source visualizations may be classified into various classes, such as single metric, multi-metric, time-based metric, etc.

[0140] In some embodiments, a single metric source visualization may be a visualization having one metric (e.g., source visualization 502). Similarly, in some embodiments, a multi-metric source visualization may include single metrics decomposed into multiple categories. For example, a single metric source visualization such as visualization 502 has a metric of "Case". The multi-metric source visualization can also plot the values for "Case", but is decomposed by city. In some embodiments, the metric engine may be arranged to consider a multi-metric metric as one metric having multiple categories rather than as separate metrics.

[0141] In one or more of various embodiments, the available metrics or metric types may be affected by the definition or syntax of the source visualization specification. Thus, in some embodiments, if the syntax of the source visualization specification cannot represent multi-metric visualization, they may not be available for surfacing metrics. In some embodiments, some of the actions for classifying the source visualization may include determining which metrics within the source visualization can be identified or how they can be associated with other metrics or sub-metrics.

[0142] In one or more of various embodiments, the metric engine may be arranged to employ the structural information embedded in the source visualization specification to determine the metrics of interest. Similarly, in some embodiments, the structural information may be employed to determine how to capture or extract the values of the metrics. For example, the source visualization specification may include a data binding syntax that maps marks or plot lines within the source visualization to fields within a data model or data source. Thus, in one or more of various embodiments, the metric engine may be arranged to employ this information to determine how to extract data from the corresponding source visualization model.

[0143] In some embodiments, the metric engine may be arranged to decode / inverse-reference labels, indirect references, position indicators, etc., that may be associated with the metric definitions included in the source visualization specification.

[0144] Accordingly, in one or more of various embodiments, the metric engine may be arranged to interpret or parse the source visualization specification of the source visualization in order to determine information for inclusion in the metric profile. Note that in one or more of various embodiments, the metric engine may require rules, code, or instructions specifically tailored to the definition / syntax rules associated with the source visualization specification.

[0145] Accordingly, in some embodiments, the same visualization classifier that classified the source visualization may include rules for determining metrics. In some embodiments, the classification of the source visualization may provide some or all of the information about the metric profile.

[0146] In one or more of various embodiments, the metric engine may be arranged to provide a metric profile that includes (or references) information necessary to sample metric values, including scripts, query expressions, connection strings, credential information, etc., the details of which are determined based on the source visualization specification or the visualization classifier.

[0147] In some embodiments, the metric engine may be arranged to include (or reference) a metric visualization specification included in or associated with the metric profile. In some embodiments, the metric visualization specification may include visualization specification information that can be used to determine how to generate a metric visualization that displays metric values.

[0148] Thus, in one or more of the various embodiments, the metric engine may be arranged to employ rules, specifications, parameters, grammars, parsers, etc., provided via configuration information to account for differences between visualization specifications used by different visualization platforms. Similarly, in some embodiments, the configuration information may be used to provide user or organizational preferences regarding the appearance of metric visualizations and the like.

[0149] Furthermore, in one or more of the various embodiments, the metric profile may also be employed to store sampling rules. In other embodiments, the metric profile may be associated (e.g., registered) with the sampling rules. Thus, in some embodiments, some metric profiles may share the same sampling rules without the need to individually store or reference them within the metric profile.

[0150] In block 1006, in one or more of the various embodiments, the metric engine may be arranged to register a metric profile and set one or more sample rules for source visualization. In one or more of the various embodiments, the metric engine may be arranged to add the metric profile to a sampling registry (e.g., a database, catalog, lookup table, etc.) indicating that the metric should be sampled according to the sampling rules associated with the metric. Thus, in one or more of the various embodiments, the metric engine may be arranged to query the metric profile registry to determine which metric should be sampled at a given time. Alternatively, in some embodiments, the registry may include a time-based partition that can be walked through so that the metric engine can select those metric profiles to sample the metric if the metric profiles are registered during the time partition being visited.

[0151] In decision block 1008, in one or more of various embodiments, if a metric value is to be sampled, control may proceed to block 1010; otherwise, control may loop back to decision block 1008. In one or more of various embodiments, if a metric engine determines that one or more metric profiles are to be sampled, the metric profiles may be retrieved and processed accordingly. In some cases, when time-division sampling (such as sampling every 10 minutes) is used, there may be no metric profiles registered for sampling in the time division. Also, in some cases, if the conditions required by one or more sampling rules are not met, process 1000 may wait for those conditions to be met before sampling the metric.

[0152]

[0153]

[0154] ​​In one or more of various embodiments, the metric engine may be enabled to sample metrics even if the source visualizations to which they may be associated are inactive or otherwise not in use. In some embodiments, if the source visualization may be inactive (e.g., not displayed on a hardware display), the visualization engine may perform one or more actions to update the metric value before the metric value is sampled. For example, in some embodiments, the visualization engine may update the metric value before it is sampled by the metric engine if it determines that the metric engine is attempting to sample a value from an inactive source visualization.

[0155] Alternatively, in some embodiments, the metric profile may include a query formula, connection string, section residency information, etc. that enables the metric value to be retrieved directly from the data source.

[0156] In some embodiments, the metric may be based on one or more data source fields that may be modified by one or more functions or formulas. Thus, in one or more of various embodiments, the metric engine may be arranged to execute the same one or more functions or formulas against the metric value retrieved from the data source. In some cases, the formula or function may be provided directly to the data source that may execute them before returning the results to the metric engine.

[0157] In one or more of various embodiments, the metric engine may be arranged to store the sampled metric values in a metric data store. In some embodiments, the metric data store may be a time series database that enables metric values to be associated with specific time bins (or buckets). In other embodiments, the metric engine may store sample time information along with the metric values that can be stored in the stored metrics, without requiring a formal / dedicated time series database.

[0158] In decision block 1012, in one or more of various embodiments, if sampling can be terminated, control may return to the calling process; otherwise, control may loop back to decision block 1008. Generally, in some embodiments, the metric engine may continuously sample metrics based on their registered metric profiles. If sampling is disabled or interrupted, control may return to the calling process.

[0159] Figure 11 shows a flowchart of a process 1100 for providing and surfacing metrics for visualization and displaying metric visualizations for visualization according to one or more of various embodiments. After the START block, in decision block 1102, in one or more of various embodiments, if metric visualizations can be displayed, control can proceed to block 1104; otherwise, control can loop back to decision block 1102. In one or more of various embodiments, a visualization platform can provide one or more user interface controls that enable a user to view metric visualizations that can be associated with one or more key metrics. In some embodiments, a dashboard creator can include user interface controls that enable a user to activate one or more metric visualizations. Similarly, in some embodiments, one or more metric visualizations can be included in a dashboard user interface along with one or more other visualizations.

[0160] In some embodiments, a metric engine can be arranged to automatically activate one or more metric visualizations that are associated with anomalies or otherwise meet one or more defined conditions.

[0161] In one or more of various embodiments, a metric engine can be provided with references or other identifiers that can be associated with a metric profile, origin source visualization, origin dashboard, and the like.

[0162] In block 1104, in one or more of various embodiments, the metric engine may be arranged to provide a metric profile. In one or more of various embodiments, the metric engine may be arranged to determine a metric profile for metric visualization. In some embodiments, an identifier or other reference to the metric profile may be provided. Thus, in some embodiments, the metric engine may be arranged to retrieve the metric profile from a database, catalog, hash map, lookup table, etc.

[0163] In block 1106, in one or more of various embodiments, the metric engine may be arranged to determine one or more metric values from a metric data store. In one or more of various embodiments, the metric engine may be arranged to employ connection information (e.g., connection string, credentials, port, network address, API call, query formula, etc.) included in or associated with the metric profile to access metric values stored in the metric data store.

[0164] In one or more of various embodiments, the metric profile may include window boundaries for limiting the number of values retrieved from the metric data store. For example, if metric values are stored in a time series database (or otherwise associated with time buckets), the metric engine may be arranged to retrieve, e.g., "the last 5 days" of metric values.

[0165] In block 1108, in one or more of various embodiments, the metric engine can be arranged to generate a metric visualization based on a metric profile and one or more metric values. In one or more of various embodiments, the metric profile can include or reference a metric visualization specification that can be adopted by the visualization engine to render the metric visualization. In some embodiments, the metric visualization can be based on one or more default metric visualization specifications. Alternatively, in some embodiments, a user or organization may be enabled to customize one or more metric visualizations.

[0166] Thus, in one or more of various embodiments, the metric engine can be arranged to adopt a metric profile, metric values, metric visualization specifications, etc., to enable the visualization engine to generate or display a metric visualization.

[0167] Next, in one or more of various embodiments, control can be returned to the calling process.

[0168] Figure 12 shows a flowchart of process 1200 for monitoring metrics related to anomalies as part of providing and surfacing metrics for visualization according to one or more of various embodiments. After the START block, at block 1202, in one or more of various embodiments, the metric engine may be arranged to determine a metric profile for a metric. In one or more of various embodiments, the metric engine may be arranged to register a metric with the metric profile. In one or more of various embodiments, the metric profile for a metric may include information such as a sampling rate, source visualization, etc. In some embodiments, the metric engine may be arranged to associate one or more registered metric profiles with a timer, cron job, watchdog process, etc.

[0169] At block 1204, in one or more of various embodiments, the metric engine may be arranged to sample values for one or more metrics and store them in a metric data store. As described above, the metric engine may be arranged to continuously sample metric values and store them in the metric data store. In one or more of various embodiments, a sample rate, collection size (number of samples to maintain), etc. may be determined based on the metric profile associated with the metric.

[0170] In block 1206, in one or more of various embodiments, the metric engine may be arranged to employ one or more anomaly detectors to evaluate metric values to discover one or more anomalies. In one or more of various embodiments, the metric engine may be arranged to determine one or more anomaly detectors that may be employed to identify anomalies in metric values. In some embodiments, some metrics may be associated with specific preferred anomaly detectors that may be adjusted according to one or more characteristics of the metric or its associated visualization.

[0171] In one or more of various embodiments, the metric engine may be arranged to execute two or more anomaly detectors against metric values. Thus, in some embodiments, in some cases, two or more anomalies may be detected (almost) simultaneously. In some embodiments, a composite anomaly detector may be an anomaly detector that includes one or more other anomaly detectors.

[0172] In one or more of various embodiments, the anomaly detector may be arranged to apply one or more well-known or conventional statistical operations to determine anomalies such as outliers, missing values, etc. Further, in some embodiments, the metric engine may be arranged to employ one or more anomaly detectors customized based on user or organizational preferences. For example, in some embodiments, a user may have an arbitrarily defined test that checks specific metric values that may not appear anomalous using conventional statistical methods.

[0173] In one or more of various embodiments, one or more anomaly detectors may include heuristics that may be tested before or after performing more conventional statistical tests.

[0174] In one or more of the various embodiments, the simplest anomaly detection scheme may look for changes greater than "normal", calculate the daily differences, and look for those that are statistically anomalous, e.g., a number of standard deviations from the average difference. In some embodiments, the anomaly detector may be arranged to employ a more complex scheme that fits an analytical model to the metric values and then flags an anomaly if the predicted and actual metric values deviate beyond an acceptable threshold amount.

[0175] In one or more of the various embodiments, one or more anomaly detectors may be arranged to generate an intensity score or the like related to the importance or criticality of a given anomaly. Similarly, in some embodiments, the anomaly detector may be arranged to provide a confidence score or the like related to the certainty of the detected anomaly or "closeness of match".

[0176] Thus, in some embodiments, the metric engine may be arranged to enable a user or organization to set notification rules or filters based on, for example, intensity scores, confidence scores, and the like.

[0177] In decision block 1208, in one or more of the various embodiments, if an anomaly can be detected, control may proceed to block 1210; otherwise, control may loop back to block 1204. In some embodiments, if one or more anomalies can be detected by one or more anomaly detectors, the metric engine may be arranged to compare one or more of the anomaly type, affected metric, intensity score, confidence score, etc. to one or more notification rules to determine whether the detected anomaly should be ignored or discarded. In some embodiments, such notification rules may be associated with a user, organization, metric or metric profile, dashboard, visualization, etc.

[0178] In block 1210, in one or more of various embodiments, the metric engine may be arranged to generate one or more notifications or report information based on detected anomalies. In one or more of various embodiments, in the absence of rules to ignore or suppress anomalies, the metric engine may be arranged to determine one or more notification methods or one or more notification targets. In one or more of various embodiments, the metric engine may be arranged to employ rules or instructions provided via configuration information to determine a notification method or a notification target. In some embodiments, the metric engine may generate one or more event messages associated with the anomaly and provide them to a third-party monitoring service, which may route the notification to a responsible party or, if any, track its resolution.

[0179] In one or more of various embodiments, the notification may include sending a text message, an email, other messages via other messaging applications, an audio alert, etc. In some embodiments, the metric engine may be arranged to register the notification with a visualization engine so that alert information associated with the anomaly can be displayed on a user interface. In some cases, for some embodiments, visual alerts may be displayed on a dashboard user interface including source visualization or metric visualization associated with the anomaly. In some embodiments, the notification may include generating a log entry that can be reviewed later.

[0180] In one or more of various embodiments, the content of the notification may vary depending on the type of anomaly, metric, source visualization, dashboard, etc. Thus, in some embodiments, the metric engine may be arranged to determine notification rules or notification formats based on configuration information to account for local situations or local requirements.

[0181] Next, in one or more of the various embodiments, control may be returned to the calling process.

[0182] It will be understood that each block in each flowchart diagram, and combinations of blocks in each flowchart diagram, can be implemented by computer program instructions. These program instructions can be provided to a processor to generate a machine, such that the instructions executed on the processor create means for implementing the actions specified in one or more of each flowchart block. The computer program instructions can be executed by a processor to cause the processor to execute a series of operational steps to provide steps for implementing the actions specified in one or more of each flowchart block to generate a computer implemented process. The computer program instructions can also cause at least some of the operational steps shown in the blocks of each flowchart to be executed simultaneously. Further, some of the steps can also be executed across two or more processors so as to occur in a multiprocessor computer system. Additionally, one or more blocks or combinations of blocks in each flowchart diagram can be executed simultaneously with other blocks or combinations of blocks, or in a different order than that shown, without departing from the scope or spirit of the present invention.

[0183] Accordingly, each block of each flowchart diagram supports a combination of means for performing a specified action, a combination of steps for performing a specified action, and program instruction means for performing a specified action. It will also be understood that each block of each flowchart diagram, and combinations of blocks of each flowchart diagram, can be implemented by a dedicated hardware-based system for performing a specified action or step, or by a combination of dedicated hardware and computer instructions. The foregoing examples are not to be construed as limiting or exhaustive, but rather as illustrative use cases for showing at least one implementation of various embodiments of the present invention.

[0184] Furthermore, in one or more embodiments (not shown), the logic within an exemplary flowchart can be executed using an embedded logic hardware device such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable array logic (PAL), or a combination thereof, instead of a CPU. The embedded logic hardware device can directly execute its embedded logic to perform an action. In one or more embodiments, a microcontroller can be arranged to directly execute its own embedded logic to perform an action and to access its own internal memory and its own external input and output interfaces (e.g., hardware pins or wireless transceivers) to perform an action, such as in a system on chip (SOC).

Claims

1. A method for generating a metric based on visualization using one or more processors that execute instructions for performing an action, the method comprising: Providing a dashboard associated with one or more source visualizations that each display a current value of one or more metrics from one or more source visualization models, where each source visualization corresponds to a specification and a source visualization model; Evaluating each specification to determine one or more characteristics of each source visualization, where the one or more source visualizations are classified based on one or more classifiers and the one or more characteristics; Determining the one or more metrics for each classified source visualization based on the one or more classifiers; Generating one or more metric profiles corresponding to the one or more metrics based on the one or more classifiers; Sampling the one or more source visualization models to provide one or more values of the one or more metrics, where the sampling rate is based on the one or more metric profiles; Storing the one or more sampled values in a metric data store along with one or more time values, where the one or more time values correspond to when the one or more values were sampled; Generating one or more metric visualizations based on the one or more values and the one or more time values, where the one or more metric visualizations display one or more previously sampled values of the one or more metrics; A method comprising the above steps.

2. The step of evaluating each specification comprises: Repeatedly processing the one or more classifiers to determine a class of visualizations corresponding to the one or more source visualizations; Performing one or more actions to determine the one or more characteristics of each source visualization based on its corresponding class; Excluding each of the one or more source visualizations that remain unclassified; The method according to claim 1, further comprising: **Claim 3** The step of displaying the one or more metric visualizations further comprises displaying the one or more metric visualizations on the dashboard or another user interface. The method according to claim 1. **Claim 4** The step of determining the one or more metrics further comprises determining one or more of one or more single-value metrics or one or more multi-value metrics, wherein each of the one or more multi-value metrics is a single metric divided into two or more categories. The method according to claim 1. **Claim 5** Providing one or more anomaly detectors configured to identify one or more statistical anomalies present in the one or more values of the one or more metrics; In response to determining one or more statistical anomalies based on the one or more anomaly detectors, Providing one or more alerts including one or more of one or more notifications, one or more alerts, or one or more reports, and Communicating the one or more alerts to one or more responsible parties or one or more services; Performing further actions including: The method according to claim 1, further comprising: **Claim 6** The step of sampling the one or more source visualization models to provide the one or more values of the one or more metrics further comprises sampling the one or more source visualization models while the dashboard or the one or more source visualizations are inactive, and the inactive dashboard or the one or more inactive source visualizations are omitted from being displayed. The method according to claim 1. **Claim 7** A network computer for generating metrics based on visualization, including at least a memory for storing instructions, and one or more processors for executing instructions to perform actions , wherein the actions include: providing a dashboard associated with one or more source visualizations that each display a current value of one or more metrics from one or more source visualization models, each source visualization corresponding to a specification and a source visualization model; evaluating each specification to determine one or more characteristics of each source visualization, the one or more source visualizations being classified based on one or more classifiers and the one or more characteristics; determining the one or more metrics for each classified source visualization based on the one or more classifiers; generating one or more metric profiles corresponding to the one or more metrics based on the one or more classifiers; sampling the one or more source visualization models to provide one or more values of the one or more metrics, the sampling rate being based on the one or more metric profiles; storing the one or more sampled values in a metric data store along with one or more time values, the one or more time values corresponding to when the one or more values were sampled; generating one or more metric visualizations based on the one or more values and the one or more time values, the one or more metric visualizations displaying one or more previously sampled values of the one or more metrics . **Claim 8** The step of evaluating each specification includes: repeatedly processing the one or more classifiers to determine a class of visualizations corresponding to the one or more source visualizations; Performing one or more actions to determine the one or more characteristics of each source visualization based on its corresponding class; Excluding each of the one or more source visualizations that remain unclassified; The network computer according to claim 7, further comprising.

9. The step of displaying the one or more metric visualizations further includes the step of displaying the one or more metric visualizations on the dashboard or another user interface. The network computer according to claim 7.

10. The step of determining the one or more metrics further includes determining one or more of one or more single-value metrics or one or more multi-value metrics, and each of the one or more multi-value metrics is a single metric divided into two or more categories. The network computer according to claim 7.

11. The one or more processors are Providing one or more anomaly detectors arranged to identify one or more statistical anomalies present in the one or more values of the one or more metrics; In response to determining one or more statistical anomalies based on the one or more anomaly detectors, Providing one or more alerts including one or more of one or more notifications, one or more alerts, or one or more reports, and Communicating the one or more alerts to one or more responsible parties or one or more services Including performing further actions; Executing instructions to perform actions further including. The network computer according to claim 7.

12. The step of sampling the one or more source visualization models to provide the one or more values of the one or more metrics further includes sampling the one or more source visualization models while the dashboard or the one or more source visualizations are inactive, and the inactive dashboard or the one or more inactive source visualizations are omitted from being displayed. The network computer according to claim 7.

13. A system for generating metrics based on visualizations via a network, comprising: a network computer; and a client computer wherein the network computer comprises at least a memory for storing instructions, and one or more processors for executing instructions for performing actions, and the actions include: providing a dashboard associated with one or more source visualizations that each display a current value of one or more metrics from one or more source visualization models, wherein each source visualization corresponds to a specification and a source visualization model; evaluating each specification to determine one or more characteristics of each source visualization, wherein the one or more source visualizations are classified based on one or more classifiers and the one or more characteristics; determining, based on the one or more classifiers, the one or more metrics for each classified source visualization; generating, based on the one or more classifiers, one or more metric profiles corresponding to the one or more metrics; and sampling the one or more source visualization models to provide one or more values of the one or more metrics, wherein the sampling rate is based on the one or more metric profiles. ​ Storing the one or more sampled values in a metric data store along with one or more time values, wherein the one or more time values correspond to when the one or more values were sampled; Generating one or more metric visualizations based on the one or more values and the one or more time values, wherein the one or more metric visualizations display one or more previously sampled values of the one or more metrics; comprising; The client computer comprises; At least a memory storing instructions; One or more processors executing instructions to perform actions; wherein the actions comprise; Displaying the dashboard and the one or more source visualizations; A system.

14. The step of evaluating each specification comprises; Repeatedly processing the one or more classifiers to determine a class of visualizations corresponding to the one or more source visualizations; Performing one or more actions to determine the one or more characteristics of each source visualization based on its corresponding class; Excluding each of the one or more source visualizations that remain unclassified; The system according to claim 13, further comprising.

15. A network computer for generating metrics based on visualizations, comprising; At least a memory storing instructions; One or more processors executing instructions to perform actions; wherein the actions comprise; Providing a dashboard associated with one or more source visualizations that each display a current value of one or more metrics from one or more source visualization models, wherein each source visualization corresponds to a specification and a source visualization model; Evaluating each specification to determine one or more characteristics of each source visualization, wherein the one or more source visualizations are classified based on one or more classifiers and the one or more characteristics; determining, based on the one or more classifiers, the one or more metrics for each classified source visualization; generating, based on the one or more classifiers, one or more metric profiles corresponding to the one or more metrics; sampling the one or more source visualization models to provide one or more values of the one or more metrics, wherein a sampling rate is based on the one or more metric profiles; storing the one or more sampled values in a metric data store along with one or more time values, wherein the one or more time values correspond to when the one or more values were sampled; generating one or more metric visualizations based on the one or more values and the one or more time values, wherein the one or more metric visualizations display one or more previously sampled values of the one or more metrics; A network computer including the above.

Images