Data visualization objects in virtual environments

Abstract

Multiple visualization objects can be provided for representing one or more data sets in a virtual 3D space. The visualization objects may include funnels, containers, business cards, etc. The visualization objects can be arranged in a circular wheel that can rotate around the position of a virtual camera or user in a VR / AR environment. Individual data points in the visualization object can be rotated, resized, positioned, colored, or otherwise characterized based on the properties of the corresponding data point. Individual data points can also be animated as transitions between visualization objects in a unified view. Voice commands can be interpreted as part of an interactive environment that can provide views of visualization objects to multiple devices simultaneously.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is a non-provisional application of U.S. Provisional Application No. 62 / 748,504, filed on October 21, 2018, entitled “3D VISUALIZATION SERVICES INTEGRATING VARIOUS DATA SOURCES AND DISPLAY DEVICES,” and claims the benefit and priority thereof under 35 U.S.C. 119(e), the entire contents of which are incorporated herein by reference for all purposes.

[0003] This application is also related to the following four U.S. patent applications:

[0004] U.S. Patent Application No. 16 / 658,162, filed on October 21, 2019 by Moroze et al., entitled “INTERACTIVE DATA EXPLORER AND 3-D DASHBOARD ENVIRONMENT” (Attorney Docket No. 088325-1142774), which is incorporated herein in its entirety.

[0005] U.S. Patent Application No. 16 / 658,165, filed on October 21, 2019 by Moroze et al., entitled “FUNNEL VISUALIZATION WITH DATAPOINT ANIMATIONS AND PATHWAYS” (Attorney Docket No. 088325-1142775), which is incorporated herein in its entirety.

[0006] U.S. Patent Application No. 16 / 658,169, filed on October 21, 2019 by Moroze et al., entitled “OPTIMIZING VIRTUAL DATA VIEWSUSING VOICE COMMANDS AND DEFINED PERSPECTIVES” (Attorney Docket No. 088325-1142776), which is hereby incorporated by reference in its entirety.

[0007] U.S. Patent Application No. 16 / 658,177, filed on October 21, 2019 by Moroze et al., entitled “ANIMATION BETWEEN VISUALIZATION OBJECTS IN A VIRTUAL DASHBOARD” (Attorney Docket No. 088325-1142782), which is hereby incorporated in its entirety. Background Art

[0008] One of the most effective ways to communicate information is through visualization. Capturing and categorizing information is often not enough to gain a complete understanding of the conclusions that can be drawn from the data. While two-dimensional data visualizations dominate many of our communication techniques, many of these visualization techniques are not optimized for highlighting or emphasizing various aspects of the data collection. Effective data visualization involves creating informative visuals that communicate one aspect of the data very quickly. Sometimes, a simple visualization is sufficient. However, with complex multi-dimensional data, the complexity of the data set itself can make adequate visualization difficult for a limited dimensional space. Summary of the invention

[0009] A method for visualizing time-series data in an interactive virtual environment may include: receiving one or more data sets, the one or more data sets including multiple time-series sequences of data points; plotting each of the multiple time-series sequences of data points in a three-dimensional (3D) line graph in the virtual environment; generating one or more views of the 3D line graph in the virtual environment using one or more virtual cameras configured to move around in the 3D line graph; and sending the one or more views of the 3D line graph to one or more client devices. The one or more client devices correspond to the one or more virtual cameras.

[0010] A non-transitory computer-readable medium may include instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: receiving one or more data sets, the one or more data sets including multiple time-series sequences of data points; plotting each of the multiple time-series sequences of data points in a three-dimensional (3D) line graph in a virtual environment; generating one or more views of the 3D line graph in the virtual environment using one or more virtual cameras configured to move around in the 3D line graph; and sending the one or more views of the 3D line graph to one or more client devices. The one or more client devices correspond to the one or more virtual cameras.

[0011] A system may include one or more processors and one or more memory devices, the one or more memory devices may include instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including: receiving one or more data sets, the one or more data sets including multiple time-series sequences of data points; plotting each of the multiple time-series sequences of data points in a three-dimensional (3D) line graph in a virtual environment; generating one or more views of the 3D line graph in the virtual environment using one or more virtual cameras configured to move around in the 3D line graph; and sending the one or more views of the 3D line graph to one or more client devices. The one or more client devices correspond to the one or more virtual cameras.

[0012] In any embodiment, any of the following features may be implemented in any combination and without limitation. Plotting each of a plurality of time series of data points in a virtual environment as a 3D line graph may include plotting the position of each of the plurality of time series of data points in a coordinate system of the virtual environment; and connecting each of the positions with a 3D line object. The method / operation may also include converting the coordinate system of the 3D line graph to the coordinate system of the virtual environment. The coordinate system of the 3D line graph may include: a first horizontal axis corresponding to time; a second horizontal axis corresponding to an index in the plurality of time series of data points; and a vertical axis corresponding to the value of the data point. One or more data sets may include test failure data of different systems over time. The method / operation may also include: loading a visualization component with parameterized inputs; binding one or more data sets to the parameterized inputs; and plotting each of the plurality of time series of data points in a virtual environment as a 3D line graph according to the visualization component. The client device in the one or more client devices may include a virtual reality device. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] A further understanding of the nature and advantages of the present invention may be achieved by reference to the remainder of this specification and the drawings, wherein like reference numerals are used throughout the several drawings to refer to similar components. In some cases, a sub-label is associated with a reference numeral to indicate one of multiple similar components. When a reference numeral is referenced without specifying an existing sub-label, it is intended to refer to all such multiple similar components.

[0014] Figure 1 A system for authoring, providing, and using interactive data displays is shown in accordance with some embodiments.

[0015] Figure 2 An example of a virtual object that has been bound to an enterprise dataset is shown in accordance with some embodiments.

[0016] Figure 3 A user interface for importing components from a component exchanger into an application is shown.

[0017] Figure 4 A block diagram of a process for downloading and instantiating components in an application is shown in accordance with some embodiments.

[0018] Figure 5 A virtual dashboard created in a virtual environment is shown in accordance with some embodiments.

[0019] Figure 6 It shows how various types of client device systems and input devices can all interface with the same virtual environment in an EDIE environment simultaneously according to some embodiments.

[0020] Figure 7 A view of a virtual environment through a 2D screen interface is shown in accordance with some embodiments.

[0021] Figure 8 A view of a virtual environment is shown with multiple users interacting with a virtual dashboard in accordance with some embodiments.

[0022] Fig. 9 Shown is a view of a real world environment that can be used to interact with a virtual environment in accordance with some embodiments.

[0023] Fig.10 A flow chart of a method for displaying data in a multi-dimensional dashboard according to some embodiments is shown.

[0024] Fig.11 A diagram illustrating a visualization component for displaying a time-series sequence of data points in a virtual environment in accordance with some embodiments.

[0025] Fig.12 According to some embodiments, Fig.11 Perspectives Different angles of view of a line drawing object.

[0026] Fig.13 A line drawing object is shown as it is rendered by a virtual environment in accordance with some embodiments.

[0027] Fig.14 Shown is how a user can interact with data in a virtual environment according to some embodiments.

[0028] Fig.15 A circular bar chart component is shown that may be used to visualize the same data that is bound to a line chart component, according to some embodiments.

[0029] Fig.16A circular graph is shown that operates in a similar manner to a line graph in accordance with some embodiments.

[0030] Fig.17 A flow chart of a method for visualizing time series data in an interactive virtual environment according to some embodiments is shown.

[0031] Fig.18 Depicted is a simplified diagram of a distributed system for implementing some embodiments.

[0032] Fig.19 is a simplified block diagram of one or more components of a system environment according to some embodiments through which services provided by one or more components of an embodiment system may be provided as a cloud service.

[0033] Fig. 20 An exemplary computer system is shown in which various embodiments may be implemented. DETAILED DESCRIPTION

[0034] Described herein are embodiments of immersive user interface (UI) experiences. Immersive user interface experiences can be packaged as individual components that can be plugged into a component exchange or clearinghouse, which can later be used by application developers or service developers to add interactive UI components to their applications or services. These UI components can be used to differentiate applications in a crowded marketplace and to provide user-friendly interactive immersive experiences that provide detailed and intuitive data visualization for many different data types on many different user devices.

[0035] Figure 1 A system for creating, providing, and using interactive data displays according to some embodiments is shown. The system may include a component exchanger 124. The component exchanger 124 may be part of an application development kit that allows users to develop applications and / or services using a custom development environment and a library of predefined tools and / or interfaces. The application development kit may be a cloud-hosted development environment that can be accessible to many different client devices as a software as a service (SaaS) model. The development environment can be configured to develop and / or deploy many different types of applications, including: cloud-based services or microservices, web applications, mobile device applications, enterprise applications, and / or the like. Applications designed in the application development kit can be integrated with other SaaS applications via a representative state transfer (REST) ​​interface.

[0036] The development environment in an application development kit may include many user-friendly tools that developers can use to streamline the development process and make the entire application development cycle more efficient. These tools may include WYSIWYG tools and other graphical interfaces that allow developers to drag and drop various pre-packaged components into the application development space. The development environment may provide various components, such as reusable business objects, processes, web controls, user interface controls, themes, templates, and other aspects of an application.

[0037] Developers can use the development environment to design and test applications, which can then be deployed to various operating environments. When the application is deployed to the operating environment, the development environment can also deploy any additional code or libraries required by the application so that it can run in the target environment. Therefore, the flexibility and reusable components provided by the development environment can be ported to any operating environment without losing any of the functionality provided by the development environment.

[0038] In this example, the development environment may include a component exchanger. For example, you can use a component exchanger such as Visual The development environment may include various toolkits, such as a collection of proprietary and / or open source JavaScript libraries that can be used to build applications. For example, the development environment may include JavaScript Extension Toolkit (JET) Components in these toolkits may include a rich set of user interface (UI) components, including data models, view models, and ports for binding data to components. In some embodiments, the development environment may provide a unified component plug-in mechanism based on standard components such as the W3C Web component standard, while still allowing custom components to work with standard components.

[0039] The component exchanger 124 can allow component developers to upload components to the component exchanger 124 for use by application developers. This allows component developers to provide a flexible component architecture that can be maintained and upgraded throughout the component life cycle. For example, these components may include standard web components, such as a check box component 116 or a slider component 114 that can be used in various user interfaces. These components 114, 116 can be uploaded to the component exchanger 124 and then updated and maintained by the component development team throughout their life cycle. They can then be provided to application developers as components 121, 122 in the component exchanger 124.

[0040] In addition to standard web components, the embodiments described herein also allow users to develop special visualization objects that can be used in an enterprise data interactive resource manager (EDIE) environment. These visualization objects provide multidimensional views of enterprise data sets that can be viewed simultaneously by multiple client devices through separate network connections. These visualization objects can be imported into the component exchanger 124 in the same manner as standard web components are imported into the component exchanger 124. These visualization objects can then be imported into various applications being developed. In those applications, developers can bind various enterprise data sets from many different sources to parameterized inputs on the visualization objects. The visualization objects can then be used to generate rendered views of virtual objects that represent and visualize enterprise data in a multidimensional environment.

[0041] Various examples of these visualization objects may be described in more detail below in this disclosure. Figure 1 Some examples of visualization objects that can be imported into the component exchanger 124 as regular objects are shown. For example, a funnel object 112 may include a rotating funnel object that shows the progression of data points in an enterprise data set over time. A variety of visual effects can be provided with the funnel object 112, which can be used to highlight various aspects of the data set bound to the funnel object 112. For example, a user can search the data set along various dimensions, change individual data points, view the path of data points over time, see a visual representation of the data as it is accelerated through time along various dimensions, and so on.

[0042] Figure 1 Also shown is a three-dimensional histogram or bar graph object 110 that can be used to represent various enterprise data sets. In the component exchanger 124, the graphic object 110 can be disconnected from any underlying data set. However, when imported into an actual application, the graphic object 110 can be bound to the enterprise data set, and the individual bars in the graphic can be resized based on the data binding. The visualization object can be used to generate a three-dimensional (3D) virtual object in a virtual environment that can be viewed by various client devices. For example, the graphic object 110 can be implemented as a plurality of three-dimensional cylinders, each of which represents a bar in the graphic. Users can enter the virtual environment and walk around in the virtual object, interact with the object, change the value in the object, manipulate the parts of the object, and have those changed values ​​seen by other users and stored back to the underlying database storing the data set.

[0043] The above visualization objects 110, 112 are provided as examples and are not meant to be limiting. Any multidimensional object that can be used to visualize enterprise data sets can be used as a visualization object in the present disclosure. Other visualization objects may include charts, animations, pictures, graphics, etc.

[0044] In the component exchanger 124, the visualization objects can be represented as components 118, 120, which can be selected by the developer and imported into the application during development. For example, the developer can drag and drop the component 118 of the funnel object 112 into the application. The visualization object can then be bound to a data set in the development environment or the deployment environment, depending on the embodiment.

[0045] In some embodiments, applications including one or more visualization objects may be deployed to various environments. These environments may include web applications, mobile applications, client-side applications, applications running on gaming systems, applications running on desktop or laptop computers, applications for virtual environments (such as virtual reality or augmented reality systems), and / or any other computing environments. When an application is deployed to an environment, component exchanger 124 may deploy any necessary libraries, third-party libraries, or other software components or libraries required to run the visualization object. For example, certain visualization objects running in a 3D environment may include OpenGL libraries or other 3D visualization libraries required for their operation.

[0046] The application can be deployed to an environment 102, generally referred to herein as an EDIE environment, which can include a number of standard components and one or more visualization objects as described above. The environment 102 can include a virtual environment 104 in which visualization objects can be instantiated. When visualization objects are instantiated, they can be bound to one or more underlying data sets. These data sets can be obtained from a variety of sources such as database tables, multidimensional data cubes, web services, server-side applications, API interfaces, etc. In Figure 1 In an example of , the enterprise database 108 can provide values ​​from one or more database tables to the environment 102 for use with at least one of the visualization objects. In this example, the funnel object 112 can be deployed to the environment 102 in the virtual environment 104. The funnel object 112 can include a set of inputs, such as parameterized inputs, that can be individually and / or collectively bound to tables or rows in an enterprise dataset from the enterprise database 108. Bindings 106 between dataset values ​​and parameterized inputs of visualization objects can be created in a development environment, in a deployment environment 102, and / or at any other time. The bindings 106 can also be adjusted or reformulated with different datasets dynamically at runtime, or adjusted or reformulated with different datasets before executing the application.

[0047] When a data set is bound to a visualization object, the visualization object can be used to generate a virtual object in the virtual environment 104. This can include using values ​​from the data set to determine the size or dimensions of various parts of the virtual object. Values ​​from the data set can also be used to generate animation speeds or motion vectors for objects within the virtual object. Values ​​from the data set can also be used to color or otherwise affect the visual representation of objects in the virtual object. For example, a data point can rotate around a virtual funnel created by the funnel object 112. The data point can have a color determined by one dimension, a size or radius determined by a second dimension, a rotation speed determined by a third dimension, a trajectory or motion vector direction determined by a fourth dimension, and so on.

[0048] The visualization objects generated in the virtual environment 104 can be viewed and / or interacted with by a plurality of different client devices 126, 128, 130, 132. These client devices may include laptops, desktop computers, workstations, tablet computers, smart phones, smart watches, smart TVs, video game consoles, digital home assistants, smart glasses, virtual reality helmets, augmented reality helmets, and any other computing devices. Each of these client devices 126, 128, 130, 132 can receive a unique view into the virtual environment 104, which corresponds to the uniquely placed virtual camera of each client device 126, 128, 130, 132. The user can then interact with the virtual object 105 in the virtual environment 104 to view, manipulate, edit, and / or update the underlying data set using the virtual object 105. As described below, each user can be provided with a unique specific view of the virtual object 105, and different settings can allow the user to change the underlying data so that the virtual object 105 changes in the view provided to other client devices. At each client device 126, 128, 130, 132, automatic hardware detection may be used to provide a correct view of the virtual environment 104. For example, if the client device 126 is implemented by a desktop computer, the environment 102 may provide a two-dimensional (2D) view of the virtual environment 104. In contrast, if the client device 128 is implemented by a virtual reality helmet, the environment 102 may provide a 3D view of the virtual environment 104 so that the user of the client device 128 is immersed in the virtual environment 104 to interact therein.

[0049] Figure 2 2 shows an example of a virtual object 202 that has been bound to an enterprise dataset according to some embodiments. As described above, the virtual object 202 may correspond to Figure 1The funnel object 112 uploaded as a component 118 in the component exchanger 124 of the virtual object 202 may include a 3D funnel object, wherein the funnel is formed by a plurality of individual data points. The individual data points from the data set may be used to define the position, color, trajectory, motion, speed, size, shape, and / or any other visual characteristics of each individual data point.

[0050] The underlying data set may be derived from a database table, a multidimensional data cube, and / or any other data structure. The data set may include data points represented by rows in a database table or points in a multidimensional data cube. The various columns in the database or dimensions in the data cube may be used to define the characteristics of each sphere in the virtual object 202. For example, the first dimension or first column in the database may be used to determine the size or circumference of each sphere. The second dimension or column value may be used to define the color or texture applied to each sphere in the virtual environment. The third dimension or column value may be used to define the position of each sphere in the funnel. The fourth dimension or column value may define the speed at which the sphere rotates around the central axis of the funnel. The fifth dimension or column value may define the trajectory or motion path as it passes through the various levels of the funnel, and so on.

[0051] While the application is running, data can remain bound to each element of the virtual object 202. For example, while wearing a virtual reality headset, a user can approach a virtual object 202 in the virtual environment 200. Using a virtual selection handheld device, the user can reach out and "grab" one of the spheres 204 that rotate around the funnel axis. The user can call up a user interface that displays additional information about the underlying data represented by the sphere 204. The user can change the values ​​in the user interface to change the underlying values ​​in the corresponding database. The user can also "place" the sphere 204 back into position in the virtual object 202. If the new position of the sphere 204 is different from the old position, the dimensions used to determine the position of the sphere 204 in the virtual object 202 can be updated in the underlying database.

[0052] Figure 3 A user interface 300 is shown for importing components from a component exchanger into an application. The user interface 300 may include a list 302 of available components that may be imported into a project. By selecting one of the components in the list 302, the component exchanger may display additional information for downloading, instantiating, and / or using the component. In this example, the user has selected the funnel object component described previously. When this selection is made from the list 302, the right hand side of the interface 300 may display additional information for using the funnel object component.

[0053] The additional information may include a description 306 of the component. The description 306 may describe the operation of the component, how it may be used, how it may interact with other components, typical usage scenarios, tips and suggestions for using the component, and / or other information that may be useful to new and / or experienced users of the component. The description 306 may be accompanied by sample code 308 that illustrates how the component may be integrated into the code of an application. The sample code 308 may be provided in a variety of programming languages, and the sample code 308 may be copied and pasted from the sample code 308 into the code of an application.

[0054] In addition, some embodiments may provide an install button 304. The install button may execute a process for installing the component in the user's application. Selecting the install button 304 may launch additional windows, wizards, and / or interfaces that introduce the user to the process of installing the component in his or her application. For example, the install button 304 may trigger the generation of one or more web forms that allow the user to select a data set that can be bound to the parameterized interface of the component. For example, the install button 304 may generate an interface that allows the user to bind the sphere in the funnel object to the various columns in the database. The install button 304 may also automatically download any code necessary to operate the component. The code may include a third-party database, a graphics library, a function library, and / or other shared code required to perform the operations provided by the component.

[0055] Figure 4 A block diagram of a process for downloading and instantiating components in an application according to some embodiments is shown. Figure 3 , a user may select a funnel object 402 for use as part of an application under development. The code for the funnel object 402 may be downloaded to an operating environment on a client device that is configured to utilize components of this type. For example, the funnel object 402 may be downloaded to an EDIE environment 102 running on any of the client devices described above. The EDIE environment 102 may include a virtual environment 104 that may provide an environment in which three-dimensional (3D) components may be instantiated for both two-dimensional (2D) and 3D viewing.

[0056] In addition to downloading the code for the funnel object 402, the component exchanger 124 may also provide any additional code required to run the component in the EDIE environment 102. The additional code may be stored in a code library 404, which is stored and made available by the component exchanger 124. The additional code may also include a library 406 from a third party that is not hosted and / or provided by the component exchanger 124. For example, the component exchanger 124 may cause a library of graphics operation code to be downloaded from a third party website as part of the installation process of the funnel object 402. All of these codes may be downloaded to the EDIE environment 102 so that the funnel object 402 may operate in an independent manner for all of its desired functions.

[0057] When the funnel object 402 is downloaded to the EDIE environment 102 and instantiated in the virtual environment 104, the EDIE environment 102 may cause a binding to occur between the parameterized inputs of the funnel object 104 and one or more data sources. For example, the EDIE environment 102 may present a user interface to the user, allowing the user to select from one of a plurality of available data sources. After selecting a data source, a request may be made to the user to bind various fields in the data source to various parameterized inputs. For example, the user may select a particular column in a database so that the value in the column determines the color corresponding to the sphere in the funnel object in the virtual environment 104. The binding between the parameterized input and the various data sources may be stored as part of the application so that whenever the user launches the application, the funnel object 402 retrieves data from the underlying data source to generate the display of the funnel object in the virtual environment 104. The binding may be updated and / or changed when the application is launched, or dynamically updated and / or changed when the application is running to visualize different data sets in the virtual environment 104.

[0058] Figure 4 The example of shows an enterprise data repository 108 that can be used to bind enterprise data to parameterized inputs of various virtualized objects. In other examples, different data sources can be bound to different aspects of a visualization object. For example, some embodiments can allow columns or fields from multiple databases and multiple locations to be bound to parameterized inputs of the same virtualized object. Some embodiments can use online data sources that provide real-time data to visualization components. Any combination of data sources can be provided to parameterized inputs of a virtualized object based on the needs of individual application designers.

[0059] Figure 3 to Figure 4 The use of the funnel object 402 in is provided as an example only and is not meant to be limiting. It will be understood that any visualization object designed to operate in the EDIE environment 102 may be implemented in the EDIE environment 102 using a similar process.

[0060] Figure 5 A virtual dashboard created in a virtual environment 104 according to some embodiments is shown. Although the visualization components described herein can be used in any type of application, they may be particularly suitable for designing virtual dashboards. As used herein, the term dashboard may include a visualization that aggregates, analyzes and / or displays information from one or more data sources in a user interface. A dashboard may be provided on the homepage of an organization's website. A dashboard may also be displayed as an entry screen in an application or mobile application. A dashboard typically displays a "widget" or other graphical representation of data in a two-dimensional environment. Users may often be able to select individual widgets on a dashboard to display additional information. For example, a user may click on a widget that displays summary sales information for the current month. A widget may be configured to provide additional information related to the summary sales information displayed in the dashboard. Some dashboards may include multiple widgets that may be configured to display the latest information from the underlying data source when the dashboard is accessed by various user client devices.

[0061] The embodiments described herein implement dashboards in a 3D virtual environment 104 using the above-described visualization components. For example, instead of displaying 2D widgets on a computer screen, a user may use a client device that is enabled to interact with a 3D virtual environment to provide a 3D visualization experience to view and manipulate dashboard data. Some embodiments may use virtual reality helmets or augmented reality devices that allow users to enter the virtual environment 104 and perform real-time interactions with visualizations that display dashboard data. In the virtual environment 104, users may walk around in visualizations, "grab" and manipulate parts of visualizations, interact with other users, and have their manipulations of visualizations update the underlying data source. This turns the traditional 2D dashboard experience into an interactive, immersive virtual reality experience in which data may be viewed in an interactive, real-time manner.

[0062] The virtual environment may include multiple visual objects. Instead of multiple 2D widgets, the 3D dashboard may include visual objects placed in the virtual 3D environment 104. When a user enters or views the 3D virtual environment, they can move around in the virtualized objects to view them from any angle and manipulate them from any position. Other examples of users interacting with the virtual environment 104 to interact with the visual objects are described in more detail below.

[0063] As described above, a visualization object may refer to a software object represented by a body of code and / or its associated library. What is displayed in the virtual environment 104 may be referred to as a view of a visualization object. Each visualization object may be associated with a plurality of different views, and each view may provide a different look and feel for the visualization object. For example, a funnel object may include a traditional business view that displays data in a more implicit manner suitable for a business environment. A funnel object may also include an additional view that displays data in a less formal manner that includes brighter and more vivid colors, more interesting shapes, and / or other variations of the manner in which the funnel object is viewed. For simplicity, the remainder of this disclosure will not distinguish between the underlying visualization object represented by the executable code and the view of the visualization object displayed in the virtual environment 104. Instead, a visualization object (e.g., a funnel object) may be collectively referred to as a visualization object to include both the code and display in the virtual environment 104.

[0064] In this example, the virtual environment 104 may include a dashboard with three visualization objects. The funnel object 508 may be linked to the enterprise data store 502 described above. The graphic object 510 may be linked to a multidimensional data cube 504, wherein each dimension in the data cube is bound to each element of the graphic object 510. The personnel object 512 may include a summary display of an individual in a three-dimensional grid in the virtual environment 104 (e.g., similar to a virtual business card display). The personnel object may be bound to a web service 506 so that the personnel object can download information from the web service 504 to fill each item in the summary display of the individual. These visualization objects may be presented in various locations in the virtual environment 104 (e.g., in a semicircle). When a user enters the virtual environment 104, they may view the visualization objects 508, 510, 512 as objects in the area in front of them. In order to begin using the virtual dashboard, the user may approach the visualization objects 508, 510, 512, view the visualization objects, and / or manipulate the elements of the visualization objects as desired.

[0065] Figure 6It is shown how various types of client device systems and input devices can all interface with the same virtual environment in the EDIE environment 102 at the same time according to some embodiments. The EDIE environment 102 may include a 3D data exploration platform that serves many different purposes and provides many different benefits. For example, the EDIE environment 102 can serve as an integrated link between important converged new technologies that can make the exploration of enterprise data more immersive and smooth. These technologies may include conversational UIs (e.g., voice-based or chat-based) 610, virtual reality (VR) 608, augmented reality (AR) 604, and many other interactive technologies. Users can also access these visualization technologies using traditional 2D computing tools (such as applications on mobile devices 606, web browsers 612 on display screens, etc.). The EDIE environment 102 can provide visualizations that move seamlessly through transitions between devices. For example, a user can view a 3D visualization in a 2D environment such as a screen of a mobile device 606. The user can then put on a pair of virtual reality goggles and transition into a virtual reality space that includes visualization objects that were once displayed on the screen of the mobile device 606 in 2D. In the virtual environment, the user may interact with the visualized objects using hand controllers, which are typically included in VR systems as described below.

[0066] The EDIE environment 102 may include a hardware detection process 620 that detects the type of user input device used by the client device. For example, if the client device uses a traditional 2D display screen with a web browser 612, the hardware detection process 620 may detect that the 2D environment is interfacing with the virtual environment 104. The EDIE environment 102 may include hardware detection processes 620 for Figure 6 Interface code for each of these different types of input devices is shown. The hardware detection process 620 can identify the various input options available and load the interface code into the EDIE environment 102 accordingly. The interface code can then generate a view of the virtual environment 104 that is compatible with the corresponding input device on the client device.

[0067] As described in more detail below, the EDIE environment 102 can operate in a network mode so that the virtual environment 104 can be accessed by multiple different client devices simultaneously. This allows users to interact with each other in the virtual environment 104 while viewing visualization objects tied to underlying enterprise data. The hardware detection module 620 allows users to interact together in the virtual environment 104 based on their own viewing experience of their client devices. For example, a user entering the virtual environment 104 using a virtual reality device 608 can be viewed as an avatar displayed in the virtual environment 104 for another user viewing the virtual environment 104 from a 2D screen interface of a web browser 612.

[0068] Figure 7 A view of a virtual environment 104 through a 2D screen interface according to some embodiments is shown. A user 702 may access the EDIE environment 102 through a desktop computer, laptop computer, or other computing device equipped with a 2D screen interface. When viewing the virtual environment 104 on the 2D screen 704, a virtual camera may be placed in the virtual environment 104 to present a view of the virtual environment 104 from the perspective of the virtual camera. The rendered image may be displayed on the screen 704 of the user 702. When various objects in the virtual environment 104 are displayed on the screen 704, the user may interact with the various objects in the virtual environment 104 using a mouse or a touch screen.

[0069] The virtual environment may include multiple visualization objects arranged in the dashboard display as described above. These visualization objects may include funnel objects 508, graphic objects 510 and / or any other visualization objects. In addition, the image captured by the virtual camera and displayed on the screen 704 may include the position of other users in the virtual environment. Each user's view may be captured by a corresponding virtual camera in the virtual environment 104. Instead of displaying the virtual camera, the position of these cameras may include the display of an avatar or other representations of other users. This example includes a visualization of user 708 in the virtual environment 104. User 702 can interact and communicate with user 708 through the virtual environment 104 as if they were all in the virtual environment 104 together. When user 708 turns its virtual camera to the virtual camera of user 702, they can see the corresponding visualization or avatar representing user 702 in the virtual environment 104.

[0070] As described above, the EDIE environment 102 can automatically determine that the screen 704 should display a 2D view of the virtual environment 104. The user 708 may be wearing a pair of virtual reality goggles, and the EDIE environment 102 operating on the client device can provide a 3D view of the virtual dashboard in the virtual environment 104. Therefore, the users can all interact together using various devices in the same virtual environment. The type of device used by some users may not be obvious to other users in the virtual environment 104. For example, the user 708 may look at a virtual camera used to display for the user 702 on the screen 704, and instead of seeing the virtual camera, they may see an avatar in the virtual environment 104. The user 708 may not be able to distinguish whether the user 702 is using a virtual reality helmet or a 2D display screen 704.

[0071] Figure 8A view of a virtual environment 104 is shown with multiple users interacting with a virtual dashboard in accordance with some embodiments. In this example, users 804, 806 are represented by avatars having a human appearance. Users can select an avatar that looks like themselves or any other virtual character. These avatars can move around in the virtual environment 104 as the user moves around in the real world environment using a virtual reality helmet. As the user moves his head in the real world environment, the virtual camera can move, rotate, pan, tilt, etc. This allows the user to "walk around" the virtual dashboard in the virtual environment 104 to see various visualizations from different angles.

[0072] The movements of various users may be represented in the virtual environment 104 so that they are visible to other users. For example, compatible virtual reality devices include virtual reality helmets and selection devices that can be held in the hands of users. When the selection device or virtual reality helmet is moved in the real world environment, the movement may be used to drive the movement of the avatar in the virtual environment 104. For example, when the user 408 raises his hands in the real world environment, the avatar's hands may also be raised in the virtual environment 104 so that they are visible to the user 806. This allows the user to gesture and point to visualized objects in the virtual dashboard and have these actions visible to other users.

[0073] Figure 8 Also shown is a virtual camera 802 that can be used to capture a view of the virtual environment 104 for displaying an image on the 2D screen. As described above, the depiction of the virtual camera 802 can be replaced with an avatar of the user 702 viewing the screen 704. Alternatively, the virtual camera 802 may have no visual equivalent in the virtual environment 104, or may be replaced by any other visual indicator that is a 3D object that can be presented for other users to see.

[0074] Users using both the 2D and 3D displays described above can interact with elements of the visualization objects in the virtual dashboard. For example, user 804 can approach funnel object 508 and select one of the spheres rotating around funnel object 508. From the 2D display, the sphere can be selected using a finger tap or a mouse click. From the 3D display, the sphere can be selected by using one of the handheld virtual reality devices held by user 804 to reach out and "grab" the sphere when one of the spheres rotates around funnel object 508. The sphere can be bound to a data object in the data store as described above. For example, a sphere can represent a row in a database, and the position, color, speed, trajectory, etc. of the sphere in the funnel object 508 can be determined by different values ​​in each column of the row of the database. When user 804 holds the sphere, additional information about the sphere can be displayed in a user interface or a head-up display (HUD). For example, each column value in the row of the database can be displayed for user 804. The sphere may be passed back and forth between user 804 and user 806 so that both users 804 , 806 may view details of the underlying data object.

[0075] When the user 804 has finished checking the sphere, the user 804 can place the sphere back into the funnel object 508. In some embodiments, the user 804 can "place" the sphere back into the funnel object 508, and the sphere can automatically return to its previous position / rotation in the funnel object 508. In other embodiments, the user 804 can place the sphere in the funnel object at a position different from the position where the sphere was initially retrieved. As described above, the position of the sphere in the funnel object 508 can be determined by the value in the underlying data set bound to the funnel object 508. When the position of the sphere in the funnel object 508 is changed, the value corresponding to the position can also be changed. In some embodiments, the value corresponding to the new position in the funnel object 508 can be written back to the underlying data set. This allows the user to manipulate the elements of the virtualized objects in the virtual environment 104 and change the values ​​in the data set to which the visualization object is bound.

[0076] In some embodiments, the view of the visualization object provided to each of the users 804, 806 may be the same. For example, as described above, when the user 804 pulls the sphere out of the funnel object 508, the user 806 will see the sphere leave the funnel object 508. Similarly, when the user 804 places the sphere back into the funnel object 508, the user 806 can see the new position of the sphere in the funnel object 508. Thus, changes to the visualization object and its underlying data set may be common to each user using the virtual dashboard in the virtual environment 104.

[0077] In other embodiments, each of the users 804, 806 may be provided with its own view of the visualization object. In these embodiments, the state of each visualization object may be uniquely saved for each user 804, 806. For example, when the user 804 pulls the sphere out of the funnel object 508 as described above, the user 806 will continue to see the sphere rotating in the funnel object 508. Therefore, the changes made by one user 804 to the elements of the visualization object will only be seen by the particular user 804. Other users 806 will continue to see the visualization object in an unchanged state. The embodiment may process the changes made by one user to the underlying data set according to the stored preferences. For example, some embodiments may propagate the changes to the underlying data set, while other embodiments may provide the data set in a "read-only" configuration so that the changes to the visualization object are not reflected in the underlying data set. Some embodiments may present the changes to all users present in the virtual environment 104 for approval before writing the changes to the underlying data set.

[0078] As described above, each visualization object may have one or more views or view models associated with the visualization object. These view models may control the look and feel of the visualization object when the visualization object is presented in the virtual environment 104. The EDIE environment 102 allows each user to select a separate view model for the same visualization object. For example, user 804 may choose to view the funnel object 508 using a business view model that uses muted colors and smaller spheres. User 806 may choose to view the funnel object 508 using an informal view model that uses brighter colors and larger spheres. This allows each individual user to customize the look and feel of the visualization objects in the virtual dashboard without changing the view model for other users.

[0079] Fig. 9 A view of a real world environment 908 that can be used to interact with a virtual environment 104 is shown in accordance with some embodiments. A user 902 may use a virtual reality device including a virtual reality helmet 904 and / or one or more handheld selection devices 906. As the user 902 moves around in the real world environment 908, a virtual camera in the virtual environment 104 may move around in the virtual environment 104 in a corresponding manner. The virtual camera may capture a view of the virtual environment 104 displayed to the user 902 through the virtual reality helmet 904. As the user 902 moves his or her hand holding the selection device 906, the hand of the corresponding avatar in the virtual environment 104 may also move and select objects in the virtual environment 104, as described above.

[0080] Fig.10A flowchart 1000 of a method for displaying data in a multidimensional dashboard according to some embodiments is shown. The method may include accessing multiple 3D visualization components (1002). Each 3D visualization component may include a parameterized input for receiving a data set. The 3D visualization components may be downloaded or otherwise received from a component exchanger that allows developers to provide 3D visualization components to be used in multiple applications being developed. The 3D visualization components may be downloaded to a client device, or may be downloaded to an operating environment such as the EDIE environment described above. For example, the 3D visualization components may include funnel objects, graphic objects, and / or other visualization components described herein.

[0081] The method may also include receiving one or more data sets (1004). The data sets may represent enterprise data received from enterprise applications and / or databases. For example, the data sets may include a customer relationship management (CRM) database, a human capital management (HCM) database, a financial software database, and / or any other type of enterprise data available to the operating environment.

[0082] The method may further include binding parameterized inputs of multiple 3D visualization components to one or more data sets (1006). For example, for each 3D visualization component, the corresponding parameterized input may be bound to one of the one or more data sets. The parameterized inputs of the 3D visualization components may make the 3D visualization components generic so that their appearance and operation may be customized by virtue of the bound data sets. When the components are placed in an application, the data sets may be bound to the corresponding 3D visualization components at design time. When the 3D visualization components are instantiated at runtime, the data sets may be bound alternatively or additionally. During runtime, the system may receive input (e.g., from a user) to change the data set binding to a different data set. Some 3D visualization components may be bound to multiple different data sets, each of which may define a visual or operational aspect of the 3D visualization component.

[0083] The method may also include presenting multiple 3D virtual objects (1008) in a virtual environment based on multiple 3D visualization components and one or more data sets. Each 3D visualization component may include a view model, an animation model, a wireframe skeleton and / or other graphical structures that can be used to generate 3D virtual objects in a virtual environment. A virtual environment can be implemented in an EDIE operating environment to create a 3D virtual scene, which includes objects other than 3D virtual objects from 3D visualization components. For example, the virtual environment can be configured as a carousel of 3D virtual objects. In another example, the virtual environment can be configured as an enterprise dashboard that displays 3D objects as virtual "widgets" that can be viewed by users in the virtual environment. Each 3D virtual object in the dashboard can be bound to different enterprise data sets from different databases and / or different applications. For example, the dashboard can provide summary data of finance, employees, operations, customers, etc. in a unified display all in a single virtual environment.

[0084] The method may also include receiving connections to the virtual environment from multiple client devices (1010). Some embodiments may include a client device having a hardware detection process installed thereon, the hardware detection process being configured to detect the type of display and / or input device used with the client device. The hardware detection process may automatically configure the interaction with the virtual environment to match the type of client device. For example, if the client device includes a virtual reality helmet, the hardware detection process may be configured to provide an immersive view and / or interaction with the virtual environment so that the user feels as if they are working in the virtual environment rather than the real world environment. If the client device includes a 2D screen (e.g., a monitor, a laptop, etc.), the hardware detection process may be configured to place a virtual camera in the virtual environment to capture a 2D image of the virtual environment at a location to be displayed on the 2D screen.

[0085] The method may further include providing multiple views of multiple 3D virtual objects in a virtual environment to multiple client devices (1012). A separate view of the virtual environment may be provided to each client device. In addition, each client device may be configured to allow a user to interact with the 3D virtual objects in the virtual environment to change its display. For example, a user may remove elements corresponding to individual elements in a corresponding data set from a virtual object. A user may see a display including summary information related to data points in the data element, and they may be allowed to change certain data points. These changes may be updated in the underlying data source to which the 3D virtual object is bound. In some embodiments, the changes may be immediately viewed by other users in the virtual environment. Alternatively, some changes may only be visible to the user who made the change, so that a personalized view of the virtual environment is provided to each user.

[0086] It should be understood that Fig.10 The specific steps shown provide specific methods for displaying data in a multidimensional dashboard according to various embodiments. According to alternative embodiments, other step sequences may also be performed. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. In addition, Fig.10 The various steps shown may include multiple sub-steps, which may be performed in various orders suitable for the various steps. In addition, other steps may be added or removed depending on the specific application. Those of ordinary skill in the art will recognize many variations, modifications and substitutions.

[0087] Time series alternative visualization component

[0088] Many different visualization components can be used with the virtual environment described above. Multiple visualization components can be used in the same application to create an interactive environment that presents data in a variety of different ways. For example, a single virtual environment may include visualization components corresponding to a funnel object, a bar graph object, a representation of a beaker filled with a liquid-like volume, a profile layout of a person profile, and / or other visualization components that can be presented in 3D space for viewing and manipulation by users on multiple client devices. Depending on the type of data, some visualization components can be adjusted to highlight aspects of that data more than others. The designer of the virtual environment can determine which visualization components best highlight aspects of the data for the target audience.

[0089] In the embodiments described herein, a specific type of visualization component is described that highlights data that includes a time component. For example, some data sets may include a time series sequence of data points. Multiple values ​​may be recorded and stored as data points in a data table or other data structure. Each of these values ​​may be sampled or recorded at regular time intervals. For example, it may be assumed that five consecutive data values ​​have been recorded and there is the same time delay between each value. The sequence of data points may also include a value indicating the sampling rate or interval between each data point. In some embodiments, each data point in the time series may include both a value and a timestamp indicating when the system obtained or recorded the value. This allows irregular intervals between data points. A data set may include more than one time series sequence of data points. For example, data may be sampled for different systems at the same time, or data may be sampled repeatedly and sequentially for the same system in different sampling sessions.

[0090] Regardless of the manner in which time series of data points are stored, it may be beneficial to display these data points using a visualization component that can depict the time dependencies of the data. Loading time series of data points into a dimensionally limited visualization component may mask the time dependencies of the data points or cause multiple time series to be displayed together in a single dimension, thereby masking the differences between the different time series. Therefore, some visualization components may include a dimension dedicated to time as well as a separate dimension for each time series of data points in the data set.

[0091] Fig.11 A view of a visualization component for displaying a time series of data points in a virtual environment according to some embodiments is shown. The visualization component may be referred to as a line graph component. When the line graph component is instantiated or loaded in the virtual environment, the line graph component may receive one or more data sets to be bound to parameterized inputs as described above. Each data set may include one or more time series of data points. For example, each data set may include one or more data tables, wherein each row in the data table corresponds to a data value in the time series, and each column of each row includes an attribute value or feature of the data value. These attribute values ​​may include timestamps, user IDs, session IDs, other data values, client devices, test processes, and / or any other data values ​​that can be described as or associated with the data values ​​recorded for the row in the data table. Rows that do not include timestamps or other sampling rate identifiers may be assumed to have regular intervals between each data value, and the sampling rate or intervals between data values ​​may be specified separately. The parameterized input may receive each of these time series of data points as a vector, an array, a list, a sequence, and / or any other data structure including a variable number or a fixed number of data values.

[0092] When the line graph component receives and processes input in a time series sequence of data points through parameterized input, the line graph component can generate a line graph object 1100 in the virtual environment. The line graph object 1100 may include a multi-dimensional line graph designed to show different dimensions of time series data so that it is clearly distinguishable to the user. Different dimensions or axes in the line graph can be assigned to different dimensions for each data value in the display.

[0093] exist Fig.11In the example of , a line graph object 1100 includes a vertical axis 1102 that can be associated with a value of each data point. The line graph object 1100 may also include a first horizontal axis 1106 corresponding to time. The line graph object 1100 may also include a second horizontal axis 1104 corresponding to an index in a plurality of time series of data points. For example, if one or more data sets include a plurality of time series of data points, each time series of data points may be assigned a particular index along the horizontal axis 1104. This spreads out the individual time series so that they are visually distinguishable from one another.

[0094] As an example, the embodiment described below uses test data to illustrate how a line graph object 1100 can display timing data. The test data may include a number of faults received from a particular machine over time. Each index along the horizontal axis 1104 may correspond to a different machine under test. The time according to the horizontal axis 1106 may correspond to the time when the fault was detected. Finally, the value according to the horizontal axis 1102 may correspond to the severity of the fault, the number of faults detected, and / or any other method of characterizing a hardware / software fault. The test data is used as an example because it illustrates the advantage of separating various timing series of data points from each other. It also illustrates the advantage of using a time axis 1106 that is orthogonal to another horizontal axis 1104 that separates each timing series.

[0095] Each time series of data points can be plotted in the line graph object 1100 at different indices along the horizontal axis 1104. In some embodiments, each time series of data points can be represented by a continuous line using a different color so that they can be visually distinguished from adjacent lines. Alternatively or additionally, each time series of data points can be represented by a different line style (e.g., dashed line, dot-dash line, solid line, etc.). In some embodiments, each value in the time series of data points can be compared to a threshold value, and the line style, line color, line transparency, and / or other visual characteristics or effects of each line can be changed to indicate that the threshold value has been exceeded. For example, line 1108 includes a relatively high value that may exceed the threshold value of faulty data. Line 1108 can be highlighted or colored in different ways. Line 1108 can also include different line styles, such as thicker point values, to indicate that the threshold value has been exceeded. This can help users visually distinguish time series of data of interest based on thresholds (e.g., to identify machines that have been extensively faulty).

[0096] Fig.12 According to some embodiments, Fig.11The view of the line drawing object 1100 from different angles. As described in detail above, the virtual environment in which the line drawing object 1100 is generated can be a 3D virtual environment that allows user interaction from multiple client devices. Users can be assigned corresponding virtual cameras in the virtual environment, and the virtual camera moves according to their inputs. These inputs may depend on the type of client device each user is using. For example, a desktop computer may include a virtual camera that presents an image from the virtual environment, which is displayed on a two-dimensional display screen in the real world environment. Inputs for "moving around" in the virtual environment can be received from a mouse, keyboard, touch screen display, from voice commands, etc. These inputs can cause the virtual camera to move around in the virtual environment to provide different perspectives of the 3D view of the line drawing object 1100.

[0097] If a user accesses a virtual environment through a virtual reality device such as a virtual reality helmet, a corresponding virtual camera in the virtual environment can provide images so that the user feels as if they have entered the virtual environment. Their physical movements in the real world environment can correspond to the movements of the virtual camera in the virtual environment. This allows the user to "walk around" in different visualizations in the virtual environment. As described above, this also allows the user to interact with various data points in the visualization objects as they are presented in the virtual environment.

[0098] For example, the user can stand Fig.11 As they move from left to right in their view frustrum, they can view difference lines representing the time series of data points. Fig.11 1122. If the user wishes to view the time series of data points from a different perspective, they can generate input (e.g., motion) in the real world environment to move their virtual camera to a new position in the virtual environment. For example, the user can walk around to the right of line graph object 1100 so that they can "look down" at each line in line graph object 1100 to see the data from a different perspective. The perspective in Fig.12 This new position may correspond to Fig.11 and Fig.12 When the user moves to a new location 1120, it may appear as if they are walking through a solid object in the virtual environment. In this way, they may be able to walk around or in the line map object 1100 to view the data from any desired angle or position.

[0099] It should be noted that from Fig.12 The data from position 1120 is different from the data from Fig.11For example, when the line representing the time series of data points is at Fig.12 When compared to the dimensions in Fig.11 In addition to moving around in the virtual environment, the user can also control the scale at which they view the virtual environment. When using a virtual reality device such as a virtual reality helmet, this can correspond to scaling the user's virtual "size". Making the user larger / taller allows them to Fig.11 The line drawing object 1100 is viewed from the front / overhead angle shown. In contrast, making the user smaller / shorter allows them to view the line drawing object 1100 from the front / overhead angle shown. Fig.12 A more ground level view of the line drawing object 1100 is shown. When using a 2D interface, the user can zoom the view by moving the position of the virtual camera and zooming in / out.

[0100] Fig.13 A line graph object is shown being presented by a virtual environment according to some embodiments. In order to present a view of the line graph object 1100, a 3D object can first be instantiated and placed in the virtual environment. The line graph object 1100 may include multiple lines, each line corresponding to a respective time series of data points in one or more data sets bound to the parameterized input of the line graph component. In order to generate the line graph object, the virtual environment can establish a coordinate system for the line graph object, convert the coordinate system to the coordinate system of the virtual environment, plot the respective data points in each time series, and connect those data points with visible lines so that the user can see how each time series progresses along the time axis.

[0101] The coordinate system of the 3D line graph can correspond to the units of the above-mentioned different axes. The vertical axis corresponding to the value of the data point can be scaled based on the maximum value in the data point. The first and second horizontal axes corresponding to the time and index in multiple time series can be scaled based on the total time represented and / or the number of time series. In order to convert the coordinate system into the coordinate system of the virtual environment, the overall dimensions of the line graph object 1100 can be specified. Each of these dimensions can be represented by units such as inches, feet, etc. represented in the virtual environment. Then, the total length of any dimension can be divided by the number of divisions in the line graph coordinate system to convert those units into units in the virtual environment. For example, if the time axis includes 20 divisions corresponding to five minutes each, and the length of the time dimension in the virtual environment is 10 virtual feet, the coordinate conversion can be scaled by 5 minutes every 6 inches. A similar process can be performed for each dimension in the line graph object.

[0102] After establishing the scale between the coordinate systems for the line graph object 1100, the individual data points can be plotted for each time series. Fig.13The time series of data points represented by line 1302 in can be plotted as follows. Each of the individual data points can be plotted in corresponding coordinates in the virtual environment. For example, data point 1304 can be plotted based on its corresponding time and value. When each of these data points has been plotted for a particular time series, each data point can be sequentially connected in the virtual environment by line objects. This can correspond to lines, rays, polygonal strips, and / or any other 3D construction that can represent these connections in line graph object 1100. The same process can be performed for each time series in the data set provided to the visualization component. Depending on the embodiment, these can be plotted sequentially and / or simultaneously.

[0103] Fig.14 100 in the virtual environment. As described above, a user equipped with a virtual reality device and / or a corresponding virtual camera in the virtual environment can move around in the virtual environment. This may include allowing the user to move over portions of the line drawing object 1100 in the virtual environment, move within portions of the line drawing object 1100 in the virtual environment, and / or pass through portions of the line drawing object 1100 in the virtual environment.

[0104] In this example, there may be two users in the virtual environment. These users may be represented by virtual cameras in the virtual environment. Instead of displaying a representation of the virtual camera in the virtual environment, some embodiments may instead generate avatars 1402, 1404 corresponding to the respective users. Thus, as the user moves around the virtual environment by moving their virtual camera, other users in the virtual environment may see the avatars 1402, 1404 moving around the virtual environment in an animated manner. When the user of avatar 1402 views the virtual environment, they may be able to view the avatars 1402, 1404 in an animated manner. Fig.14 The perspective of his avatar 1402 is shown viewing the virtual environment including the line drawing object 1100 and another user's avatar 1404 .

[0105] The user can scale themselves in the virtual environment to a level that allows them to interact with the time series of the desired data. For example, the user in this example has scaled down their avatars 1402, 1404 so that they can interact with the time series represented by line 1302. In some embodiments, as they get closer to line 1302, individual data points can be highlighted on line 1302 to make them easier to interact with. For example, as avatar 1402 moves next to line 1302, data point 1304 can become visible as a sphere so that the user can interact with the data point 1304. When the user is not in close proximity to the data point location, these spheres can be hidden to avoid cluttering the view of the trajectory of the line representing the time series.

[0106] The user can cause his avatar 1402 to interact with the data point 1304 by "grabbing" the corresponding sphere in the virtual environment. This can be done using the hand controls of the virtual reality system as described above. The user can then cause additional information for the data point 1304 to be displayed, such as the attribute values ​​of the data point. The user can then change any of those attribute values ​​through a pop-up user interface and replace the data point 1304 in the row 1302 for the time series.

[0107] Some embodiments may also allow the user to change the position of a data point. For example, a user may cause the avatar 1402 to "grab" the sphere of data point 1304 and move its position up or down. The corresponding value and / or time of the data point 1304 may be changed accordingly. These changes may be visible to both users through their avatars 1402, 1404. These changes may also be limited to only the user who made those changes. Some embodiments may also write these changes back to the underlying data source, while other embodiments may only reflect the changes in the virtual environment without writing back to the underlying data source.

[0108] As described above, many different visualization components can be used to display a time series with a time-dependent component in the data. The examples of line graph components discussed above are provided only as examples and are not meant to be limiting. Other embodiments can use any type of visualization component that includes a dimension corresponding to time.

[0109] Fig.15 A circular bar graph component according to some embodiments is shown, which can be used to visualize the same data that is bound to a line graph component. The circular bar graph component can be used to generate a circular bar graph object 1500 in a virtual environment. Like other visualization components, the circular bar graph object 1500 can be bound to one or more data sets used to generate the object 1500.

[0110] As described above, multiple visualization components can be instantiated or generated in a virtual environment to form a virtual dashboard of information. The story data structure can be used to animate and transform data between various visualization components. In this example, a time series from a line graph object can be transformed or animated into a circular bar graph object 1500 to display the same data in different ways.

[0111] The circular bar graph object 1500 may include lines emanating from the center of the graph 1500. Each of these lines may represent a single time series of data points. The time dimension may be represented as each of these lines emanating from the center of the circle divided into regularly spaced sectors. The value of each of these data points may be represented by a vertical bar extending from the edge of the circle. Each of these bars may be spaced apart to represent a respective time series. The circular bar graph 1500 may thus represent the time and / or value dimensions while still visually grouping the respective time series together for the user.

[0112] The circular bar graph 1500 may form part of a virtual environment occupied by a user as described above. For example, a user represented by an avatar 1402, 1404 may walk along the top of the circular bar graph object 1500 in the virtual environment. They may highlight different time series and / or values ​​by stepping on or otherwise contacting corresponding portions of the circle.

[0113] Circular bar graph 1500 can be used to display any type of data, not just the time series data described above. Lines emanating from the center of the circle can represent groupings of data, and sectors separated from these lines can represent sub-groupings of data. The vertical bars extending downward can represent any value in any dimension. Therefore, the time series sequence of data points represented by circular bar graph 1500 is presented only as an example and is not meant to limit the operation of circular bar graph 1500.

[0114] Fig.16 A circular graph 1500 is shown that operates in a similar manner to line graph 1100 according to some embodiments. In this example, each circle in circular graph 1500 can correspond to a single time series of data points. The radial lines extending from the center of the circle can correspond to the divisions in the time dimension. The vertical bars extending upward correspond to the values ​​of each data point in the time series. Circular graph 1600 can be drawn in a manner similar to that of drawing line graph 1100 as described above. For example, line graph 1100 can be wrapped around itself to form circular graph 1600.

[0115] The circular graph 1600 may be advantageous for viewing data that is cyclical. For example, when data is repeated (e.g., a test is repeated, input data is repeated, etc.), the data may form a repeating pattern that may be represented by repeated traversals of the circle in the circular graph 1600. Each cycle around the graph may aggregate the new data with the data in the previous position so that the vertical bars form a histogram representation of the data in the vertical dimension.

[0116] The pie chart 1600 can be used with any type of data, and the radial lines extending from the center of the circle need not necessarily represent divisions in the time dimension. The pie chart 1600 can represent any other type of data. In addition, the pie chart 1600 can be part of a virtual dashboard that includes other visualization components, such as the circular bar chart 1500 and / or the line chart 1100. The virtual dashboard can also include visualization components such as the funnel object described above.

[0117] Fig.17 A flowchart 1700 of a method for visualizing time series data in an interactive virtual environment according to some embodiments is shown. The method may include receiving one or more data sets including a plurality of time series sequences of data points (1702). The data sets may include test data, enterprise data, data from an enterprise application or database, data received from a web service or any other data source, etc. Each time series sequence of data points may include a time dimension such that each data point is associated with a timestamp and / or is received at regular time intervals.

[0118] The method may also include plotting each of the plurality of time series of data points in the virtual environment as a 3D line graph (1704). As described above, this may include generating a coordinate system for the line graph, converting the coordinate system to dimensions found in the virtual environment, and plotting the individual data points in those coordinates. Some embodiments may connect these coordinates together using lines or other 3D constructs to represent the time series of data points as they traverse the time dimension. Some embodiments may also separate each of the time series along another dimension to make them easily distinguishable in the virtual environment.

[0119] The method may further include generating one or more views of the 3D line drawing using one or more virtual cameras configured to move around in the 3D line drawing in the virtual environment (1706). Each virtual camera may generate a view by capturing a view of the 3D line drawing in the virtual environment and presenting a 2D view of the image to be displayed on a display device. Each virtual camera may correspond to a client device operated by a user. These client devices may include devices with 2D screens (e.g., laptop computers, desktop computers, etc.) or devices with 3D display capabilities (e.g., augmented reality glasses, virtual reality helmets, etc.).

[0120] The method may also include sending one or more views of the 3D line graph to one or more client devices (1708). The views may be sent in real time so that they are displayed at an interactive frame rate on each client device. This may allow users of the client devices to interact with each other in the virtual environment. This may also allow any user to interact with individual data points in the 3D line graph. For example, a user may change the values ​​of individual data points in the 3D line graph by manipulating them in the virtual environment. These changes may result in changes to the underlying data bound to the visualization component.

[0121] It should be understood that Fig.17 The specific steps shown provide specific methods for visualizing time series data in an interactive virtual environment according to various embodiments. According to alternative embodiments, other step sequences may also be performed. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. In addition, Fig.17 The various steps shown may include multiple sub-steps, which may be performed in various orders suitable for the various steps. In addition, other steps may be added or removed depending on the specific application. Those of ordinary skill in the art will recognize many variations, modifications, and substitutions.

[0122] Each method described herein can be implemented by a computer system. Each step of these methods can be automatically performed by a computer system, and / or can be provided with input / output involving a user. For example, a user can provide input for each step in the method, and each of these inputs can be in response to a specific output of requesting such input, wherein the output is generated by a computer system. Each input can be received in response to a corresponding request output. In addition, input can be received from a user, from another computer system as a data stream, input can be retrieved from a memory location, input can be retrieved through a network, input can be requested from a web service, etc. Similarly, output can be provided to a user, to another computer system as a data stream, stored in a memory location, sent through a network, provided to a web service, etc. In short, each step of the method described herein can be performed by a computer system, and can involve any number of inputs, outputs and / or requests to or from a computer system, which can involve a user or can not involve a user. Those steps that do not involve a user can be said to be automatically performed by a computer system without human intervention. Therefore, in view of this disclosure, it will be understood that each step of each method described herein can be modified to include input and output to and from a user, or can be completed automatically by a computer system without human intervention, with any determinations made by a processor. In addition, some embodiments of each method described herein can be implemented as a set of instructions stored on a tangible, non-transitory storage medium to form a tangible software product.

[0123] Fig.18 A simplified diagram of a distributed system 1800 for implementing one of the embodiments is depicted. In the illustrated embodiment, the distributed system 1800 includes one or more client computing devices 1802, 1804, 1806, and 1808 configured to execute and operate client applications, such as web browsers, proprietary clients (e.g., Oracle Forms), etc., over one or more networks 1810. A server 1812 can be communicatively coupled to the remote client computing devices 1802, 1804, 1806, and 1808 via the network 1810.

[0124] In various embodiments, server 1812 may be adapted to run one or more services or software applications provided by one or more components of the system. In some embodiments, these services may be provided as web-based services or cloud services, or provided to users of client computing devices 1802, 1804, 1806, and / or 1808 under a software as a service (SaaS) model. Users operating client computing devices 1802, 1804, 1806, and / or 1808 may in turn utilize one or more client applications to interact with server 1812 to utilize the services provided by these components.

[0125] In the configuration depicted in the figure, software components 1818, 1820, and 1822 of system 1800 are shown as being implemented on server 1812. In other embodiments, one or more components of system 1800 and / or the services provided by these components may also be implemented by one or more of client computing devices 1802, 1804, 1806, and / or 1808. Then, the user operating the client computing device can utilize one or more client applications to use the services provided by these components. These components may be implemented with hardware, firmware, software, or a combination thereof. It should be appreciated that various system configurations are possible, which may be different from distributed system 1800. The embodiment shown in the figure is therefore an example of a distributed system for implementing an embodiment system, rather than being intended to be limiting.

[0126] Client computing devices 1802, 1804, 1806, and / or 1808 may be portable handheld devices (e.g., Cellular phone, computing tablets, personal digital assistants (PDAs), or wearable devices (e.g., Google head mounted display), running software such as Microsoft Windows and / or software for various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, etc.), and enabling Internet, email, short message service (SMS), or other communication protocols. The client computing device may be a general-purpose personal computer, including, for example, a computer running various versions of Microsoft Apple The client computing device may be a personal computer and / or laptop computer running any of a variety of commercially available Alternatively or additionally, client computing devices 1802, 1804, 1806, and 1808 may be any other electronic device capable of communicating over network(s) 1810, such as a thin client computer, an Internet-enabled gaming system (e.g., with or without a PCIe slot), or a workstation computer running a UNIX-like operating system (including, but not limited to, various GNU / Linux operating systems, such as, for example, Google Chrome OS). A gesture input device for a Microsoft Xbox gaming console) and / or a personal messaging device.

[0127] Although exemplary distributed system 1800 is shown with four client computing devices, any number of client computing devices may be supported. Other devices (such as devices with sensors, etc.) may interact with server 1812 .

[0128] The network(s) 1810 in the distributed system 1800 may be any type of network familiar to those skilled in the art that may support data communications using any of a variety of commercially available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (Systems Network Architecture), IPX (Internet Message Exchange), AppleTalk, and the like. By way of example only, the network(s) 1810 may be a local area network (LAN), such as a LAN based on Ethernet, Token Ring, and the like. The network(s) 1810 may be a wide area network and the Internet. It may include virtual networks, including but not limited to virtual private networks (VPNs), intranets, extranets, public switched telephone networks (PSTNs), infrared networks, wireless networks (e.g., in accordance with the Institute of Electrical and Electronics Engineers (IEEE) 802.11 protocol suite, and / or any other wireless protocol operating in any one of the networks); and / or any combination of these and / or other networks.

[0129] The server 1812 may be composed of one or more general-purpose computers, dedicated server computers (including, for example, PC (personal computer) servers, The server 1812 may be a server, a mid-range server, a mainframe computer, a rack-mounted server, etc.), a server farm, a server cluster, or any other suitable arrangement and / or combination. In various embodiments, the server 1812 may be adapted to run one or more services or software applications described in the foregoing disclosure. For example, the server 1812 may correspond to a server for performing the processing described above according to an embodiment of the present disclosure.

[0130] Server 1812 may run any of the operating systems discussed above, as well as any commercially available server operating system. Server 1812 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, Servers, database servers, etc. Exemplary database servers include, but are not limited to, those commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), etc.

[0131] In some implementations, server 1812 may include one or more applications to analyze and integrate data feeds and / or event updates received from users of client computing devices 1802, 1804, 1806, and 1808. By way of example, data feeds and / or event updates may include, but are not limited to, feed, Real-time updates and continuous data streams updated or received from one or more third-party information sources, which may include real-time events related to sensor data applications, financial quote machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automobile traffic monitoring, etc. Server 1812 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client computing devices 1802, 1804, 1806, and 1808.

[0132] Distributed system 1800 may also include one or more databases 1814 and 1816. Databases 1814 and 1816 may reside in various locations. As an example, one or more of databases 1814 and 1816 may reside on a non-transient storage medium local to server 1812 (and / or residing in server 1812). Alternatively, databases 1814 and 1816 may be away from server 1812 and communicate with server 1812 via a network-based connection or a dedicated connection. In one set of embodiments, databases 1814 and 1816 may reside in a storage area network (SAN) familiar to those skilled in the art. Similarly, any necessary files for executing the functions that server 1812 has may be appropriately stored locally on server 1812 and / or stored remotely. In one set of embodiments, databases 1814 and 1816 may include a relational database suitable for storing, updating and retrieving data in response to commands in SQL format, such as a database provided by Oracle.

[0133] Fig.19 1 is a simplified block diagram of one or more components of a system environment 1900 according to an embodiment of the present disclosure, through which services provided by one or more components of the embodiment system can be provided as cloud services. In the illustrated embodiment, the system environment 1900 includes one or more client computing devices 1904, 1906, and 1908 that can be used by users to interact with a cloud infrastructure system 1902 that provides cloud services. The client computing devices can be configured to operate client applications, such as web browsers, proprietary client applications (e.g., Oracle Forms), or some other application that can be used by users of the client computing devices to interact with the cloud infrastructure system 1902 to use the services provided by the cloud infrastructure system 1902.

[0134] It should be appreciated that the cloud infrastructure system 1902 depicted in the figure may have other components in addition to those depicted. In addition, the embodiment shown in the figure is only one example of a cloud infrastructure system that can be combined with embodiments of the present invention. In some other embodiments, the cloud infrastructure system 1902 may have more or fewer components than shown in the figure, may combine two or more components, or may have a different configuration or arrangement of components.

[0135] Client computing devices 1904 , 1906 , and 1908 may be similar devices to those described above with respect to 1802 , 1804 , 1806 , and 1808 .

[0136] Although the exemplary system environment 1900 is shown with three client computing devices, any number of client computing devices can be supported. Other devices such as devices with sensors can interact with the cloud infrastructure system 1902.

[0137] Network(s) 1910 can facilitate data communications and exchanges between clients 1904, 1906, and 1908 and cloud infrastructure system 1902. Each network can be any type of network familiar to those skilled in the art that can support data communications using any of a variety of commercially available protocols, including those described above for network(s) 1810.

[0138] Cloud infrastructure system 1902 may include one or more computers and / or servers, which may include those described above with respect to server 1812 .

[0139] In some embodiments, the services provided by the cloud infrastructure system may include many services available on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database processing, managed technical support services, etc. The services provided by the cloud infrastructure system can be dynamically expanded to meet the needs of users of the cloud infrastructure system. The specific instantiation of the services provided by the cloud infrastructure system is referred to as a "service instance" in this article. In general, any service available to users from a cloud service provider's system via a communication network (such as the Internet) is referred to as a "cloud service." Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's system are different from the customer's own local servers and systems. For example, a cloud service provider's system can host applications, and users can order and use applications on demand via a communication network such as the Internet.

[0140] In some examples, services in a computer network cloud infrastructure may include protected computer network access to storage devices, hosted databases, hosted web servers, software applications, or other services provided by a cloud provider to users, or as otherwise known in the art. For example, a service may include password-protected access to remote storage devices on the cloud over the Internet. As another example, a service may include a hosted relational database and scripting language middleware engine based on web services for private use by networked developers. As another example, a service may include access to an email software application hosted on a cloud provider's website.

[0141] In certain embodiments, the cloud infrastructure system 1902 may include a suite of application, middleware, and database service offerings delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such a cloud infrastructure system is the Oracle public cloud provided by the present assignee.

[0142] In various embodiments, the cloud infrastructure system 1902 may be adapted to automatically provision, manage, and track customer subscriptions to services provided by the cloud infrastructure system 1902. The cloud infrastructure system 1902 may provide cloud services via different deployment models. For example, services may be provided in accordance with a public cloud model, where the cloud infrastructure system 1902 is owned by an organization that sells cloud services (e.g., owned by Oracle), and the services are available to the general public or businesses in different industries. As another example, services may be provided in accordance with a private cloud model, where the cloud infrastructure system 1902 operates only for a single organization and may provide services to one or more entities within the organization. Cloud services may also be provided in accordance with a community cloud model, where the cloud infrastructure system 1902 and the services provided by the cloud infrastructure system 1902 are shared by several organizations in a related community. Cloud services may also be provided in accordance with a hybrid cloud model, which is a combination of two or more different models.

[0143] In some embodiments, the services provided by cloud infrastructure system 1902 may include one or more services provided under a software as a service (SaaS) category, a platform as a service (PaaS) category, an infrastructure as a service (IaaS) category, or other service categories including hybrid services. A customer may subscribe to one or more services provided by cloud infrastructure system 1902 via a subscription order. Cloud infrastructure system 1902 then performs processing to provide the services in the customer's subscription order.

[0144] In some embodiments, the services provided by the cloud infrastructure system 1902 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall into the SaaS category. For example, a SaaS platform may provide the ability to build and deliver on-demand application suites on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure used to provide SaaS services. By utilizing the services provided by the SaaS platform, customers may utilize applications executed on the cloud infrastructure system. Customers may obtain application services without the need for customers to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business flexibility to large organizations.

[0145] In some embodiments, platform services can be provided by a cloud infrastructure system via a PaaS platform. The PaaS platform can be configured to provide cloud services that fall into the PaaS category. Examples of platform services may include, but are not limited to, services that enable organizations (such as Oracle) to integrate existing applications on a shared public architecture and to fully utilize the shared services provided by the platform to build new applications. The PaaS platform can manage and control the underlying software and infrastructure for providing PaaS services. Customers can obtain PaaS services provided by a cloud infrastructure system without the need for customers to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), etc.

[0146] By utilizing the services provided by the PaaS platform, customers can adopt programming languages ​​and tools supported by the cloud infrastructure system and also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware Services), and Java cloud services. In one embodiment, the database cloud service may support a shared service deployment model that enables organizations to pool database resources and supply databases as a service to customers in the form of a database cloud. In the cloud infrastructure system, the middleware cloud service can provide customers with a platform for developing and deploying various business applications, and the Java cloud service can provide customers with a platform for deploying Java applications.

[0147] A variety of different infrastructure services can be provided by IaaS platforms in cloud infrastructure systems. Infrastructure services facilitate the management and control of underlying computing resources (such as storage, networks, and other basic computing resources) for customers to utilize services provided by SaaS platforms and PaaS platforms.

[0148] In some embodiments, the cloud infrastructure system 1902 may also include infrastructure resources 1930 for providing resources for providing various services to customers of the cloud infrastructure system. In one embodiment, the infrastructure resources 1930 may include a combination of pre-integrated and optimized hardware (such as servers, storage devices, and networking resources) to perform services provided by the PaaS platform and the SaaS platform.

[0149] In some embodiments, resources in cloud infrastructure system 1902 can be shared by multiple users and dynamically reallocated as needed. In addition, resources can be allocated to users in different time zones. For example, cloud infrastructure system 1930 can enable a first group of users in a first time zone to utilize the resources of the cloud infrastructure system for a specified number of hours, and then enable the same resources to be reallocated to another group of users in a different time zone, thereby maximizing the utilization of the resources.

[0150] In certain embodiments, a plurality of internal shared services 1932 may be provided that are shared by different components or modules of the cloud infrastructure system 1902 and services provided by the cloud infrastructure system 1902. These internal shared services may include, but are not limited to: security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, cloud-enabled services, email services, notification services, file transfer services, and the like.

[0151] In some embodiments, cloud infrastructure system 1902 can provide comprehensive management of cloud services (e.g., SaaS, PaaS, and IaaS services) in the cloud infrastructure system. In one embodiment, cloud management functions can include capabilities for provisioning, managing, and tracking customer subscriptions received by cloud infrastructure system 1902, etc.

[0152] In one embodiment, as depicted in the figure, cloud management functionality may be provided by one or more modules, such as order management module 1920, order orchestration module 1922, order provisioning module 1924, order management and monitoring module 1926, and identity management module 1928. These modules may include or be provided using one or more computers and / or servers, which may be general purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0153] In exemplary operation 1934, a customer using a client device (such as client device 1904, 1906, or 1908) may interact with cloud infrastructure system 1902 by requesting one or more services provided by cloud infrastructure system 1902 and placing an order to subscribe to one or more services provided by cloud infrastructure system 1902. In certain embodiments, the customer may access a cloud user interface (UI) (cloud UI 1912, cloud UI 1914, and / or cloud UI 1916) and place a subscription order via these UIs. Order information received by cloud infrastructure system 1902 in response to a customer placing an order may include information identifying the customer and one or more services provided by cloud infrastructure system 1902 to which the customer wants to subscribe.

[0154] After a customer places an order, order information is received via cloud UIs 1912 , 1914 , and / or 1916 .

[0155] At operation 1936, the order is stored in order database 1918. Order database 1918 may be one of several databases operated by cloud infrastructure system 1918 and in conjunction with other system elements.

[0156] At operation 1938, the order information is forwarded to order management module 1920. In some cases, order management module 1920 may be configured to perform billing and accounting functions related to the order, such as validating the order, and booking the order after validation.

[0157] At operation 1940, information about the order is transmitted to order orchestration module 1922. Order orchestration module 1922 can use the order information to orchestrate the provision of services and resources for the order placed by the customer. In some cases, order orchestration module 1922 can use the services of order provisioning module 1924 to orchestrate the provision of resources to support the subscribed services.

[0158] In certain embodiments, order orchestration module 1922 enables management of business processes associated with each order and applies business logic to determine whether an order should proceed to provisioning. At operation 1942, upon receiving an order for a new subscription, order orchestration module 1922 sends a request to order provisioning module 1924 to allocate resources and configure those resources required to fulfill the subscription order. Order provisioning module 1924 enables allocation of resources for services ordered by a customer. Order provisioning module 1924 provides an abstraction layer between the cloud services provided by cloud infrastructure system 1900 and the physical implementation layer for provisioning resources for providing the requested services. Thus, order orchestration module 1922 can be isolated from implementation details such as whether services and resources are actually provisioned on the fly or pre-provisioned and allocated / assigned only upon request.

[0159] At operation 1944 , once the services and resources are provisioned, notifications of the provided services may be sent to customers on client devices 1904 , 1906 , and / or 1908 via the order provisioning module 1924 of the cloud infrastructure system 1902 .

[0160] At operation 1946, the order management and monitoring module 1926 can manage and track the customer's subscription order. In some cases, the order management and monitoring module 1926 can be configured to collect usage statistics for the services in the subscription order, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and system downtime.

[0161] In certain embodiments, cloud infrastructure system 1900 may include identity management module 1928. Identity management module 1928 may be configured to provide identity services, such as access management and authorization services in cloud infrastructure system 1900. In some embodiments, identity management module 1928 may control information about clients that wish to utilize services provided by cloud infrastructure system 1902. Such information may include information authenticating the identities of these clients and information describing which actions these clients are authorized to perform with respect to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). Identity management module 1928 may also include management of descriptive information about each client and how and by whom such descriptive information may be accessed and modified.

[0162] Fig. 20 An exemplary computer system 2000 in which various embodiments of the present invention may be implemented is shown. System 2000 may be used to implement any of the above-described computer systems. As shown, computer system 2000 includes a processing unit 2004 that communicates with multiple peripheral subsystems via a bus subsystem 2002. These peripheral subsystems may include a processing acceleration unit 2006, an I / O subsystem 2008, a storage subsystem 2018, and a communication subsystem 2024. Storage subsystem 2018 includes a tangible computer-readable storage medium 2022 and a system memory 2010.

[0163] The bus subsystem 2002 provides a mechanism for allowing the various components and subsystems of the computer system 2000 to communicate with each other by intention. Although the bus subsystem 2002 is schematically shown as a single bus, the alternative embodiment of the bus subsystem can utilize multiple buses. The bus subsystem 2002 can be any of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, and a local bus using any various bus architectures. For example, this architecture can include an industrial standard architecture (ISA) bus, a microchannel architecture (MCA) bus, an enhanced ISA (EISA) bus, a video electronics standard association (VESA) local bus, and a peripheral component interconnect (PCI) bus, which can be implemented as a Mezzanine bus manufactured by the IEEE P1386.1 standard.

[0164] The processing unit 2004, which may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers), controls the operation of the computer system 2000. One or more processors may be included in the processing unit 2004. These processors may include single-core or multi-core processors. In certain embodiments, the processing unit 2004 may be implemented as one or more independent processing units 2032 and / or 2034, wherein a single-core or multi-core processor is included in each processing unit. In other embodiments, the processing unit 2004 may also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0165] In various embodiments, the processing unit 2004 can execute various programs in response to the program code and can maintain multiple concurrently executed programs or processes. At any given time, some or all of the program code to be executed may reside in (one or more) processors 2004 and / or in the storage subsystem 2018. Through appropriate programming, (one or more) processors 2004 can provide the various functions described above. The computer system 2000 can additionally include a processing acceleration unit 2006, which can include a digital signal processor (DSP), a special processor, and the like.

[0166] I / O subsystem 2008 may include user interface input devices and user interface output devices. User interface input devices may include keyboards, pointing devices such as a mouse or trackball, touch pads or touch screens incorporated into displays, scroll wheels, click wheels, dials, buttons, switches, keyboards, audio input devices with voice command recognition systems, microphones, and other types of input devices. User interface input devices may include, for example, motion sensing and / or gesture recognition devices, such as Microsoft Motion sensors that enable users to control apps such as Microsoft 360 game controller. The user interface input device may also include an eye gesture recognition device, such as detecting eye activity from the user (e.g., a "wink" when taking a picture and / or making a menu selection) and translating eye gestures to the input device (e.g., Google ) in Google In addition, the user interface input device may include a user interface that enables the user to communicate with the voice recognition system (e.g., Navigator) interactive voice recognition sensing device.

[0167] The user interface input device may also include, but is not limited to, a three-dimensional (3D) mouse, a joystick or pointing stick, a game panel and a drawing board, and audio / video equipment such as a speaker, a digital camera, a digital video camera, a portable media player, a webcam, an image scanner, a fingerprint scanner, a barcode reader 3D scanner, a 3D printer, a laser rangefinder, and a sight tracking device. In addition, the user interface input device may include, for example, a medical imaging input device such as a computer tomography, a magnetic resonance imaging, a positron emission tomography, a medical ultrasound device. The user interface input device may also include, for example, an audio input device such as a MIDI keyboard, a digital musical instrument, etc.

[0168] The user interface output devices may include a display subsystem, indicator lights, or non-visual displays such as audio output devices, etc. The display subsystem may be a cathode ray tube (CRT), a flat panel device such as using a liquid crystal display (LCD) or plasma display, a projection device, a touch screen, etc. In general, the use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from the computer system 2000 to a user or other computers. For example, the user interface output devices may include, but are not limited to, various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0169] Computer system 2000 may include a storage subsystem 2018 containing software elements, shown as currently located in system memory 2010. System memory 2010 may store program instructions that may be loaded and executed on processing unit 2004, as well as data generated during execution of these programs.

[0170] Depending on the configuration and type of computer system 2000, system memory 2010 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that are immediately accessible to and / or currently being operated and executed by processing unit 2004. In some implementations, system memory 2010 may include multiple different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some implementations, a basic input / output system (BIOS), such as one that contains basic routines that help transfer information between elements of computer system 2000 during startup, may typically be stored in ROM. By way of example, but not limitation, system memory 2010 also shows application programs 2012 that may include client applications, web browsers, middle-tier applications, relational database management systems (RDBMS), etc., program data 2014, and an operating system 2016. By way of example, operating system 2016 may include various versions of Microsoft Apple and / or Linux operating systems, various commercially available or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google operating system, etc.) and / or such as iOS, Phone, OS, 10OS and OS operating system's mobile operating system.

[0171] The storage subsystem 2018 may also provide a tangible computer-readable storage medium for storing basic programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provide the above-described functionality when executed by the processor may be stored in the storage subsystem 2018. These software modules or instructions may be executed by the processing unit 2004. The storage subsystem 2018 may also provide a repository for storing data used in accordance with the present invention.

[0172] The storage subsystem 2000 may also include a computer-readable storage media reader 2020 that may be further connected to a computer-readable storage medium 2022. Together with the system memory 2010 and, optionally, in conjunction therewith, the computer-readable storage medium 2022 may comprehensively represent remote, local, fixed and / or removable storage devices plus storage media for temporarily and / or more permanently containing, storing, transmitting, and retrieving computer-readable information.

[0173] The computer-readable storage medium 2022 containing the code or portions of the code may also include any suitable media known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage and / or transmission of information. This may include tangible computer-readable storage media, such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other tangible computer-readable media. This may also include non-tangible computer-readable media, such as data signals, data transmissions, or any other media that can be used to send the desired information and can be accessed by the computing system 2000.

[0174] As examples, computer-readable storage media 2022 may include a hard drive that reads from or writes to non-removable nonvolatile magnetic media, a magnetic disk drive that reads from or writes to removable nonvolatile magnetic disks, and a magnetic disk drive that reads from or writes to removable nonvolatile optical disks (such as CD ROMs, DVDs, and optical drives). Computer readable storage media 2022 may include, but are not limited to, The computer readable storage medium 2022 may include a solid-state drive (SSD) based on non-volatile memory (such as a flash memory-based SSD, an enterprise flash drive, a solid-state ROM, etc.), a volatile memory-based SSD (such as a solid-state RAM, a dynamic RAM, a static RAM), a DRAM-based SSD, a magnetoresistive RAM (MRAM) SSD, and a hybrid SSD using a combination of DRAM and flash memory-based SSDs. The disk drive and its associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the computer system 2000.

[0175] The communication subsystem 2024 provides an interface to other computer systems and networks. The communication subsystem 2024 is used as an interface for receiving data from other systems and sending data from the computer system 2000 to other systems. For example, the communication subsystem 2024 can enable the computer system 2000 to be connected to one or more devices via the Internet. In some embodiments, the communication subsystem 2024 may include a radio frequency (RF) transceiver component for accessing a wireless voice and / or data network (e.g., using cellular phone technology, advanced data network technology such as 3G, 4G or EDGE (Enhanced Data Rate for Global Evolution), WiFi (IEEE 802.11 series standards), or other mobile communication technologies, or any combination thereof), a global positioning system (GPS) receiver component and / or other components. In some embodiments, as an addition or replacement of a wireless interface, the communication subsystem 2024 may provide a wired network connection (e.g., Ethernet).

[0176] In some embodiments, the communication subsystem 2024 may also receive input communications in the form of structured and / or unstructured data feeds 2026 , event streams 2028 , event updates 2030 , and the like on behalf of one or more users who may use the computer system 2000 .

[0177] As an example, the communication subsystem 2024 may be configured to receive data feeds 2026 in real time from users of a social network and / or other communication service, such as feed, Updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0178] Additionally, the communication subsystem 2024 may also be configured to receive data in the form of a continuous data stream, which may include event streams 2028 and / or event updates 2030, which may be continuous or unbounded in nature, real-time events without explicit termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial quote machines, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, and the like.

[0179] The communication subsystem 2024 may also be configured to output structured and / or unstructured data feeds 2026, event streams 2028, event updates 2030, etc. to one or more databases, which may be in computer communication with one or more streaming data source computers coupled to the computer system 2000.

[0180] Computer system 2000 may be one of various types, including a handheld portable device (e.g., Cellular phone, computing tablets, PDAs), wearable devices (e.g., Google head mounted display), PC, workstation, mainframe, kiosk, server rack, or any other data processing system.

[0181] Due to the ever-changing nature of computers and networks, the description of the computer system 2000 depicted in the figure is intended only as a specific example. Many other configurations with more or fewer components than the system depicted in the figure are possible. For example, customized hardware may also be used and / or specific elements may be implemented with hardware, firmware, software (including applets) or a combination thereof. In addition, connections to other computing devices such as network input / output devices may also be employed. Based on the disclosure and teachings provided herein, one of ordinary skill in the art will recognize other ways and / or methods of implementing various embodiments.

[0182] In the foregoing description, for the purpose of illustration, many specific details are set forth in order to provide a thorough understanding of various embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without some of these specific details. In other cases, well-known structures and devices are shown in block diagram form.

[0183] The foregoing description provides only exemplary embodiments and is not intended to limit the scope, applicability or configuration of the present disclosure. Rather, the foregoing description of the exemplary embodiments will provide an enabling description for implementing the exemplary embodiments to those skilled in the art. It should be understood that various changes may be made to the function and arrangement of the elements without departing from the spirit and scope of the present invention as set forth in the appended claims.

[0184] Specific details are given in the foregoing description to provide a thorough understanding of the embodiments. However, one of ordinary skill in the art will appreciate that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may have been shown as components in block diagram form so as not to obscure the embodiments in unnecessary detail. In other cases, well-known circuits, processes, algorithms, structures, and techniques may have been shown without unnecessary detail to avoid obscuring the embodiments.

[0185] In addition, it is noted that various embodiments may have been described as a process, which is depicted as a flow chart, a flowchart, a data flow diagram, a structure diagram, or a block diagram. Although the flow chart may have described the operations as a sequential process, many operations can be performed in parallel or simultaneously. In addition, the order of the operations can be rearranged. The process terminates when its operations are completed, but may have other steps not included in the figure. A process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination can correspond to the function returning to the calling function or the main function.

[0186] The term "computer-readable medium" includes, but is not limited to, portable or fixed storage devices, optical storage devices, wireless channels, and various other media capable of storing, containing, or carrying one or more instructions and / or data. Code segments or machine-executable instructions may represent procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or hardware circuit by passing and / or receiving information, data, independent variables, parameters, or memory contents. Information, independent variables, parameters, data, etc. may be delivered, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0187] In addition, the embodiments may be implemented by hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments that perform the necessary tasks may be stored in a machine-readable medium. One or more processors may perform the necessary tasks.

[0188] In the foregoing description, various aspects of the present invention are described with reference to their specific embodiments, but those skilled in the art will appreciate that the present invention is not limited thereto. Each feature and aspect of the foregoing invention may be used alone or in combination. In addition, without departing from the broader spirit and scope of this description, the embodiments may be used in any number of environments and applications other than those described herein. Accordingly, this description and the accompanying drawings should be considered illustrative rather than restrictive.

[0189] In addition, for the purpose of illustration, the method is described in a specific order. It should be understood that in alternative embodiments, these methods can be performed in an order different from the described order. It should also be understood that the above method can be performed by hardware components, or can be embodied in a sequence of machine executable instructions, which can be used to make a machine (such as a general or special processor or a logic circuit programmed with instructions) perform these methods. These machine executable instructions can be stored in one or more machine-readable media (such as CD-ROM or other types of optical disks, floppy disks, ROM, RAM, EPROM, EEPROM, magnetic cards or optical cards, flash memory, or other types of machine-readable media suitable for storing electronic instructions). Alternatively, these methods can be performed by a combination of hardware and software.

Claims

1. A method for visualizing time series data in an interactive virtual environment, the method comprising: receiving one or more data sets, the one or more data sets comprising a plurality of time-series sequences of data points; plotting each of the plurality of time series of data points in a three-dimensional 3D line graph in a virtual environment; generating one or more views of the 3D line drawing in the virtual environment using one or more virtual avatars configured to move around in the 3D line drawing; and sending the one or more views of the 3D line drawing to one or more client devices, wherein the one or more client devices correspond to the one or more virtual avatars, and The one or more client devices are capable of controlling the scale and position of at least one of the one or more virtual avatars in the virtual environment to allow the at least one virtual avatar to interact with the time sequence of interest.

2. The method of claim 1, wherein: Plotting each of the plurality of time series of data points in a virtual environment as a 3D line graph comprises: plotting the position of each data point in the plurality of time-series sequences of data points in a coordinate system of the virtual environment; and Each of the locations is connected with a 3D line object. 3 . The method of claim 2 , further comprising converting a coordinate system of the 3D line drawing into a coordinate system of the virtual environment.

4. The method of claim 3, wherein: The coordinate system of the 3D line graph includes: a first horizontal axis, which corresponds to time; a second horizontal axis corresponding to indices in the plurality of time series of data points; and The vertical axis corresponds to the values ​​of the data points.

5. The method of claim 1, wherein: The one or more data sets include test failure data for different systems over time.

6. The method of claim 1, further comprising: Loading visualization components with parameterized inputs; binding the one or more data sets to the parameterized input; and Each of the plurality of time-series sequences of data points is plotted in the virtual environment with the 3D line graph according to the visualization component.

7. The method of claim 1, wherein: A client device among the one or more client devices comprises a virtual reality device.

8. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving one or more data sets, the one or more data sets comprising a plurality of time-series sequences of data points; plotting each of the plurality of time-series sequences of data points in a virtual environment as a 3D line graph; generating one or more views of the 3D line drawing in the virtual environment using one or more virtual avatars configured to move around in the 3D line drawing; and sending the one or more views of the 3D line drawing to one or more client devices, wherein the one or more client devices correspond to the one or more virtual avatars, and The one or more client devices are capable of controlling the scale and position of at least one of the one or more virtual avatars in the virtual environment to allow the at least one avatar to interact with the time sequence of interest.

9. The non-transitory computer readable medium of claim 8, wherein: Plotting each of the plurality of time series of data points in a virtual environment as a 3D line graph comprises: plotting the position of each data point in the plurality of time-series sequences of data points in a coordinate system of the virtual environment; and Each of the locations is connected with a 3D line object.

10. The non-transitory computer readable medium of claim 9, wherein: The operations also include converting a coordinate system of the 3D line drawing into a coordinate system of the virtual environment.

11. The non-transitory computer readable medium of claim 10, wherein: The coordinate system of the 3D line graph includes: a first horizontal axis, which corresponds to time; a second horizontal axis corresponding to indices in the plurality of time series of data points; and The vertical axis corresponds to the values ​​of the data points.

12. The non-transitory computer readable medium of claim 8, wherein: The one or more data sets include test failure data for different systems over time.

13. The non-transitory computer readable medium of claim 8, wherein: The operations also include: Loading visualization components with parameterized inputs; binding the one or more data sets to the parameterized input; and Each of the plurality of time-series sequences of data points is plotted in the virtual environment with the 3D line graph according to the visualization component.

14. The non-transitory computer readable medium of claim 8, wherein: A client device among the one or more client devices comprises a virtual reality device.

15. A system for visualizing time series data in an interactive virtual environment, comprising: one or more processors; and One or more memory devices comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving one or more data sets, the one or more data sets comprising a plurality of time-series sequences of data points; plotting each of the plurality of time-series sequences of data points in a virtual environment as a 3D line graph; generating one or more views of the 3D line drawing in the virtual environment using one or more virtual avatars configured to move around in the 3D line drawing; and sending the one or more views of the 3D line drawing to one or more client devices, wherein the one or more client devices correspond to the one or more virtual avatars, and The one or more client devices are capable of controlling the scale and position of at least one of the one or more virtual avatars in the virtual environment to allow the at least one avatar to interact with the time sequence of interest.

16. The system of claim 15, wherein: Plotting each of the plurality of time series of data points in a virtual environment as a 3D line graph comprises: plotting the position of each data point in the plurality of time-series sequences of data points in a coordinate system of the virtual environment; and Each of the locations is connected with a 3D line object.

17. The system of claim 16, wherein: The operations also include converting a coordinate system of the 3D line drawing into a coordinate system of the virtual environment.

18. The system of claim 17, wherein: The coordinate system of the 3D line graph includes: a first horizontal axis, which corresponds to time; a second horizontal axis corresponding to indices in the plurality of time series of data points; and The vertical axis corresponds to the values ​​of the data points.

19. The system of claim 15, wherein: The one or more data sets include test failure data for different systems over time.

20. The system of claim 15, wherein: The operations also include: Loading visualization components with parameterized inputs; binding the one or more data sets to the parameterized input; and Each of the plurality of time-series sequences of data points is plotted in the virtual environment with the 3D line graph according to the visualization component.

21. The system of claim 15, wherein: A client device among the one or more client devices comprises a virtual reality device.

22. A computer program product for visualizing time series data in an interactive virtual environment, comprising instructions which, when executed by a computer, cause the computer to perform a method including that described in any one of claims 1-7.

Images