Method, medium, system, and computer program product for determining an optimal view of a visualization object

Abstract

Multiple visualization objects can be provided to represent one or more data sets in a virtual 3D space. The visualization objects can include funnels, containers, business cards, etc. The visualization objects can be arranged in a circular turntable that can rotate around a virtual camera or the user's position in a VR / AR environment. Individual data points within the visualization objects can be rotated, resized, positioned, colored, or otherwise characterized based on the properties of the corresponding data points. The individual data points can also be animated when transitioning between visualization objects in a unified view. Voice commands can be interpreted as part of an interactive environment that can provide views of the visualization objects to multiple devices simultaneously.

Description

[0001] Cross - Reference to Related Applications

[0002] This application is a non - provisional application of U.S. Provisional Application No. 62 / 748,504, titled "3D VISUALIZATION SERVICES INTEGRATING VARIOUS DATA SOURCES AND DISPLAY DEVICES", filed on October 21, 2018, and claims the benefit and priority thereof under 35 U.S.C. 119(e). For all purposes, the entire content thereof is incorporated herein by reference.

[0003] This application also relates to the following five U.S. patent applications:

[0004] U.S. Patent Application No. 16 / 658,162, titled "INTERACTIVE DATA EXPLORER AND 3 - D DASHBOARD ENVIRONMENT", filed on October 21, 2019, by Moroze et al. (Attorney Docket No. 088325 - 1142774), which is incorporated herein in its entirety.

[0005] U.S. Patent Application No. 16 / 658,165, titled "FUNNEL VISUALIZATION WITH DATA POINT ANIMATIONS AND PATHWAYS", filed on October 21, 2019, by Moroze et al. (Attorney Docket No. 088325 - 1142775), which is incorporated herein in its entirety.

[0006] U.S. Patent Application No. 16 / 658,169, titled "OPTIMIZING VIEWS USING VOICE COMMANDS", filed on October 21, 2019, by Moroze et al. (Attorney Docket No. 088325 - 1142776), which is incorporated herein in its entirety.

[0007] U.S. Patent Application No. 16 / 658,177, titled "ANIMATION BETWEEN VISUALIZATION OBJECTS IN A VIRTUAL DASHBOARD", filed on October 21, 2019, by Moroze et al. (Attorney Docket No. 088325 - 1142782), which is incorporated herein in its entirety.

[0008] U.S. Patent Application No. 16 / 658,180 (Attorney Docket No. 088325-1143002) titled "DATA VISUALIZATION OBJECTS IN A VIRTUAL ENVIRONMENT" filed by Moroze et al. on October 21, 2019, which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0009] The present disclosure relates to methods, media, systems, and computer program products for determining an optimal view of a visualization object. BACKGROUND ART

[0010] One of the most effective ways to convey a structure is through visualization. Generally, capturing and categorizing information is not sufficient to fully understand the conclusions that can be drawn from the data. Although two-dimensional data visualization dominates most of our communication technologies, many of these visualization techniques are not optimized for highlighting or emphasizing aspects of data collection. Effective data visualization involves creating information-rich visuals that can convey aspects of the data very quickly. Sometimes, simple visualizations are sufficient. However, for complex multi-dimensional data, the complexity of the dataset itself can make it difficult to visualize in a limited dimensional space. SUMMARY OF THE INVENTION

[0011] A method for determining an optimal view of a visualization object may include receiving one or more datasets; generating 3D objects corresponding to data points in the one or more datasets; arranging the 3D objects in a predetermined 3D shape; receiving a command indicating a first view of the predetermined 3D shape; determining a distance associated with the first view; and providing a display of the first view of the predetermined 3D shape. The 3D objects may be visually emphasized relative to one or more 3D constructs that represent the structure of the predetermined 3D shape at least in part based on the distance associated with the first view.

[0012] A non-transitory computer-readable medium may include instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: receiving one or more datasets; generating 3D objects corresponding to data points in the one or more datasets; arranging the 3D objects in a predetermined 3D shape; receiving a command indicating a first view of the predetermined 3D shape; determining a distance associated with the first view; and providing a display of the first view of the predetermined 3D shape. The 3D objects may be visually emphasized relative to one or more 3D constructs that represent the structure of the predetermined 3D shape at least in part based on the distance associated with the first view.

[0013] A system may include one or more processors and one or more memory devices that may include instructions that, when executed by the one or more processors, cause the one or more processors to perform operations that include: receiving one or more data sets; generating 3D objects corresponding to data points in the one or more data sets; arranging the 3D objects in a predetermined 3D shape; receiving a command indicating a first view of the predetermined 3D shape; determining a distance associated with the first view; and providing a display of the first view of the predetermined 3D shape. The 3D objects may be visually emphasized relative to one or more 3D constructs that represent the structure of the predetermined 3D shape at least in part based on the distance associated with the first view.

[0014] In any embodiment, any of the following features may be implemented in any combination and without limitation. The command indicating the first view may include a plain English voice command. The method / operation may further include converting the plain English voice command into a predetermined command among a plurality of predetermined commands. The method / operation may further include providing the predetermined command to one or more neural networks. Providing the display of the first view of the predetermined 3D shape uses the view position of a virtual camera determined by the output of one or more neural networks. One or more data sets may be selected based on the output of one or more neural networks. The 3D objects and the predetermined 3D shape may be selected based on the output of one or more neural networks. The first view position includes a view that fills approximately 90% of the view area with the predetermined 3D shape. The method / operation may further include determining that the distance associated with the first view is greater than a first threshold distance, and emphasizing one or more 3D constructs while not emphasizing the 3D objects. The method / operation may further include determining that the first view has been scaled to a second view position that is lower than a second threshold but higher than the first threshold, and gradually de-emphasizing the predetermined 3D shape while gradually emphasizing the 3D objects. The method / operation may further include determining that the distance associated with the first view is less than a second threshold distance, and emphasizing the 3D objects while not emphasizing one or more 3D constructs. The 3D objects may be visually emphasized relative to one or more 3D constructs by changing the transparency of the 3D objects relative to the transparency of the one or more 3D constructs. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] A further understanding of the nature and advantages of the present invention may be realized by reference to the remaining portions of the specification and the drawings, wherein like reference numerals are used throughout the several drawings to refer to like components. In some cases, a sub-label is associated with a reference numeral to denote one of a plurality of like components. When referring to a reference numeral without specifying an existing sub-label, it is intended to refer to all such plurality of like components.

[0016] Figure 1Illustrates a system for creating, providing, and using interactive data displays according to some embodiments.

[0017] Figure 2 Illustrates an example of a virtual object that has been bound to an enterprise data set according to some embodiments.

[0018] Figure 3 Illustrates a user interface for importing components from a component exchanger into an application.

[0019] Figure 4 Illustrates a block diagram of a process for downloading and instantiating components in an application according to some embodiments.

[0020] Figure 5 Illustrates a virtual dashboard created in a virtual environment according to some embodiments.

[0021] Figure 6 Illustrates how various types of client device systems and input devices can all interface with the same virtual environment in an EDIE environment simultaneously according to some embodiments.

[0022] Figure 7 Illustrates a view of a virtual environment through a 2D screen interface according to some embodiments.

[0023] Figure 8 Illustrates a view of a virtual environment with multiple users interacting with a virtual dashboard according to some embodiments.

[0024] Figure 9 Illustrates a view of a real-world environment that can be used to interact with a virtual environment according to some embodiments.

[0025] Figure 10 Illustrates a flowchart of a method for displaying data in a multi-dimensional dashboard according to some embodiments.

[0026] Figure 11 Illustrates an example of a funnel-shaped visualization object according to some embodiments.

[0027] Figure 12A Illustrates a view of a 3D object representing a first distance from a data point according to some embodiments.

[0028] Figure 12B Illustrates a diagram of a system that uses a model-based system to interpret views and / or data display commands according to some embodiments.

[0029] Figure 13 Illustrates a closer view of a 3D shape according to some embodiments.

[0030] Figure 14 Illustrates an optimized view of a data set according to some embodiments.

[0031] Figure 15 Illustrates a fully zoomed-in view of a rendered visualization component according to some embodiments.

[0032] Figure 16 Illustrates a flowchart of a method for determining an optimal view of a visualization component as described above.

[0033] Figure 17 Illustrates a flowchart of a method for generating visualization objects from a set of data points according to some embodiments.

[0034] Figure 18 Depicts a simplified diagram of a distributed system for implementing some embodiments.

[0035] Figure 19 Is a simplified block diagram of one or more components of a system environment that can provision services provided by one or more components of an embodiment system as cloud services according to some embodiments.

[0036] Figure 20 Illustrates an exemplary computer system in which various embodiments can be implemented. Detailed Description

[0037] Described herein are embodiments of an immersive user interface (UI) experience. The immersive UI experience can be packaged as a separate component that can be inserted into a component exchanger or exchange house and then used by application developers or service developers to add interactive UI components to their applications or services. These UI components can be used to differentiate applications in a crowded market and provide a user-friendly interactive immersive experience, thereby providing detailed and intuitive data visualization for many different data types on many different user devices.

[0038] Figure 1Illustrated is a system for creating, providing, and using interactive data displays according to some embodiments. The system can include a component exchanger 124. The component exchanger 124 can be part of an application development kit that allows users to develop applications and / or services using a custom development environment and a library of predefined tools and / or interfaces. The application development kit can be a cloud-hosted development environment that can be accessed as a software as a service (SaaS) model for many different client devices. The development environment can be configured to develop and / or deploy many different types of applications, including cloud-based services or microservices, web applications, mobile device applications, enterprise applications, and so on. Applications designed in the application development kit can be used with other SaaS applications through a representational state transfer (REST) interface.

[0039] The development environment in the application development kit can include a plurality of user-friendly tools that developers can use to simplify the development process and make the entire application development cycle more efficient. These tools can include WYSIWYG (what you see is what you get) tools and other graphical interfaces that allow developers to drag and drop various pre-packaged components into the application development space. Various components can be provided by the development environment, such as reusable business objects, processes, web controls, user interface controls, themes, templates, and other aspects of the application.

[0040] Developers can use the development environment to design and test applications, and then deploy the applications to various operating environments. When deploying an application to an operating environment, the development environment can also deploy any additional code or libraries required by the application so that it can run in the target environment. Therefore, the flexibility and reusable components provided by the development environment can be ported to any operating environment without losing any functionality provided by the development environment.

[0041] In this example, the development environment can include a component exchanger. For example, any commercially available development environment can be used to implement the development environment, such as from Visual The development environment can include various toolkits, such as a collection of proprietary and / or open-source JavaScript libraries that can be used to build applications. For example, the development environment can include the JavaScript Extension Toolkit from The components in these toolkits can include a rich user interface (UI) collection of components that includes a data model, a view model, and ports for binding data to the components. In some embodiments, the development environment can provide a unified component plug-in mechanism based on standard components such as the W3C Web Components standard, while still allowing custom components to work with the standard components. The components in these toolkits can include a rich user interface (UI) collection of components that includes a data model, a view model, and ports for binding data to the components. In some embodiments, the development environment can provide a unified component plug-in mechanism based on standard components such as the W3C Web Components standard, while still allowing custom components to work with the standard components.

[0042] The component exchanger 124 may allow component developers to upload components to the component exchanger 124 for use by application developers. This allows component developers to provide a flexible component architecture that can be maintained and upgraded throughout the component life cycle. For example, these components may include standard web components, such as checkbox components (116) or slider components (114) that can be used in various user interfaces. These components 114, 116 can be uploaded to the component exchanger 124 and then updated and maintained by the component development team throughout their life cycle. They can then be provided to application developers as components 121, 122 in the component exchanger 124.

[0043] In addition to standard web components, the embodiments described herein also allow users to develop special visualization objects that can be used in an enterprise data interactive explorer (EDIE) environment. These visualization objects provide multi-dimensional views of enterprise data sets that can be viewed simultaneously by multiple client devices over separate network connections. These visualization objects can be imported into the component exchanger 124 in the same manner as standard web components are imported into the component exchanger 124. They can then be imported into the various applications being developed. In these applications, developers can bind various enterprise data sets from multiple different sources to parameterized inputs to the visualization objects. The visualization objects can then be used to generate rendered views of virtual objects that represent and visualize enterprise data in a multi-dimensional environment.

[0044] Various examples of these visualization objects can be described in more detail below in the present disclosure. Figure 1 Several examples of visualization objects that can be imported into the component exchanger 124 as regular objects are illustrated. For example, the funnel object 112 may include a rotating funnel object that illustrates the progression of data points in an enterprise data set over time. The funnel object 112 can provide various visual effects that can be used to highlight various aspects of the data set bound to the funnel object 112. For example, a user can search the data set along various dimensions, change individual data points, view the data point pathways over time, view an intuitive representation of the data when accelerating the data over time along various dimensions, etc.

[0045] Figure 1Also illustrated is a three-dimensional histogram or bar chart object 110 that can be used to represent various enterprise data sets. In the component exchanger 124, the graphical object 110 can be disconnected from any underlying data set. However, when the graphical object 110 is imported into an actual application, the graphical object can be bound to an enterprise data set, and the dimensions of the individual bars in the graph can be adjusted based on the data binding. The visualization object can be used to generate three-dimensional (3D) virtual objects in a virtual environment viewable by various client devices. For example, the graphical object 110 can be implemented as a plurality of three-dimensional cylinders, each three-dimensional cylinder representing a bar in the graph. A user can enter the virtual environment and walk around the virtual object, interact with the object, change the values in the object, manipulate parts of the object, and have the changed values be seen by other users and stored back in the underlying database storing the data set.

[0046] The above visualization objects 110, 112 are provided as examples and are not meant to be limiting. Any multi-dimensional object that can be used to visualize an enterprise data set can be used as a visualization object in the present disclosure. Other visualization objects can include charts, animations, pictures, graphics, and the like.

[0047] In the component exchanger 124, the visualization object can be represented as components 118, 120, which can be selected by a developer and imported into the application during development. For example, a developer can drag the component 118 of the funnel object 112 into the application. Then, according to an embodiment, this visualization object can be bound to a data set in a development environment or a deployment environment.

[0048] In some embodiments, an application including one or more visualization objects can be deployed to various environments. These environments can include web applications, mobile applications, client-side applications, applications running on a gaming system, applications running on a desktop or laptop computer, applications for virtual environments (such as virtual reality or augmented reality systems), and / or any other computing environment. When the application is deployed to an environment, the component exchanger 124 can deploy any necessary libraries, third-party libraries, or other software components or libraries required to run the visualization object. For example, some visualization objects operating in a 3D environment can include an OpenGL library or other 3D visualization libraries required for their operation.

[0049] An application can be deployed to an environment 102, generally referred to herein as an EDIE environment, which can include multiple standard components and one or more visualization objects as described above. Environment 102 can include a virtual environment 104 in which visualization objects can be instantiated. When visualization objects are instantiated, they can be bound to one or more underlying data sets. These data sets can be obtained from various sources, such as database tables, multidimensional data cubes, web services, server-side applications, API interfaces, etc. In Figure 1 the example of

[0050] Figure 1 , an enterprise database 108 can provide values from one or more database tables to environment 102 for use with at least one of the visualization objects. In this example, a funnel object 112 can be deployed to environment 102 in virtual environment 104. The funnel object 112 can include a collection of inputs, such as parameterized inputs that can be bound separately and / or jointly to tables or rows in an enterprise data set from enterprise database 108. Bindings 106 between the data set values and the parameterized inputs of the visualization objects can be created in a development environment, in deployment environment 102, and / or at any other time. The bindings 106 can also be adjusted or reformed dynamically with different data sets at runtime or before the application is executed.

[0051] Visualization objects generated in virtual environment 104 can be viewed and / or interacted with by multiple different client devices 126, 128, 130, 132. These client devices can include laptop computers, desktop computers, workstations, tablet computers, smart phones, smart watches, smart TVs, video game consoles, digital home assistants, smart glasses, virtual reality headsets, augmented reality headsets, and any other computing device. Each of these client devices 126, 128, 130, 132 can receive a unique view of virtual environment 104, which corresponds to a uniquely positioned virtual camera for each client device 126, 128, 130, 132. Then, a user can interact with virtual object 105 in virtual environment 104 to view, manipulate, edit, and / or update the underlying data set using virtual object 105. As described below, a unique specific view of virtual object 105 can be provided to each user, and different settings can allow the user to change the underlying data such that virtual object 105 changes in the views provided to other client devices. At each client device 126, 128, 130, 132, automatic hardware detection can be used to provide the correct view of virtual environment 104. For example, if client device 126 is implemented by a desktop computer, then environment 102 can provide a two-dimensional (2D) view of virtual environment 104. In contrast, if client device 128 is implemented by a virtual reality headset, then environment 102 can provide a 3D view of virtual environment 104 such that the user of client device 128 is immersed in virtual environment 104 to interact therein.

[0052] Figure 2 Illustrates an example of a virtual object 202 that has been bound to an enterprise data set according to some embodiments. As described above, virtual object 202 can correspond to the funnel object 112 uploaded as component 118 in the Figure 1 component exchanger 124. Virtual object 202 can include a 3D funnel object, where the funnel is formed by multiple individual data points. Individual data points from the data set can be used to define the position, color, trajectory, movement, speed, size, shape, and / or any other visual feature of each of the individual data points.

[0053] The underlying dataset can be sourced from a database table, a multidimensional data cube, and / or any other data structure. The dataset can include data points represented by rows in a database table or points in a multidimensional data cube. Each column in the database or dimension in the data cube can be used to define properties for each sphere in the virtual object 202. For example, the first dimension or first column in the database can be used to determine the size or circumference of each sphere. The second dimension or column value can be used to define the color or texture applied to each sphere in the virtual environment. The third dimension or column value can be used to define the position of each sphere in the funnel. The fourth dimension or column value can define the speed at which the sphere rotates around the central axis of the funnel. The fifth dimension or column value can define the trajectory or motion path by which it moves through the various levels of the funnel, and so on.

[0054] When the application is running, the data can remain bound to each element of the virtual object 202. For example, when wearing a virtual reality headset, the user can approach the virtual object 202 in the virtual environment 200. Using a virtual selection handheld device, the user can reach out and "grab" one of the spheres 204 that rotates around the funnel axis. The user can bring up a user interface that displays additional information about the underlying data represented by the sphere 204. The user can change the values in the user interface to change the underlying values in the corresponding database. The user can also "place" the sphere 204 back into its position in the virtual object 202. If the new position of the sphere 204 is different from the old position, then the dimension used to determine the position of the sphere 204 in the virtual object 202 can be updated in the underlying database.

[0055] Figure 3 A user interface 300 for importing components from a component exchanger into an application is illustrated. The user interface 300 can include a list 302 of available components that can be imported into a project. By selecting one of the components in the list 302, the component exchanger can display additional information for downloading, instantiating, and / or using the component. In this example, the user has selected the funnel object component described previously. When this selection is made from the list 302, additional information for using the funnel object component can be displayed on the right hand side of the interface 300.

[0056] The additional information can include a description 306 of the component. This description 306 can describe the operation of the component, how to use it, how it can interact with other components, typical usage scenarios, tips and suggestions for using the component, and / or other information that may be useful to new and / or experienced users of the component. The description 306 can be accompanied by sample code 308 that illustrates how the component can be integrated into the code of the application. The sample code 308 can be provided in various programming languages, and the sample code 308 can be copied and pasted from the sample code 308 into the code of the application.

[0057] In addition, some embodiments may provide an install button 304. The install button may perform the process of installing a component in the user's application. Selecting the install button 304 may launch additional windows, wizards, and / or interfaces that guide the user through the process of installing the component in their application. For example, the install button 304 may trigger the generation of one or more web forms that allow the user to select a data set that can be bound to a parameterized interface of the component. For example, the install button 304 may generate an interface that allows the user to bind the spheres in a funnel object to individual columns in a database. The install button 304 may also automatically download any code necessary to operate the component. This code may include third-party databases, graphics libraries, function libraries, and / or other shared code required to perform the operations provided by the component.

[0058] Figure 4 A block diagram illustrating a process for downloading and instantiating a component in an application according to some embodiments is shown. Using the user interface described above in Figure 3 the user may select a funnel object 402 to be used as part of an application under development. Code for the funnel object 402 may be downloaded to an operating environment on a client device that is configured to utilize this type of component. For example, the funnel object 402 may be downloaded to the EDIE environment 102 operating on any of the client devices described above. The EDIE environment 102 may include a virtual environment 104 that may provide an environment in which 3D components may be instantiated for both two-dimensional (2D) and three-dimensional (3D) views.

[0059] In addition to downloading the code for the funnel object 402, the component exchanger 124 may also provide any additional code required to run the component in the EDIE environment 102. This additional code may be stored in a code library 404 that is stored and made available by the component exchanger 124. The additional code may also include libraries 406 from third parties that are not hosted and / or provided by the component exchanger 124. For example, as part of the installation process for the funnel object 402, the component exchanger 124 may cause a library of graphics manipulation code to be downloaded from a third-party website. All of this code may be downloaded to the EDIE environment 102 such that the funnel object 402 may operate in a stand-alone manner with all of its required functionality.

[0060] When the funnel object 402 is downloaded into the EDIE environment 102 and instantiated in the virtual environment 104, the EDIE environment 102 can cause bindings to occur between the parameterized inputs of the funnel object 402 and one or more data sources. For example, the EDIE environment 102 can present a user interface to the user, thereby allowing the user to select from one of a plurality of available data sources. After selecting a data source, a request can be issued to the user to bind the respective fields in the data source to the respective parameterized inputs. For example, the user can select a specific column in a database such that the values in that column determine the color of the corresponding sphere in the funnel object in the virtual environment 104. The bindings between the parameterized inputs and the respective data sources can be stored as part of the application such that whenever the user launches the application, the funnel object 402 retrieves data from the underlying data source to generate a display of the funnel object in the virtual environment 104. This binding can be updated and / or changed when the application is launched or while the application is running to dynamically update and / or change to visualize different data sets in the virtual environment 104.

[0061] Figure 4 An example of illustrates the enterprise database 108, which can be used to bind enterprise data to the parameterized inputs of various visualization objects. In other examples, different data sources can be bound to different aspects of the visualization object. For example, some embodiments can allow columns or fields from multiple databases and multiple locations to be bound to the parameterized inputs of the same virtualized object. Some embodiments can use online data sources that provide real-time data to the visualization component. Any combination of data sources can be provided to the parameterized inputs of the virtualized object based on the needs of the respective application designers.

[0062] Figure 3 - Figure 4 The use of the funnel object 402 in is provided only as an example and is not meant to be limiting. It will be understood that any visualization object designed to operate in the EDIE environment 102 can be implemented in the EDIE environment 102 using a similar process.

[0063] Figure 5Illustrated is a virtual dashboard created in a virtual environment 104 according to some embodiments. Although the visualization components described herein can be used in any type of application, they may be particularly suitable for designing virtual dashboards. As used herein, the term dashboard may include a visualization that aggregates, analyzes, and / or displays information from one or more data sources in a user interface. A dashboard may be provided on the home page of an organization's website. A dashboard may also be displayed as an input screen in an application or a mobile app. A dashboard generally displays "widgets" or other graphical representations of data in a two-dimensional environment. A user can often select individual widgets on the dashboard to display additional information. For example, a user can click on a widget that displays summary sales information for the current month. The widget can be configured to provide additional information about the summary sales information displayed on the dashboard. Some dashboards may include multiple widgets that can be configured to display the latest information from underlying data sources when the dashboard is accessed by various user client devices.

[0064] The embodiments described herein use the above visualization components to implement a dashboard in a 3D virtual environment 104. For example, instead of displaying 2D widgets on a computer screen, a user can use a client device enabled for interaction with a 3D virtual environment to provide a 3D visualization experience to view and manipulate dashboard data. Some embodiments may use a virtual reality headset or an augmented reality device that allows a user to enter the virtual environment 104 and perform real-time interactions with visualization objects that display dashboard data. In the virtual environment 104, a user can walk around the visualization objects, "grab" and manipulate parts of the visualization objects, interact with other users, and have their operations on the visualization objects update the underlying data sources. This transforms the traditional 2D dashboard experience into an interactive, immersive virtual reality experience in which data can be viewed in an interactive, live manner.

[0065] The virtual environment may include multiple visualization objects. Instead of multiple 2D widgets, a 3D dashboard may include visualization objects placed in a virtual 3D environment 104. When a user enters or views the 3D virtual environment, they can move around the virtual objects to view them from any angle and operate on them at any location. Additional examples of users interacting with the virtual environment 104 to interact with visualization objects are described in more detail below.

[0066] As described above, a visualization object can refer to a software object represented by the body of code and / or its associated libraries. The content displayed in the virtual environment 104 can be referred to as a view of the visualization object. Each visualization object can be associated with multiple different views, and each view can provide a different look and feel for the visualization object. For example, a funnel object can include a traditional business view that displays data in a more conservative manner suitable for a business environment. The funnel object can also include additional views that display data in a less formal manner, including brighter, more vibrant colors, more interesting shapes, and / or other variations in the way the funnel object is viewed. For simplicity, the remainder of this disclosure will not distinguish between the underlying visualization object represented by the executable code and the view of the visualization object displayed in the virtual environment 104. Instead, the visualization object (e.g., the funnel object) can be collectively referred to as the visualization object to encompass both the code and the display in the virtual environment 104.

[0067] In this example, the virtual environment 104 can include a dashboard with three visualization objects. The funnel object 508 can be linked to the enterprise data repository 502 described above. The graph object 510 can be linked to the multidimensional data cube 504, where the dimensions in the data cube are bound to the respective elements of the graph object 510. The person object 512 can include a summary display of an individual in a three-dimensional grid in the virtual environment 104 (e.g., similar to a virtual business card display). The person object can be bound to the web service 506 such that the person object can download information from the web service 506 to populate the various items in the summary display of the individual. These visualization objects can be rendered at various locations (e.g., in a semicircle) in the virtual environment 104. When a user enters the virtual environment 104, they can view the visualization objects 508, 510, 512 as objects in the area in front of them. To start using the virtual dashboard, the user can approach the visualization objects 508, 510, 512 as needed, view the visualization objects, and / or manipulate the elements of the visualization objects.

[0068] Figure 6Illustrated is how various types of client device systems and input devices according to some embodiments can all dock simultaneously with the same virtual environment in the EDIE environment 102. The EDIE environment 102 can include a 3D data exploration platform that serves multiple different purposes and provides multiple different benefits. For example, the EDIE environment 102 can serve as an integrated link between important, converging new technologies that can make the exploration of enterprise data more immersive and fluid. These technologies can include (e.g., voice- or chat-based) conversational UIs 610, virtual reality (VR) 608, augmented reality (AR) 604, and many other interactive technologies. Users can also access these visualization technologies using traditional 2D computing tools (such as apps on mobile devices 606, web browsers 612 on displays, etc.). The EDIE environment 102 can provide visualizations that seamlessly transition across device changes. For example, a user can view a 3D visualization in a 2D environment (such as on the screen of a mobile device 606). Then, the user can put on a pair of virtual reality goggles and transition to a virtual reality space that includes the visualization objects displayed in 2D on the screen of the mobile device 606. In the virtual environment, the user can interact with the visualization objects using a manual controller typically included in VR systems as described below.

[0069] The EDIE environment 102 can include a hardware detection process 620 that detects the type of user input device used by the client device. For example, if the client device is using a traditional 2D display with a web browser 612, then the hardware detection process 620 can detect that the 2D environment is docking with the virtual environment 104. The EDIE environment 102 can include interface code for Figure 6 each of these different types of input devices shown. The hardware detection process 620 can identify the various input options available and correspondingly load the interface code into the EDIE environment 102. The interface code can then generate a view of the virtual environment 104 that is compatible with the corresponding input device on the client device.

[0070] As described in more detail below, the EDIE environment 102 can operate in a network mode such that the virtual environment 104 can be accessed simultaneously by multiple different client devices. This allows users to interact with each other in the virtual environment 104 while viewing visualization objects tied to underlying enterprise data. The hardware detection process 620 allows users to interact together in the virtual environment 104 based on their own viewing experiences leveraging their client devices. For example, a user who enters the virtual environment 104 using a virtual reality device 608 can be visible in the virtual environment 104 as a display avatar of another user viewing the virtual environment 104 from the 2D screen interface of a web browser 612.

[0071] Figure 7 Illustrates a view of the virtual environment 104 through a 2D screen interface according to some embodiments. The user 702 can access the EDIE environment 102 through a desktop computer, laptop computer, or other computing device equipped with a 2D screen interface. When viewing the virtual environment 104 on the 2D screen 704, a virtual camera can be placed in the virtual environment 104 to render a view of the virtual environment 104 from the perspective of the virtual camera. The rendered image can be displayed on the screen 704 of the user 702. When various objects in the virtual environment 104 are displayed on the screen 704, the user can interact with the various objects in the virtual environment 104 using a mouse or touch screen.

[0072] The virtual environment can include a plurality of visualization objects arranged in the dashboard display as described above. These visualization objects can include a funnel object 508, a graphical object 510, and / or any other visualization object. In addition, the image captured by the virtual camera and displayed on the screen 704 can include the positions of other users in the virtual environment. The view of each user can be captured by a corresponding virtual camera in the virtual environment 104. Instead of displaying the virtual camera, the positions of these cameras can include the display of an avatar or other representation of other users. This example includes the visualization of the user 708 in the virtual environment 104. The user 702 can interact and communicate with the user 708 through the virtual environment 104 as if they were both in the virtual environment 104. When the user 708 turns their virtual camera towards the virtual camera of the user 702, they can see the corresponding visualization or avatar representing the user 702 in the virtual environment 104.

[0073] As described above, the EDIE environment 102 can automatically determine that the screen 704 should display a 2D view of the virtual environment 104. The user 708 can wear a pair of virtual reality goggles, and the EDIE environment 102 running on the client device can provide a 3D view of the virtual dashboard in the virtual environment 104. Thus, users can all interact together using various devices in the same virtual environment. In the virtual environment 104, the type of device used by some users may not be obvious to other users. For example, the user 708 can look at the virtual camera to view the display of the user 702 on the screen 704, and instead of seeing the virtual camera, they can see an avatar in the virtual environment 104. The user 708 may not be able to distinguish whether the user 702 is using a virtual reality headset or a 2D screen 704.

[0074] Figure 8Illustrated is a view of a virtual environment 104 with multiple users interacting with a virtual dashboard according to some embodiments. In this example, users 804, 806 are represented by avatars having a human appearance. Users can choose avatars that look like themselves or any other virtual character. These avatars can move within the virtual environment 104 as the user moves in the real-world environment using a virtual reality headset. As the user moves their head in the real environment, the virtual camera can move, rotate, pan, tilt, etc. This allows the user to "walk around" the virtual dashboard in the virtual environment 104 to view various visualization objects from different angles.

[0075] The movement of various users can be represented within the virtual environment 104 to be visible to other users. For example, compatible virtual reality equipment includes a virtual reality headset and a selection device that can be held in the user's hand. When the selection device or the virtual reality headset moves in the real-world environment, that movement can be used to drive the movement of the avatar in the virtual environment 104. For example, when user 408 raises their hand in the real-world environment, the hand of the avatar can also be raised in the virtual environment 104, making them visible to user 806. This allows users to gesture and point at visualization objects in the virtual dashboard and enables other users to see these movements.

[0076] Figure 8 Also illustrated is a virtual camera 802 that can be used to capture a view of the virtual environment 104 to obtain an image to be displayed on a 2D screen. As described above, the depiction of the virtual camera 802 can be replaced with an avatar for user 702 to view the screen 704. Alternatively, the virtual camera 802 may not have a visual equivalent within the virtual environment 104, or it can be replaced by any other visual indicator as a 3D object that can be rendered for other users to view.

[0077] Users who use both the above 2D and 3D displays can interact with the elements of the visualization objects in the virtual dashboard. For example, user 804 can approach the funnel object 508 and select one of the spheres that rotate around the funnel object 508. In the 2D display, a finger tap or a mouse click can be used to select the sphere. In the 3D display, the sphere can be selected by using one of the handheld virtual reality devices held by user 804 to reach out and "grab" one of the spheres while the sphere rotates around the funnel object 508. As described above, the spheres can be bound to data objects in the data repository. For example, the spheres can represent rows in the database, and the position, color, speed, trajectory, etc. of the spheres in the funnel object 508 can be determined by different values in the columns of the rows in the database. When user 804 holds the sphere, additional information about the sphere can be displayed in the user interface or a heads-up display (HUD). For example, each column value in the row of the database can be displayed for user 804. The spheres can be passed back and forth between user 804 and user 806 so that both users 804 and 806 can view the details of the underlying data objects.

[0078] When user 804 has finished inspecting the sphere, user 804 can put the sphere back into the funnel object 508. In some embodiments, user 804 can "put back" the sphere into the funnel object 508, and the sphere can automatically return to its previous position / rotation in the funnel object 508. In other embodiments, user 804 can place the sphere in a position in the funnel object that is different from the original retrieval position. As described above, the position of the sphere in the funnel object 508 can be determined by the values in the underlying dataset bound to the funnel object 508. When the position of the sphere in the funnel object 508 changes, the value corresponding to that position can also change. In some embodiments, the value corresponding to the new position in the funnel object 508 can be written back to the underlying dataset. This allows the user to manipulate the elements of the virtualized objects in the virtual environment 104 and change the values in the dataset to which the visualization objects are bound.

[0079] In some embodiments, the views of the visualization objects provided to each of users 804 and 806 can be the same. For example, when user 804 pulls the sphere out of the funnel object 508 as described above, user 806 will see the sphere leave the funnel object 508. Similarly, when user 804 puts the sphere back into the funnel object 508, user 806 can see the new position of the sphere in the funnel object 508. Thus, using the virtual dashboard in the virtual environment 104, the changes to the visualization objects and their underlying datasets are common for each user.

[0080] In other embodiments, each of users 804, 806 can be provided with their own view of the visualization object. In these embodiments, the state of each visualization object can be saved uniquely for each user 804, 806. For example, when user 804 pulls the sphere out of the funnel object 508 as described above, user 806 will continue to see the sphere rotating within the funnel object 508. Thus, changes made by one user 804 to an element of the visualization object will only be seen by that particular user 804. Other users 806 will continue to see the visualization object in its unchanged state. Embodiments can handle changes made by one user to the underlying data set according to stored preferences. For example, some embodiments can propagate the changes to the underlying data set, while other embodiments can provide the data set in a "read-only" configuration such that changes to the visualization object are not reflected in the underlying data set. Some embodiments can present the changes made to all users present in the virtual environment 104 for approval before they are written to the underlying data set.

[0081] As described above, each visualization object can have one or more views or view models associated with the visualization object. These view models can control the appearance and feel of the visualization object when it is rendered in the virtual environment 104. The EDIE environment 102 allows each user to select an individual view model for the same visualization object. For example, user 804 can choose to view the funnel object 508 using a business view model that uses muted colors and smaller spheres. User 806 can choose to view the funnel object 508 using an informal view model that uses brighter colors and larger spheres. This allows each individual user to customize the appearance and feel of the visualization object in the virtual dashboard without changing the view model for other users.

[0082] Figure 9 A view of a real-world environment 908 that can be used to interact with the virtual environment 104 is illustrated according to some embodiments. User 902 can use virtual reality equipment, including a virtual reality headset 904 and / or one or more handheld selection devices 906. When user 902 moves within the real-world environment 908, a virtual camera in the virtual environment 104 can move in a corresponding manner within the virtual environment 104. The virtual camera can capture a view of the virtual environment 104 that is displayed to user 902 through the virtual reality headset 904. When user 902 moves the hand that holds the selection device 906, the hand of the corresponding avatar in the virtual environment 104 can also move and select objects in the virtual environment 104 as described above.

[0083] Figure 10FIG. 1000 is a flow chart of a method for displaying data in a multi-dimensional dashboard according to some embodiments. The method may include accessing a plurality of 3D visualization components (1002). Each of the 3D visualization components may include a parameterized input for receiving a data set. The 3D visualization components may be downloaded or otherwise received from a component exchanger that allows developers to provide 3D visualization components to be used in a plurality of applications under development. The 3D visualization components may be downloaded to a client device or an operating environment (such as the EDIE environment described above). For example, the 3D visualization components may include funnel objects, graphic objects, and / or other visualization components described herein.

[0084] The method may further include receiving one or more data sets (1004). The data sets may represent enterprise data received from enterprise applications and / or databases. For example, the data sets may include customer relationship management (CRM) databases, human capital management (HCM) databases, financial software databases, and / or any other type of enterprise data available in the operating environment.

[0085] The method may additionally include binding the parameterized inputs of the plurality of 3D visualization components to one or more data sets (1006). For example, for each of the 3D visualization components, the corresponding parameterized input may be bound to one of the one or more data sets. The parameterized input of the 3D visualization component may make the 3D visualization component generic such that its appearance and operation can be customized with the bound data set. When the component is placed in an application, the data set may be bound to the corresponding 3D visualization component at design time. When the 3D visualization component is instantiated at runtime, the data set may alternatively or additionally be bound. During runtime, the system may receive input (e.g., from a user) to change the data set bound to a different data set. Some 3D visualization components may be bound to multiple different data sets, each of which may define a visual or operational aspect of the 3D visualization component.

[0086] The method may further include rendering a plurality of 3D virtual objects (1008) in a virtual environment based on a plurality of 3D visualization components and one or more data sets. Each of the 3D visualization components may include a view model, an animation model, a wireframe skeleton, and / or other graphical constructs that may be used to generate 3D virtual objects in the virtual environment. The virtual environment may be implemented in an EDIE operating environment to create a 3D virtual scene that also includes objects other than the 3D virtual objects from the 3D visualization components. For example, the virtual environment may be configured as a carousel of 3D virtual objects. In another example, the virtual environment may be configured as an enterprise dashboard that displays 3D objects as virtual "widgets" that may be viewed by a user in the virtual environment. Each 3D virtual object in the dashboard may be bound to different enterprise data sets from different databases and / or different applications. For example, the dashboard may provide summary data for finance, employees, operations, customers, etc. in a unified display within a single virtual environment.

[0087] The method may additionally include receiving connections to the virtual environment from a plurality of client devices (1010). Some embodiments may include client devices on which a hardware detection process is installed that is configured to detect the type of display and / or input device used with the client device. The hardware detection process may automatically configure the interaction with the virtual environment to match the type of client device. For example, if the client device includes a virtual reality headset, then the hardware detection process may be configured to provide an immersive view and / or interaction with the virtual environment such that the user feels as if they are working within the virtual environment rather than in the real-world environment. If the client device includes a 2D screen (e.g., a monitor, laptop computer, etc.), then the hardware detection process may be configured to place a virtual camera within the virtual environment to capture a 2D image of the virtual environment at the location to be displayed on the 2D screen.

[0088] The method may additionally include providing a plurality of views of the plurality of 3D virtual objects in the virtual environment to the plurality of client devices (1012). An individual view of the virtual environment may be provided to each client device. Additionally, each client device may be configured to allow a user to interact with the 3D virtual objects in the virtual environment to change their display. For example, a user may remove an element corresponding to an individual element in the corresponding data set from a virtual object. The user may see a display that includes summary information about the data points in that data element and may be allowed to change certain data points. These changes may be updated in the underlying data source to which the 3D virtual object is bound. In some embodiments, the changes may be immediately viewable by other users in the virtual environment. Alternatively, some changes may be visible only to the user making the changes such that an individualized view of the virtual environment is provided for each user.

[0089] It should be recognized that, according to various embodiments, Figure 10 the specific steps shown provide a specific method of displaying data in a multi-dimensional dashboard. According to alternative embodiments, other step sequences may also be performed. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Moreover, Figure 10 each of the steps shown may include multiple sub-steps, which may be performed in various orders suitable for each step. Additionally, depending on the specific application, additional steps may be added or removed. Those of ordinary skill in the art will recognize many variations, modifications, and alternatives.

[0090] Optimizing Views Using Machine Learning and Voice Commands

[0091] As described above, the imported data set may be represented by one or more visualization components in a virtual environment. Many different types of visualization components may be imported into the virtual environment to represent the data sets to which they are bound. The shapes, designs, and animations available for each visualization component may be specifically designed to provide a visual experience for users to view, understand, and manipulate complex data sets. Instead of simply viewing a graph or 2D pie chart on a web page, users may also see advanced animations in three dimensions that can illustrate the way data changes over time, how data interacts with each other, and what changes have propagated through the data to achieve a desired result. These visualization components have distinct advantages compared to traditional two-dimensional dashboards used to display and interact with enterprise data.

[0092] The following figures illustrate examples of visualization components that may be used to display and interact with specific types of data. Figure 11 An example of a visualization object in the form of a funnel 1100 is illustrated according to some embodiments. The funnel 1100 may be referred to as a final object, a funnel visualization component, or a funnel 3D object. The funnel 1100 may be useful for visualizing large data sets having thousands or even millions of data points. The funnel 1100 may provide multiple simultaneous visual indicators that can collectively convey information about each data point and about the set of data points. These visual indicators may include the coloring of the data points, the size of the data points, the radial distance of the data points from the center of the funnel 1100, the radial angle of the data points, the rotational speed of the data points around the funnel 1100, the distance along the length of the funnel 1100, etc. Attributes in the data set may be used to plot the individual data points around the funnel 1100 and control their appearance and / or animation to simultaneously convey all of this information to the user in a visual display.

[0093] As described above, the visualization component for the funnel 1100 can be designed to be agnostic to any specific data set. Instead, the funnel 1100 can include a collection of parameterized inputs to which an existing data set can be bound. By way of example, the following discussion can use a specific type of data set to illustrate how the funnel 1100 can view and manipulate the data set. This example data set can include a data table imported from or accessed by a sales cloud application. This data table can include sales opportunities in each row of the data table. Different columns in the data table can include all the attributes of the sales opportunity. These attributes can include interactions between individuals, scores, progress indicators, and / or any other characteristics for developing the sales opportunity into a completed sale. When the visualization component for the funnel 1100 is imported into an application (such as a virtual dashboard application), the application designer and / or user can select the sales opportunity table as the data set to be represented by the funnel 1100. Then, data points from the data table can be used to construct a 3D object representing the funnel 1100 to control the visual aspects of the funnel 1100. These visual aspects will be described in detail below, but in short, each row in the data table can correspond to a sphere that rotates around the funnel 1100 in the funnel 1100. The rotation speed, angle, trajectory, color, size, and / or other characteristics of each sphere can be defined by the columns in the corresponding data row.

[0094] It will be understood that the use of sales opportunity data for the underlying data set for the funnel 1100 is used only by way of example and is not meant to be limiting. As emphasized above, the design of the visualization component for the funnel 1100 does not depend on the underlying data set. Instead, any data set can be bound to the funnel 1100 in order to fill the funnel 1100 with spheres representing data points. Sales opportunity information is merely an example of a data set that illustrates how the attributes of data points can be interactively visualized using the funnel 1100. Other data sets can include financial data, scientific data, test or experimental data, survey data, and / or any other type of data source.

[0095] First, the distance 1106 on the funnel 1100 can convey the meaning of each data point. The distance 1106 from the origin of the funnel 1100 can be determined by the column value of each row corresponding to the spheres in the funnel 1100. For example, for each individual data point, the distance 1106 can indicate the relative progress of the data point from a start value to an end value. For example, for a data point representing a sales opportunity, the farther the data point is from the origin of the funnel 1100, the farther the sales opportunity is from successful completion. Thus, the data point 1107 will represent a sales opportunity in the initial stage of progress, while the data point 1108 will represent a sales opportunity close to successful completion. The shape of the funnel 1100 itself can also convey this type of meaning. Funnels generally use their physical counterparts to receive new material at a larger opening and then pour that material (e.g., liquid) "through the funnel" gradually towards the smaller end of the funnel. By using the shape of the funnel 1100 in a virtual environment, the user can intuitively understand that the data points will progress from a larger outer area towards a smaller inner area within the funnel. Thus, the funnel 1100 is very useful for showing the progress of individual data points within a large set of data points. When constructing the funnel 1100 in a virtual environment, each sphere can be placed at a position or distance 1106 within the funnel based on this value from the data set.

[0096] In addition to showing the progress of individual data points within a set of data points, the funnel 1100 is also very useful for showing the overall trend of the data point progress. In Figure 11 the illustration, in the funnel visualization, there are thousands of data points in the data set, which are represented by small spheres. In addition to the ability to focus on individual data points, this visualization also provides an illustration of the clustering or natural grouping of the data to show trends within the data set. For example, a clustering of data points far from the origin (e.g., stage 5) in the start stage (e.g., stage 1) of the funnel 1100 can represent a large number of sales opportunities in the start stage; however, a small number of data points near the origin (e.g., stage 5) of the funnel 1100 will illustrate that relatively few sales opportunities have been pursued or completed. The funnel 1100 can also quickly provide a way to assess the total number of data points in the data set. For example, a sparsely filled funnel indicates that few sales opportunities are available. Conversely, a funnel densely filled at each level indicates that there are many opportunities evenly distributed among different progress states.

[0097] By creating thresholds, bins, or groupings of the data that will be shown in the rendering of the funnel itself 1100, the natural grouping or clustering of the data that is visually apparent in the funnel 1100 can be enhanced. For example, the funnel 1100 can include a first stage 1101, a second stage 1102, a third stage 1103, a fourth stage 1104, and a fifth stage 1105. The divisions between the stages can be established by user-defined thresholds and can represent the stages that data points progress through towards a final result. For example, the first stage 1101 can represent sales opportunities that have been identified but not yet evaluated or acted upon. The fifth stage 1105 can represent sales opportunities that have been nurtured into completed sales. In some embodiments, defined boundaries, different color schemes, and / or other means such as Figure 11 shown can be used to visually depict each stage. In some embodiments, this can include a representation that appears to be the funnel itself, while other embodiments can simply form the funnel shape by means of the data points that fill the funnel.

[0098] The different stages 1101, 1102, 1103, 1104, 1105 can be regularly spaced along the length of the funnel 1100. The relative width of each of the stages 1101, 1102, 1103, 1104, 1105 can be determined based on a static interval defined by the funnel 1100. The relative width can also be determined by the number of data points within each stage. For example, when a larger number of data points reside in this first stage 1101, the width of the first stage 1101 can be larger. The colors of the individual stages can be transparent such that they cover respective regions of the funnel 1100 while still allowing the data points that cycle through them to be easily viewed. As described above, the user can interact with the funnel by selecting the individual data points that rotate within the funnel 1100. The different stages do not need to be modeled as solid surfaces in a virtual environment, but rather the user can “pass through” each stage to interact with the data points that cycle through them.

[0099] Some embodiments can present the data points in the funnel 1100 in a static rather than a moving manner. Alternatively or additionally, some embodiments can also animate the movement of the data points in the funnel 1100 based on one or more attributes. For example, Figure 11The data points in the funnel 1100 can be animated to rotate around the central axis of the funnel 1100 based on the frequency of actions involving the specific data points. In the example of data points representing sales data, the speed at which the various data points rotate around the central axis of the funnel 1100 can be determined by the number or rate of sales actions taken on that data point (customer contact, email, direct sales, etc.). This allows the user to visually identify data points that are not being acted on to interpret their relative distance from the origin of the funnel 1100. Data points that are not moving or moving slowly indicate that the sales opportunity is not progressing. Conversely, if the opportunity in the first stage 1101 is circulating around the funnel 1100 at a high rate, then the opportunity may be advancing rapidly. This also provides a natural and realistic visualization of the data points as they advance toward the starting point of the funnel 1100. Like a real-world funnel, the data points will gradually advance toward the origin of the funnel 1100 by rotating around the central axis of the funnel 1100 and increase in speed until they reach the origin. This also provides an overall overview of the efficiency and consistency of the data advancement. By animating all of the data points, it becomes intuitive to see which data points are receiving more or less attention than others, which data points are advancing slower or faster than others, and other comparable visual information that is easy to convey using this visualization.

[0100] As with any 3D visualization, there are technical issues in determining how to best scale, display, view, rotate, and otherwise present data to a user. When thousands of data points are available, displaying all of them at once may overwhelm the user with the data points and additionally obscure visible trends that might otherwise be apparent. The embodiments described herein analyze a data set and determine an optimized initial view of a visualization that is most beneficial to the user. These embodiments also provide a plurality of predetermined views that are configured to optimally display the data depending on the number of data points, the shape of a predetermined 3D shape in which the data points are arranged, and the number of different divisions in the predetermined 3D shape. In addition, the type of data displayed and / or the display view can be determined based on voice commands or other contextual queries received through a user interface. Over time, the system can learn which data sets are best suited for processing various voice commands.

[0101] Figure 12AA view of a 3D object representing data points at a first distance is illustrated. From this distance, which is considered relatively far, the views of the individual data points can be ranked behind the view of the funnel shape itself. Since the views are too far from the individual data points, they may not be emphasized. This de-emphasis can include showing a smaller number of data points, making the data points more transparent, fading the color of the data points, etc. In some embodiments, the data points can be grouped together at a greater distance and represented by a single sphere in the display. For example, every three data points can be grouped together and represented by a single data point to avoid cluttering the display of the object.

[0102] In contrast, the rendering of the visualization component (i.e., the funnel) can be emphasized at a greater distance. This allows the user to view and identify the visualization component by its shape. As described above, multiple 3D objects from the visualization component can be included as part of a virtual dashboard in a virtual environment. If the data points were shown individually, the overall predefined 3D shape in which the 3D objects representing the data points are arranged may not be as apparent. In these embodiments, the overall shape can be emphasized so that it is easily recognizable. Emphasizing the funnel object can include reducing the transparency of the object's outer shell, increasing the contrast or vividness of the color used to display the funnel object, rendering the funnel object as a solid surface or volume, etc.

[0103] In some embodiments, additional 3D constructs can be added to the virtual environment to form a predefined shape in which the 3D objects representing the data points can be arranged. In the example of the funnel visualization component, the individual 3D objects representing the data points can be arranged in a funnel shape as described above. However, when viewed from a distance, the number and / or density of the spheres may be too dense or sparse for the shape of the funnel to be easily recognizable. In some embodiments, additional 3D constructs can be added to form the overall predefined 3D shape of the visualization component at a distance.

[0104] In this example, a conical cross-section can be generated based on determining that the view of the object representing the visualization component is beyond a threshold distance. For example, if generated Figure 12AIf the virtual camera of the user view shown is a predetermined distance from the funnel (e.g., 15 feet in the virtual environment), then the system can generate one or more 3D constructs to represent the shape of a predetermined 3D shape. To construct the funnel object, a plurality of conical rings 1202 can be formed as 3D constructs and inserted into the virtual environment to define the shape of the funnel. The conical rings 1202 can be configured to form surface objects and be rendered together such that the shape of the funnel is more apparent to an observer at a certain distance. These conical rings 1202 can form the structure of the funnel object and can be emphasized on the respective spheres representing the data points as described above. For example, the conical rings 1202 can be rendered such that they are opaque. They can be generated with the colors and / or textures representing the data points in the funnel region. They can be rendered such that they contain the funnel shape formed by the respective data points, and all of these rendered spheres of the data points are inside the funnel formed by the conical rings 1202. This can cause the user to focus on the shape of the outer shell formed by the conical rings 1202 and understand the nature of the visualization component, rather than being distracted or confused by the more ambiguous shape formed by the spheres rotating at a distance.

[0105] In some embodiments, the respective spheres can continue to rotate and / or be visible inside the conical rings 1202 such that the user can also understand that real-time data is available for viewing and / or interacting within the 3D object of the visualization component. For example, the user can see the respective data points rotating from a certain distance inside the conical rings 1202. Although this distance may be large enough that the user cannot distinguish the respective data points and / or visually isolate the respective data points, it may be sufficient for them to understand that there are multiple data points available within the funnel object. The relative sizes of these spheres can also make it obvious that moving closer to the funnel object can start to emphasize the respective data points on the funnel shape formed by the conical rings 1202.

[0106] In addition to the funnel object itself, this embodiment also illustrates view controls 1204 that can be used to instantaneously switch between views. The current view of the funnel is a "Zoom Out" view that can display the funnel from a greater distance. However, by selecting any of the controls 1204, the user can immediately change the view to the view described by the corresponding control. These controls can include a "side view", an "end view", a "Zoom In" view, etc. When rendering a 2D image, these controls 1204 can change the position of the virtual camera in the corresponding virtual 3D space. When using an AR / VR device, using these controls 1204 can immediately change the position of the user in the virtual 3D space such that they are viewing the funnel 1100 from the position that will generate the corresponding view.

[0107] Input can be provided through multiple different mechanisms. For example, finger swipes, taps, pinches, etc. on a touchscreen device can be used to select the control 1204. A mouse pointer on a traditional computing device can also be used to select the control 1204. Some embodiments can also integrate artificial intelligence and speech recognition technologies to navigate around the funnel. As described above, some embodiments can use an artificial intelligence module to operate a chatbot or other dialogue-based system with the user. For example, the user can ask the system "show me a funnel view of these data", and in response, the funnel view can be automatically generated. Similarly, the user can say phrases such as "back" or "in", to convey the movement intention to the system. Then, the system can use the artificial intelligence module to decrypt the intention and accordingly select one of the controls corresponding to the intention. For example, a phrase such as "back" can be used to select the "zoom out" command from the list of controls 1204.

[0108] Figure 12B A diagram illustrating a system that uses a model-based system to interpret views and / or data display commands according to some embodiments. The client device 1220 can provide input to the system through the command interpreter 1222. The command interpreter 1222 can receive text commands and / or voice commands that request a specific view of a specific type of data. The command interpreter 1222 can include a knowledge base and / or an intention engine that interprets the commands and reformats or transforms the commands into requests known to the system. For example, the command "let me see all of the data" can be interpreted such that it is replaced with the known command "zoom out". Thus, the command interpreter 1222 can receive any plain English command received from the client device 1220 and transform the command into a command that the system can receive.

[0109] The command can then be sent to the model 1226. For example, the model can include a neural network that includes multiple inputs. In some embodiments, the inputs of the neural network can match the known commands output from the command interpreter 1222. The neural network can filter the commands from the command interpreter 1222 through the neural network to generate a signal on one of the outputs of the neural network. The output of the neural network can correspond to different view options. For example, the output of the neural network can include commands corresponding to the Figure 12A controls 1204 in to provide different predefined views in a virtual environment. In the above example, the "zoom out" command received from the command interpreter 1222 can activate an output on the neural network that activates the "zoom out" command 1206 in the controls 1204 of the interface.

[0110] Some embodiments may include a second neural network that also receives commands from the command interpreter 1222. The first neural network may receive commands related to different views of the 3D object, while the second neural network may receive commands related to the type of data to be displayed. For example, the command interpreter 1222 may receive a request from the client device 1220, such as "show me sales data for last quarter". The command interpreter 1222 may generate a known command to "display sales data". This command may be input into the model 1226, and the output of the model may indicate the specific data set to be retrieved in response to the input command. For example, this command may activate an output on the neural network that sends an SQL command to load the sales data for the indicated time interval from the sales database. The command may be executed on the enterprise database 108 to retrieve the indicated data set. Thus, the model 1226 may generate the desired view based on the user command. The model 1226 may also determine the best data set to load for the indicated command.

[0111] In some embodiments, the model 1226 may include a third neural network that also receives commands from the command interpreter 1222. The third neural network may be configured to receive these commands and provide an output indicating the specific visualization component to be loaded from the component switch 124. The parameterized input from the indicated visualization component may be bound to a data set selected from the enterprise data rendered in the virtual environment 104 according to the view determined by the first neural network. Thus, the model 1226 may receive a general voice command requesting to display a data type, and the model 1226 may cause the virtual environment 104 to load the visualization component, bind the data set, and display the data based on the model output according to the view that may respond to the command.

[0112] After rendering the view of the visualization object in the virtual environment 104, the system may monitor any view input 1224 provided by the client device 1220. These view inputs 1224 may include manipulating the view by scanning, panning, tilting, rotating, and / or otherwise moving the virtual camera of the client device 1220 in the virtual environment 104. These view inputs 1224 may also cause the data set bound to the visualization component to change (e.g., the user may select a different data set to bind to the funnel object). These view inputs 1224 may also cause the visualization object to be changed in the virtual environment 104. For example, the user may swap the funnel visualization object for a three-dimensional graphic object.

[0113] These view inputs 1224 can be interpreted as corrections to views, visual objects, and / or data sets automatically provided by the model 1226. Thus, some embodiments may send the view inputs 1224 to both the model 1226 and the virtual environment 104. When the model 1226 receives these view inputs 1224, the model may enter a training mode that causes the neural network in the model 1226 to be retrained based on the view inputs 1224. For example, in the training mode, the model 1226 may consider commands provided by the command interpreter 1222 that originally generated the views, visual objects, and data sets for the virtual environment 104. The model 1226 may also receive any view inputs 1224 received after information has been provided to the user. The view inputs may be matched with different outputs on each neural network such that it can be retrained such that the command is more likely to select the new output instead of the original output. Continuing with the above example, a view input that changes the visualization component to a 3D graphics component may be sent to the model 1226. The third neural network may be retrained such that the same command from the command interpreter 1222 causes the output corresponding to the graphics component to receive a higher weight along the neural path in the third neural network. Any minimization algorithm may be used to adjust these weights. The next time the same command is received from the command interpreter 1222, the model 1226 will be more likely to select the graphics object instead of the funnel object.

[0114] When the model 1226 is used by various client devices and users, the system allows the model 1226 to be continuously retrained. The model 1226 can adapt over time such that it becomes more responsive to user input in a way that makes it less likely for the user to provide view inputs 1224 that change the output of the model 1226. Over time, as the model 1226 is continuously trained, commands for viewing a particular type of data, views of particular data, and viewing data using particular visual objects are more likely to be correct.

[0115] Figure 13The figure shows a close-up view of a 3D shape according to some embodiments. As the user's line of sight moves closer to the funnel, compared to the 3D shape of the funnel object, the individual 3D objects representing data points can start to be emphasized. These 3D objects representing data points can be emphasized by displaying more 3D objects, reducing the transparency of the 3D objects, increasing the contrast or vividness of the colors of the 3D objects, increasing the size of the 3D objects, and / or any other visual techniques for emphasizing these 3D objects. In contrast, compared to the data points, the image of the funnel formed by the 3D shape can start to be deemphasized. Deemphasizing can increase the transparency of the conical ring 1202, soften the color of the conical ring 1202, etc. Moving closer to the funnel can also display text information, such as data point values, date labels, stage labels, and other text information that may be too small to read when viewed from a distance.

[0116] In some embodiments, as the line of sight moves closer to the view of the visualization component, any 3D constructs inserted to clearly display the 3D shape of the visualization component can be deemphasized and / or removed. In contrast, the individual 3D objects representing data points (e.g., spheres in the funnel) can gradually become more emphasized. In Figure 13 the example, after exceeding a first threshold distance from the view of the visualization component, half of the conical ring 1202 can be removed. With the remaining half of the conical ring 1202 still visible, this can provide the user with a view that is still clearly a funnel object. This view can also provide the user with more detailed information shown by the 3D objects 1304 representing the individual data points by removing a part of the 3D construct. As described above, the 3D objects representing data points and their size, color, motion, trajectory, and / or other visual characteristics providing information about the individual data points can be made more visible.

[0117] Figure 14 The figure shows an optimized view of a data set according to some embodiments. This view can be an initial view automatically provided by the system to visualize the data in the funnel object. The optimized view can be determined as the view that: (1) displays all available data points; (2) fills the display area; and (3) displays the data from the angle that is most useful to the user. In this embodiment, the funnel object substantially fills the view area for the user and displays each of the individual data points in a complete view. This view is also at an angle tilted with respect to the central axis of the funnel object, such that the 3D nature of the display is emphasized.

[0118] When the view moves closer to the view of the visualization component 1302, a second threshold may be crossed, enabling additional changes to the display of the visualization component 1302. For example, after exceeding the second threshold, a second level of detail may be displayed for each 3D object 1304 representing a data point. Instead of displaying them as "dots" that cycle around the funnel in a conical ring 1202, a full rendering of each 3D object 1304 may be rendered and displayed. The transparency of the conical ring 1202 may be increased so that all rotations and movements of the 3D objects 1304 are visible. Additionally, the full color, size, speed, and movement of all 3D objects 1304 may be displayed. In this example, when displayed together with the conical ring 1202, the color, position, size, and movement of the spheres rotating around the funnel may be fully visible.

[0119] Figure 15 Illustrated is a fully zoomed-in view of the rendered visualization component 1302 according to some embodiments. In this view, a third threshold distance may be exceeded such that the view of the visualization component 1302 substantially fills the view of the display device. For example, the view of the visualization component 1302 may fill approximately 75% of the view of the display device. This may correspond to a "zoomed-in" view from the control 1204 in Figure 12A At this closer distance, the conical ring 1202 of the funnel may be completely removed. More generally, any 3D constructs that were previously added to convey the 3D shape of the visualization component may be removed. Alternatively, only the individual 3D objects 1304 representing the respective data points may remain in the display. Due to the close proximity to the 3D objects 1304 representing the data points, the overall shape of the visualization component 1302 may be clear to the user (e.g., the shape of the funnel may be clear from the rotating spheres). Additionally, any 3D constructs that were added to convey the shape of the visualization component 1302 may only obscure the view of the 3D objects 1304 representing the respective data points. Since the user is more likely to interact with the individual data points at this close distance, they may be emphasized sufficiently compared to any previously provided structural shapes.

[0120] It should be emphasized that the visualization component of the funnel is provided only by way of example and is not meant to be limiting. Other visualization components, such as 3D bar charts, may also be used. The funnel is provided only as an example to illustrate how different views may be generated based on the distance to the view of the visualization component.

[0121] Figure 16FIG. 1600 is a flow chart of a method for determining an optimal view of a visualization component as described above. The method can start by viewing a visualization object at a relatively far distance (1602). The visualization object can be the funnel object described above. Then, the method can determine whether a first threshold has been exceeded when zooming in (1604). If the first threshold has been exceeded, then as the view zooms in, the system can gradually de-emphasize the visualization component and gradually emphasize individual data points. Emphasizing or de-emphasizing the visualization component can include emphasizing or de-emphasizing any 3D constructs representing the shape of the visualization component. Emphasizing or de-emphasizing the data points can include emphasizing or de-emphasizing 3D objects representing the data points. This gradual emphasis / de-emphasis can continue until a second threshold is exceeded (1608). At this point, the data points can be fully emphasized and the visualization object can be de-emphasized (1610).

[0122] Another method can include receiving one or more data sets from an enterprise application. The data sets can include a time series of data for a single data attribute. The method can also include generating 3D objects corresponding to each data point in the one or more data sets. The 3D objects can include 3D spheres as described above or otherwise drawn. The method can also include arranging the 3D objects in a predetermined 3D shape. The 3D shape can be formed by the 3D objects themselves or can be created independently in 3D virtual space with the 3D objects surrounding the 3D shape. For example, the 3D shape can include a funnel object as described and drawn above. The method can also include determining an optimal view position for a virtual camera to render the 3D shape. The optimal view can be determined by selecting a view distance that will include all the data points and fill a threshold percentage (e.g., 90%, 75%, etc.) of the view area. The optimal view position can also be determined based on a perspective that emphasizes the three-dimensional data. The perspective can be predetermined based on the type of visualization object. For example, a funnel object can have an initial perspective that is tilted relative to the central axis of the funnel. Other view objects may have a specific perspective that emphasizes their 3D nature. Additionally, some embodiments can determine the initial position and / or perspective based on predetermined data provided by the designer of a particular visualization object.

[0123] When viewed from a certain distance, a complex visualization showing many data points can be represented by one or more simpler shapes, which can improve performance when fine details cannot be discerned. The visibility and opacity of the data points, as well as the simpler shapes, can be proportional to the distance between the visualization and the camera. For example, the system can calculate the distance between the center of the bounding box of the visualization and the camera. When this distance is greater than a maximum threshold, the simple shapes are visible, but the data points are not. When the distance is less than a minimum threshold, the data points are visible, but the simple shapes are not. Between the minimum and maximum threshold distances, both the data points and the simple shapes are visible, but their opacities are different. The opacity of the data points is inversely proportional to the distance, such that the points are minimally opaque at the maximum threshold and maximally opaque at the minimum threshold. The opacity of the simple shapes varies directly proportional to the distance, such that the shapes are maximally opaque at the maximum threshold and minimally opaque at the minimum threshold. When the user approaches from a certain distance, initially only the simple shapes are visible. As the user gets closer, the simple shapes fade out as the data points fade in. When the user is very close, only the data points are visible.

[0124] Figure 17 The figure illustrates a flowchart of a method for generating a visualization object from a set of data points according to some embodiments. The visualization object or visualization component can be downloaded or exchanged from a component library and included in an application. The application can include a virtual environment that allows the creation and rendering of 3D objects to generate views for multiple users using a variety of client device types. As described above, a visualization object (such as a funnel object) can include a parameterized input set that can be bound to an underlying data set. The method describes how the shape and appearance of a predetermined 3D shape can be generated using these data sets.

[0125] The method can include receiving one or more data sets (1702) from an enterprise application. One or more data sets can be received from any type of data source, where the data source is such as a database, table, spreadsheet, text file, CSV file, web service, website, XML file, storage repository, and / or any other type of data structure or data storage device. The data sets can be received from an application such as an enterprise application that provides enterprise data. The data can include CRM data, HCM data, financial data, sales data, and / or any other type of data. When loading or downloading a visualization component, the designer and / or user can choose to bind one or more data sets to the visualization component through one or more parameterized inputs.

[0126] The method may further include generating 3D objects (1704) corresponding to each of the data points in one or more datasets. Each 3D object may correspond to a single data point in one or more datasets. For example, each 3D object may correspond to a row in a database. Each object may also include a plurality of attributes or attribute values that may be associated with the data point. For example, the attribute values may include the values stored in the columns of a row in a database table. Each of these values may be used to construct the appearance and / or behavior of the 3D object, as described below. The 3D object may include a 3D sphere as described above or otherwise depicted. The 3D object may also include any other type of three-dimensional object, such as bars, polygons, stars, and / or other solid objects.

[0127] The method may further include arranging the 3D objects into a predetermined 3D shape (1706). The 3D shape may be formed by the 3D objects themselves or may be created independently in the 3D virtual space, where the 3D objects surround the 3D shape. For example, the 3D shape may include a funnel object as described and depicted above. The 3D objects may be arranged such that their collective arrangement forms a predetermined 3D shape. Alternatively, additional 3D objects, such as surfaces, wireframes, and / or other 3D constructs, may be used to generate and represent at least a portion of the predetermined 3D shape. In these embodiments, the 3D objects may be arranged around the other 3D constructs such that they conform to the predetermined 3D shape.

[0128] The method may also include receiving a command (1708) that indicates a first view of a predefined 3D shape. The command may be received via a voice command, a text command, or any other type of input. In some embodiments, the command may be received as a plain English phrase, such as “show me last month’s sales data”. The command may be processed by a command interpreter that converts the plain English phrase into a standardized command. These standardized commands may include inputs corresponding to inputs of a model or a neural network. The model or neural network may receive the simple command and generate a set of outputs. In some embodiments, the neural network may include multiple neural networks, each of which generates a different type of output based on the command. For example, a specific neural network may be provided that outputs the view position of a virtual camera, a specific visualization component to be loaded, and / or a specific data set of parameterized inputs to be bound to the visualization component. These outputs from the model may be provided to the virtual environment, and the visualization component may be instantiated in the virtual environment and viewed at the defined view position. The visualization component may replace a previous visualization component that has already been instantiated in the virtual environment. Alternatively, this visualization component may be added to the virtual environment as a new 3D widget in a virtual dashboard. In a simple case, the command that indicates the first view of the predefined 3D shape may provide outputs on a neural network that determine different view positions, angles, foci, etc. Then, the existing 3D objects arranged in the 3D shape in the virtual environment may be viewed based on the outputs of the neural network. In other words, the command that indicates the first view does not need to load any additional or alternative visualization components, but may simply reorient the view of the existing visualization components.

[0129] The method may also include determining a distance associated with the first view (1710). The distance may be measured between the virtual camera and the predefined 3D shape of the visualization component. The measurement may be made in units specific to the virtual environment, and these units may correspond to physical units in the corresponding real environment. For example, in both the virtual environment and the real environment, the distance may be 20 feet. For example, for a user wearing a virtual reality headset, walking 20 feet forward in the real environment may be equivalent to walking 20 virtual feet in the virtual environment. When the user views the visualization component, it may appear to be 20 feet away in the virtual environment. The distance associated with the first view may also be measured between the predefined 3D shape and the focus or focal plane of the user's virtual camera. For example, instead of moving forward to approach the predefined 3D shape, the user may zoom in on its display on the predefined 3D shape without moving the position of the camera in the virtual environment or the position of their viewing device in the real environment.

[0130] The method may additionally include providing a display (1712) of a first view of a predefined 3D shape. As described in detail above, with respect to one or more 3D constructs of a structure representing a predefined 3D shape, 3D objects representing individual data points may be visually emphasized or de-emphasized. In the example of a funnel object, a 3D construct including a conical ring or other surface / shell may be added to the virtual environment to clearly define the shape of the funnel object. As the distance associated with the first view increases, the 3D objects representing individual data points may be de-emphasized, and the 3D constructs of the structure representing the predefined 3D shape may be emphasized. Such emphasis may include adjusting the transparency, color, brightness, size, and / or other visual characteristics of these objects. As the distance associated with the first view decreases, the 3D objects representing individual data points may be emphasized relative to the 3D constructs. For example, as the distance decreases, the 3D constructs may become more transparent. The size of the 3D objects representing individual data points may increase, and various colors indicating the attributes of the data points may be displayed. Some embodiments may use thresholds as described above to determine when to emphasize some objects relative to other objects.

[0131] It should be recognized that Figure 17 the specific steps shown in provide a particular method of generating a view of a visualization object according to various embodiments. According to alternative embodiments, other sequences of steps may also be performed. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Moreover, Figure 17 each of the steps shown in may include multiple sub-steps, which may be performed in various orders suitable for each step. Additionally, depending on the particular application, additional steps may be added or removed. Those of ordinary skill in the art will recognize many variations, modifications, and alternatives.

[0132] Each method described herein can be implemented by a computer system. Each step of these methods can be automatically performed by the computer system, and / or can involve input / output involving a user. For example, a user can provide input for each step in the method, and each of these inputs can be in response to a specific output that requests such input, where the output is generated by the computer system. Each input can be received in response to the corresponding requested output. In addition, the input can be received from a user, received as a data stream from another computer system, retrieved from a memory location, retrieved over a network, requested from a web service, etc. Similarly, the output can be provided to a user, provided as a data stream to another computer system, saved in memory, sent over a network, provided to a web service, etc. In short, each step of the methods described herein can be performed by a computer system and can involve any number of inputs, outputs, and / or requests between the computer systems that may or may not involve a user. Those steps that can be said to not involve a user are automatically performed by the computer system without human intervention. Thus, it will be understood from the present disclosure that each step of each method described herein can be changed to include inputs and outputs to and from a user, or can be automatically completed by the computer system without human intervention, where any determination is made by a processor. In addition, some embodiments of each method described herein can be implemented as an instruction set stored on a tangible non-transitory storage medium to form a tangible software product.

[0133] Figure 18 A simplified diagram of a distributed system 1800 for implementing one of the embodiments is depicted. In the illustrated embodiment, the distributed system 1800 includes one or more client computing devices 1802, 1804, 1806, and 1808, which are configured to execute and operate client applications, such as web browsers, proprietary clients (e.g., Oracle Forms), etc., over one or more networks 1810. A server 1812 can be communicatively coupled to the remote client computing devices 1802, 1804, 1806, and 1808 via the network 1810.

[0134] In various embodiments, the server 1812 can be adapted to run one or more services or software applications provided by one or more components of the system. In some embodiments, these services can be supplied to users of the client computing devices 1802, 1804, 1806, and / or 1808 as web-based services or cloud services or under a software as a service (SaaS) model. The users operating the client computing devices 1802, 1804, 1806, and / or 1808 can then utilize one or more client applications to interact with the server 1812 to utilize the services provided by these components.

[0135] In the configuration depicted in the figure, the software components 1818, 1820, and 1822 of system 1800 are shown as being implemented on server 1812. In other embodiments, one or more components of system 1800 and / or the services provided by these components may also be implemented by one or more of client computing devices 1802, 1804, 1806, and / or 1808. A user operating a client computing device can then utilize one or more client applications to use the services provided by these components. These components may be implemented in hardware, firmware, software, or a combination thereof. It should be recognized that a variety of different system configurations are possible, which may differ from distributed system 1800. The embodiment shown in the figure is thus an example of a distributed system for implementing an embodiment system and is not intended to be limiting.

[0136] Client computing devices 1802, 1804, 1806, and / or 1808 can be portable handheld devices (e.g., cellular phones, computing tablets, personal digital assistants (PDAs)), or wearable devices (e.g., Google head-mounted displays), running software such as Microsoft Windows and / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, etc.), and enabling Internet, email, short message service (SMS), or other communication protocols. The client computing device can be a general-purpose personal computer, which, by way of example, includes personal computers and / or laptop computers running various versions of Microsoft Apple and / or Linux operating systems. The client computing device can be a workstation computer running any operating system in a variety of commercially available or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, such as, for example, Google Chrome OS). Alternatively or additionally, client computing devices 1802, 1804, 1806, and 1808 can be any other electronic device capable of communicating over network(s) 1810, such as a thin client computer, an Internet-enabled gaming system (e.g., a Microsoft Xbox gaming console with or without a gesture input device) and / or a personal messaging device.

[0137] Although an exemplary distributed system 1800 with four client computing devices is shown, any number of client computing devices can be supported. Other devices (such as devices with sensors, etc.) can interact with the server 1812.

[0138] The network(s) 1810 in the distributed system 1800 can be any type of network familiar to those skilled in the art that can support data communication using any of a variety of commercially available protocols, where the protocols include but are not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), IPX (Internetwork Packet Exchange), AppleTalk, etc. Merely by way of example, the network(s) 1810 can be a local area network (LAN), such as a LAN based on Ethernet, Token Ring, etc. The network(s) 1810 can be a wide area network and the Internet. It can include virtual networks, including but not limited to virtual private networks (VPNs), intranets, extranets, public switched telephone networks (PSTNs), infrared networks, wireless networks (e.g., networks operating according to any protocol of the Institute of Electrical and Electronics Engineers (IEEE) 802.11 protocol suite, and / or any other wireless protocol); and / or any combination of these networks and / or other networks.

[0139] The server 1812 can be composed of one or more general-purpose computers, dedicated server computers (by way of example, including PC (personal computer) servers, servers, midrange servers, mainframe computers, rack-mounted servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination. In various embodiments, the server 1812 can be adapted to run one or more services or software applications described in the foregoing disclosure. For example, the server 1812 can correspond to a server for performing the processing described above according to embodiments of the present disclosure.

[0140] The server 1812 can run an operating system including any of the operating systems discussed above, as well as any commercially available server operating system. The server 1812 can also run any of a variety of additional server applications and / or middleware applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, servers, database servers, etc. Exemplary database servers include but are not limited to those commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), etc.

[0141] In some embodiments, server 1812 may include one or more applications to analyze and integrate data feeds and / or event updates received from users of client computing devices 1802, 1804, 1806, and 1808. As an example, the data feeds and / or event updates may include, but are not limited to: feeds, updates, or real-time updates received from one or more third-party information sources and continuous data streams, which may include real-time events related to sensor data applications, financial tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, etc. Server 1812 may also include one or more applications to display data feeds and / or real-time events via one or more display devices of client computing devices 1802, 1804, 1806, and 1808.

[0142] Distributed system 1800 may also include one or more databases 1814 and 1816. Databases 1814 and 1816 may reside in various locations. As an example, one or more of databases 1814 and 1816 may reside on a non-transitory storage medium local to (and / or within) server 1812. Alternatively, databases 1814 and 1816 may be remote from server 1812 and communicate with server 1812 via a network-based connection or a dedicated connection. In one set of examples, databases 1814 and 1816 may reside in a storage area network (SAN). Similarly, any necessary files for performing the functions of server 1812 may be stored appropriately locally on server 1812 and / or remotely. In one set of embodiments, databases 1814 and 1816 may include relational databases adapted to store, update, and retrieve data in response to SQL-formatted commands, such as databases provided by Oracle.

[0143] Figure 19 is a simplified block diagram of one or more components of a system environment 1900 that, in accordance with some embodiments of the present disclosure, may provide services provided by one or more components of an embodiment system as cloud services. In the illustrated embodiment, system environment 1900 includes one or more client devices 1904, 1906, and 1908 that may be used by a user to interact with a cloud infrastructure system 1902 that provides cloud services. The client computing device may be configured to operate client applications, such as a web browser, a proprietary client application (e.g., Oracle Forms), or some other application, which may be used by a user of the client computing device to interact with cloud infrastructure system 1902 to use the services provided by cloud infrastructure system 1902.

[0144] It should be recognized that the cloud infrastructure system 1902 depicted in the figure may have other components different from the depicted components. Additionally, the embodiments shown in the figure are merely one example of a cloud infrastructure system that may incorporate embodiments of the present invention. In some other embodiments, the cloud infrastructure system 1902 may have more or fewer components than those shown in the figure, two or more components may be combined, or it may have different configurations or arrangements of components.

[0145] The client devices 1904, 1906, and 1908 may be devices similar to those described above for 1802, 1804, 1806, and 1808.

[0146] Although the exemplary system environment 1900 is shown having three client computing devices, any number of client computing devices may be supported. Other devices (such as devices with sensors, etc.) may interact with the cloud infrastructure system 1902.

[0147] (One or more) networks 1910 may facilitate the communication and exchange of data between the client devices 1904, 1906, and 1908 and the cloud infrastructure system 1902. Each network may be any type of network familiar to those skilled in the art that can support data communication using any of a variety of commercially available protocols, including those described above for (one or more) networks 1810.

[0148] The cloud infrastructure system 1902 may include one or more computers and / or servers, which may include those computers and / or servers described above for server 1812.

[0149] In certain embodiments, the services provided by the cloud infrastructure system may include many services available on demand to users of the cloud infrastructure system, such as online data storage and backup solutions, Web-based email services, hosted office suites and document collaboration services, database processing, managed technical support services, etc. The services provided by the cloud infrastructure system may be dynamically scaled to meet the needs of the users of the cloud infrastructure system. The specific instantiation of the services provided by the cloud infrastructure system is referred to herein as a "service instance". Generally, any service available to a user from a cloud service provider's system via a communication network (such as the Internet) is referred to as a "cloud service". Typically, in a public cloud environment, the servers and systems that make up the cloud service provider's system are different from the customer's own on-premises servers and systems. For example, the cloud service provider's system may host applications, and users may order and use the applications on demand via a communication network such as the Internet.

[0150] In some examples, services in a computer network cloud infrastructure can include protected computer network access to storage devices, hosted databases, hosted web servers, software applications, or other services provided by a cloud provider to users, or as otherwise known in the art. For example, the service can include password-protected access to a remote storage device on the cloud over the Internet. As another example, the service can include a hosted relational database and a scripting language middleware engine based on web services for private use by networked developers. As another example, the service can include access to an email software application hosted on the cloud provider's website.

[0151] In certain embodiments, the cloud infrastructure system 1902 can include a suite of application, middleware, and database service provisions delivered to customers in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. An example of such a cloud infrastructure system is the Oracle Public Cloud provided by the present assignee.

[0152] In various embodiments, the cloud infrastructure system 1902 can be adapted to automatically provision, manage, and track customer subscriptions to the services provided by the cloud infrastructure system 1902. The cloud infrastructure system 1902 can provide cloud services via different deployment models. For example, services can be provided according to a public cloud model, where the cloud infrastructure system 1902 is owned by an organization selling cloud services (e.g., owned by Oracle), and the services are available to the general public or enterprises in different industries. As another example, services can be provided according to a private cloud model, where the cloud infrastructure system 1902 operates only for a single organization and can provide services to one or more entities within that organization. Cloud services can also be provided according to a community cloud model, where the cloud infrastructure system 1902 and the services provided by the cloud infrastructure system 1902 are shared by several organizations in a related community. Cloud services can also be provided according to a hybrid cloud model, which is a combination of two or more different models.

[0153] In some embodiments, the services provided by the cloud infrastructure system 1902 can include one or more services provided under the software as a service (SaaS) category, platform as a service (PaaS) category, infrastructure as a service (IaaS) category, or other service categories including hybrid services. A customer can order one or more services provided by the cloud infrastructure system 1902 via a subscription order. The cloud infrastructure system 1902 then performs processing to provide the services in the customer's subscription order.

[0154] In some embodiments, the services provided by the cloud infrastructure system 1902 may include, but are not limited to, application services, platform services, and infrastructure services. In some examples, the application services may be provided by the cloud infrastructure system via a SaaS platform. The SaaS platform may be configured to provide cloud services that fall into the SaaS category. For example, the SaaS platform may provide the ability to build and deliver an on-demand application suite on an integrated development and deployment platform. The SaaS platform may manage and control the underlying software and infrastructure for providing the SaaS services. By leveraging the services provided by the SaaS platform, customers can utilize applications executed on the cloud infrastructure system. Customers can obtain application services without the customers having to purchase separate licenses and support. A variety of different SaaS services may be provided. Examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business agility for large organizations.

[0155] In some embodiments, the platform services may be provided by the cloud infrastructure system via a PaaS platform. The PaaS platform may be configured to provide cloud services that fall into the PaaS category. Examples of platform services may include, but are not limited to, services that enable an organization (such as Oracle) to integrate existing applications on a shared common architecture and take advantage of the shared services provided by the platform to build new applications. The PaaS platform may manage and control the underlying software and infrastructure for providing the PaaS services. Customers can obtain the PaaS services provided by the cloud infrastructure system without the customers having to purchase separate licenses and support. Examples of platform services include, but are not limited to, Oracle Java Cloud Service (JCS), Oracle Database Cloud Service (DBCS), etc.

[0156] By leveraging the services provided by the PaaS platform, customers can adopt programming languages and tools supported by the cloud infrastructure system and also control the deployed services. In some embodiments, the platform services provided by the cloud infrastructure system may include database cloud services, middleware cloud services (e.g., Oracle Fusion Middleware services), and Java cloud services. In one embodiment, the database cloud services may support a shared service deployment model that enables an organization to pool database resources and supply database as a service to customers in the form of a database cloud. The middleware cloud services may provide a platform for customers to develop and deploy various business applications, and the Java cloud services may provide a platform for customers to deploy Java applications in the cloud infrastructure system.

[0157] A variety of different infrastructure services may be provided by the IaaS platform in the cloud infrastructure system. The infrastructure services facilitate the management and control of underlying computing resources (such as storage devices, networks, and other basic computing resources) for customers to utilize the services provided by the SaaS platform and the PaaS platform.

[0158] In some embodiments, the cloud infrastructure system 1902 may further include infrastructure resources 1930 for providing resources to customers of the cloud infrastructure system for providing various services. In one embodiment, the infrastructure resources 1930 may include a combination of pre-integrated and optimized hardware (such as servers, storage devices, and networking resources) to execute the services provided by the PaaS platform and the SaaS platform.

[0159] In some embodiments, the resources in the cloud infrastructure system 1902 may be shared by multiple users and dynamically reallocated as needed. In addition, resources can be allocated to users in different time zones. For example, the cloud infrastructure system 1902 may enable a first group of users in a first time zone to utilize the resources of the cloud infrastructure system within a specified number of hours and then enable the same resources to be reallocated to another group of users located in a different time zone, thereby maximizing the utilization of the resources.

[0160] In certain embodiments, a plurality of internal shared services 1932 may be provided that are shared by different components or modules of the cloud infrastructure system 1902 and by the services provided by the cloud infrastructure system 1902. These internal shared services may include, but are not limited to: security and identity services, integration services, enterprise repository services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, cloud-enabled support services, email services, notification services, file transfer services, etc.

[0161] In certain embodiments, the cloud infrastructure system 1902 may provide comprehensive management of the cloud services (such as SaaS, PaaS, and IaaS services) in the cloud infrastructure system. In one embodiment, the cloud management function may include the ability to provision, manage, and track customer subscriptions received by the cloud infrastructure system 1902, etc.

[0162] In one embodiment, as depicted in the figure, the cloud management function may be provided by one or more modules, such as an order management module 1920, an order orchestration module 1922, an order provisioning module 1924, an order management and monitoring module 1926, and an identity management module 1928. These modules may include one or more computers and / or servers or use one or more computers and / or servers to provide, and these computers and / or servers may be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement / or combination.

[0163] In exemplary operation 1934, a customer using a client device such as client devices 1904, 1906, or 1908 can interact with the cloud infrastructure system 1902 by requesting one or more services provided by the cloud infrastructure system 1902 and placing an order to subscribe to one or more services offered by the cloud infrastructure system 1902. In some examples, the customer can access the cloud user interface (UI), cloud UI 1912, cloud UI 1914, and / or cloud UI 1916 and place an order via these UIs. The order information received by the cloud infrastructure system 1902 in response to the customer placing an order can include information identifying the customer and the one or more services offered by the cloud infrastructure system 1902 that the customer wishes to subscribe to.

[0164] After the customer has placed an order, order information is received via cloud UIs 1912, 1914, and / or 1916.

[0165] At operation 1936, the order is stored in the order database 1918. The order database 1918 can be one of several databases operated by and operating with other system components of the cloud infrastructure system 1918.

[0166] At operation 1938, the order information is forwarded to the order management module 1920. In some cases, the order management module 1920 can be configured to perform billing and accounting functions related to the order, such as validating the order and, upon validation, booking the order.

[0167] At operation 1940, information about the order is sent to the order orchestration module 1922. The order orchestration module 1922 can utilize the order information to orchestrate the provisioning of services and resources for the order placed by the customer. In some cases, the order orchestration module 1922 can use the services of the order provisioning module 1924 to orchestrate the provisioning of resources to support the subscribed services.

[0168] In certain embodiments, the order orchestration module 1922 enables the management of the business processes associated with each order and applies business logic to determine whether the order should proceed to provisioning. At operation 1942, upon receiving a newly subscribed order, the order orchestration module 1922 sends a request to the order provisioning module 1924 to allocate resources and configure those resources required to fulfill the subscribed order. The order provisioning module 1924 enables the allocation of resources for the services ordered by the customer. The order provisioning module 1924 provides an abstraction layer between the cloud services offered by the cloud infrastructure system 1902 and the physical implementation layer for provisioning the resources used to provide the requested services. Thus, the order orchestration module 1922 can be isolated from implementation details, such as whether the services and resources are actually provisioned immediately or pre-provisioned and only allocated / assigned upon request.

[0169] At operation 1944, once the services and resources are provided, a notification of the provided services can be sent to the customers on client devices 1904, 1906, and / or 1908 via the order provisioning module 1924 of the cloud infrastructure system 1902.

[0170] At operation 1946, the order management and monitoring module 1926 can manage and track the subscription orders of the customers. In some cases, the order management and monitoring module 1926 can be configured to collect usage statistics of the services in the subscription orders, such as the amount of storage used, the amount of data transferred, the number of users, and the amount of system uptime and system downtime.

[0171] In certain embodiments, the cloud infrastructure system 1902 can include an identity management module 1928. The identity management module 1928 can be configured to provide identity services, such as access management and authorization services in the cloud infrastructure system 1902. In some embodiments, the identity management module 1928 can control information about customers who wish to utilize the services provided by the cloud infrastructure system 1902. Such information can include information for authenticating the identities of these customers and information describing which actions these customers are authorized to perform with respect to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). The identity management module 1928 can also include management of the descriptive information about each customer and about how and by whom this descriptive information can be accessed and modified.

[0172] Figure 20 An exemplary computer system 2000 in which various embodiments of the present invention can be implemented is illustrated. The system 2000 can be used to implement any of the computer systems described above. As shown, the computer system 2000 includes a processing unit 2004 that communicates with a plurality of peripheral subsystems via a bus subsystem 2002. These peripheral subsystems can include a processing acceleration unit 2006, an I / O subsystem 2008, a storage subsystem 2018, and a communication subsystem 2024. The storage subsystem 2018 includes a tangible computer-readable storage medium 2022 and a system memory 2010.

[0173] The bus subsystem 2002 provides a mechanism for enabling the various components and subsystems of the computer system 2000 to communicate with each other as intended. Although the bus subsystem 2002 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple buses. The bus subsystem 2002 can be any type of bus in several types of bus architectures, including a memory bus or memory controller, a peripheral bus, and a local bus using any architecture in various bus architectures. For example, such an architecture can include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus, which can be implemented as a Mezzanine bus manufactured according to the IEEE P1386.1 standard.

[0174] The processing unit 2004, which can be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller), controls the operation of the computer system 2000. One or more processors can be included in the processing unit 2004. These processors can include a single-core processor or a multi-core processor. In certain embodiments, the processing unit 2004 can be implemented as one or more independent processing units 2032 and / or 2034, where a single-core processor or a multi-core processor is included in each processing unit. In other embodiments, the processing unit 2004 can also be implemented as a quad-core processing unit formed by integrating two dual-core processors into a single chip.

[0175] In various embodiments, the processing unit 2004 can execute various programs in response to program code and can maintain multiple concurrently executing programs or processes. At any given time, some or all of the program code to be executed can reside in the processing unit 2004 and / or the storage subsystem 2018. Through appropriate programming, the processing unit 2004 can provide the various functions described above. The computer system 2000 can additionally include a processing acceleration unit 2006, which can include a Digital Signal Processor (DSP), a dedicated processor, etc.

[0176] The I / O subsystem 2008 can include user interface input devices and user interface output devices. User interface input devices can include a keyboard, a pointing device such as a mouse or trackball, a touchpad or touch screen incorporated into a display, a scroll wheel, a click wheel, a dial, a button, a switch, a keypad, an audio input device with a voice command recognition system, a microphone, and other types of input devices. User interface input devices can include, for example, motion sensing and / or gesture recognition devices, such as Microsoft A motion sensor that enables a user to control and interact with an input device, such as a Microsoft 360 game controller, through a natural user interface using gestures and voice commands. The user interface input device may also include an eye gesture recognition device, such as detecting eye activity from the user (e.g., "blinking" when taking a photo and / or making a menu selection) and transforming the eye gesture into an input into the input device (e.g., Google )'s Google blink detector. In addition, the user interface input device may include a voice recognition sensing device that enables the user to interact with a voice recognition system (e.g., navigator) through voice commands.

[0177] The user interface input device may also include, but is not limited to: a three-dimensional (3D) mouse, a joystick or a pointing stick, a game panel and a graphics tablet, and audio / visual devices, such as speakers, digital cameras, digital video recorders, portable media players, webcams, image scanners, fingerprint scanners, barcode readers, 3D scanners, 3D printers, laser rangefinders, and eye tracking devices. In addition, the user interface input device may include, for example, medical imaging input devices, such as computed tomography, magnetic resonance imaging, positron emission tomography, and medical ultrasound devices. The user interface input device may also include, for example, audio input devices (such as MIDI keyboards, digital musical instruments, etc.).

[0178] The user interface output device may include a display subsystem, an indicator light, or a non-visual display such as an audio output device. The display subsystem may be a cathode ray tube (CRT), a flat panel device such as using a liquid crystal display (LCD) or a plasma display, a projection device, a touch screen, etc. Generally, the use of the term "output device" is intended to include all possible types of devices and mechanisms for outputting information from the computer system 2000 to a user or other computers. For example, the user interface output device may include, but is not limited to: various display devices that visually convey text, graphics, and audio / video information, such as monitors, printers, speakers, headphones, car navigation systems, plotters, voice output devices, and modems.

[0179] The computer system 2000 may include a storage subsystem 2018 shown as currently located within the system memory 2010 and containing software elements. The system memory 2010 may store program instructions that are loadable and executable on the processing unit 2004, as well as data generated during the execution of these programs.

[0180] Depending on the configuration and type of the computer system 2000, the system memory 2010 can be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). RAM typically contains data and / or program modules that can be immediately accessed by the processing unit 2004 and / or are currently being operated on and executed by the processing unit 2004. In some embodiments, the system memory 2010 can include various different types of memory, such as static random access memory (SRAM) or dynamic random access memory (DRAM). In some embodiments, such as during startup, the basic input / output system (BIOS), which contains basic routines that help transfer information between components within the computer system 2000, can typically be stored in the ROM. By way of example and not limitation, the system memory 2010 is also shown to include application programs 2012, which can include client applications, web browsers, middleware applications, relational database management systems (RDBMS), etc., program data 2014, and an operating system 2016. By way of example, the operating system 2016 can include various versions of Microsoft Apple and / or Linux operating systems, various commercially available or UNIX-like operating systems (including but not limited to various GNU / Linux operating systems, Google OS, etc.) and / or mobile operating systems such as iOS, Phone, OS, 10OS, and OS operating systems.

[0181] The storage subsystem 2018 can also provide a tangible computer-readable storage medium for storing the basic programming and data structures that provide the functionality of some embodiments. Software (programs, code modules, instructions) that provides the above functionality when executed by a processor can be stored in the storage subsystem 2018. These software modules or instructions can be executed by the processing unit 2004. The storage subsystem 2018 can also provide a repository for storing data used in accordance with the present invention.

[0182] The storage subsystem 2000 can also include a computer-readable storage medium reader 2020 that can be further connected to a computer-readable storage medium 2022. Together with and optionally in combination with the system memory 2010, the computer-readable storage medium 2022 can comprehensively represent remote, local, fixed, and / or removable storage devices plus storage media for temporarily and / or more persistently containing, storing, sending, and retrieving computer-readable information.

[0183] A computer-readable storage medium 2022 that contains code or a portion of code may also include any suitable medium known or used in the art, including storage media and communication media, such as but not limited to: volatile and non-volatile, removable and non-removable media implemented by any method or technology for the storage and / or transmission of information. This may include tangible computer-readable storage media such as RAM, ROM, electrically erasable programmable ROM (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, magnetic tape cartridges, tapes, magnetic disk storage devices or other magnetic storage devices, or other tangible computer-readable media. This may also include non-tangible computer-readable media such as data signals, data transmissions, or any other medium that can be used to send desired information and can be accessed by computing system 2000.

[0184] As an example, computer-readable storage medium 2022 may include a hard disk drive that reads from or writes to an immovable non-volatile magnetic medium, a disk drive that reads from or writes to a removable non-volatile disk, and an optical disk drive that reads from or writes to a removable non-volatile optical disk (such as a CD ROM, DVD, and (Blu-ray) disk or other optical medium). Computer-readable storage medium 2022 may include but is not limited to: drives, flash cards, universal serial bus (USB) flash drives, secure digital (SD) cards, DVD disks, digital audio tapes, etc. Computer-readable storage medium 2022 may also include: solid-state drives (SSDs) based on non-volatile memory (such as flash memory-based SSDs, enterprise flash drives, solid-state ROMs, etc.), SSDs based on volatile memory (such as solid-state RAM, dynamic RAM, static RAM, DRAM-based SSDs, magnetoresistive RAM (MRAM) SSDs), and hybrid SSDs that use a combination of DRAM and flash memory-based SSDs. Disk drives and their associated computer-readable media may provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for computing system 2000.

[0185] The communication subsystem 2024 provides an interface to other computer systems and networks. The communication subsystem 2024 serves as an interface for receiving data from other systems and sending data from the computer system 2000 to other systems. For example, the communication subsystem 2024 may enable the computer system 2000 to connect to one or more devices via the Internet. In some embodiments, the communication subsystem 2024 may include radio frequency (RF) transceiver components for accessing wireless voice and / or data networks (e.g., using cellular phone technology, advanced data network technologies such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), WiFi (IEEE 802.11 series standards), or other mobile communication technologies, or any combination thereof), a Global Positioning System (GPS) receiver component, and / or other components. In some embodiments, as an addition or alternative to the wireless interface, the communication subsystem 2024 may provide a wired network connection (e.g., Ethernet).

[0186] In some embodiments, the communication subsystem 2024 may also receive input communications in the form of structured and / or unstructured data feeds 2026, event streams 2028, event updates 2030, etc. on behalf of one or more users who may use the computer system 2000.

[0187] As an example, the communication subsystem 2024 may be configured to receive data feeds 2026 from users of social networks and / or other communication services in real time, such as feeds, updates, web feeds such as Rich Site Summary (RSS) feeds, and / or real-time updates from one or more third-party information sources.

[0188] In addition, the communication subsystem 2024 may also be configured to receive data in the form of a continuous data stream, which may include event streams 2028 and / or event updates 2030 of real-time events that may be essentially continuous or unbounded without a clear termination. Examples of applications that generate continuous data may include, for example, sensor data applications, financial price tickers, network performance measurement tools (e.g., network monitoring and traffic management applications), clickstream analysis tools, automotive traffic monitoring, etc.

[0189] The communication subsystem 2024 may also be configured to output structured and / or unstructured data feeds 2026, event streams 2028, event updates 2030, etc. to one or more databases, which may communicate with one or more streaming data source computers coupled to the computer system 2000.

[0190] The computer system 2000 may be one of various types, including handheld portable devices (e.g., cellular phones, computing tablets, PDAs), wearable devices (e.g., Google head-mounted displays), PCs, workstations, mainframes, kiosks, server racks, or any other data processing system.

[0191] Due to the ever-changing nature of computers and networks, the description of the computer system 2000 depicted in the figure is only intended to be a specific example. Many other configurations with more or fewer components than the system depicted in the figure are possible. For example, custom hardware can also be used and / or specific elements can be implemented in hardware, firmware, software (including applets), or combinations thereof. Additionally, connections to other computing devices such as network input / output devices can be employed. Based on the disclosure and teachings provided herein, those of ordinary skill in the art will recognize other ways and / or methods of implementing the various embodiments.

[0192] In the foregoing description, for purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the various embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form.

[0193] The foregoing description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the foregoing description of the exemplary embodiments will provide those skilled in the art with an enabling description for implementing the exemplary embodiments. It should be understood that various changes can be made to the function and arrangement of the elements without departing from the spirit and scope of the invention as set forth in the appended claims.

[0194] Specific details were given in the foregoing description to provide a thorough understanding of the embodiments. However, those of ordinary skill in the art will understand that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may have been shown as components in block diagram form so as not to obscure the embodiments with unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may have been shown without unnecessary detail so as not to obscure the embodiments.

[0195] Moreover, it should be noted that each embodiment may have been described as a process depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may have described operations as sequential processes, many operations may be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. The process terminates when the operations are completed, but may have additional steps not included in the figure. The process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When the process corresponds to a function, its termination may correspond to the function returning to the calling function or the main function.

[0196] The term "computer-readable medium" includes, but is not limited to, portable or fixed storage devices, optical storage devices, wireless channels, and various other media capable of storing, containing, or carrying one or more instructions and / or data. A code segment or machine-executable instructions may represent a process, a function, a subroutine, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, a data structure, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0197] In addition, embodiments may be implemented by hardware, software, firmware, middleware, microcode, a hardware description language, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments for performing the necessary tasks may be stored in a machine-readable medium. One or more processors may perform the necessary tasks.

[0198] In the foregoing specification, aspects of the present invention have been described with reference to specific embodiments of aspects of the present invention, but those skilled in the art will recognize that the present invention is not limited thereto. The various features and aspects of the above-described invention may be used alone or in combination. Additionally, embodiments may be used in any number of environments and applications other than those described herein, without departing from the broader spirit and scope of this specification. Accordingly, this specification and the drawings should be regarded as illustrative rather than restrictive.

[0199] In addition, for purposes of illustration, the methods are described in a particular order. It should be recognized that in alternative examples, the methods may be performed in an order different from that described. It should also be recognized that the above methods may be performed by hardware components or may be implemented as a sequence of machine-executable instructions that may be used to cause a machine, such as a general or special purpose processor or logic circuitry programmed with the instructions, to perform the methods. These machine-executable instructions may be stored on one or more machine-readable media, such as a CD-ROM or other type of optical disc, a floppy disc, a ROM, a RAM, an EPROM, an EEPROM, a magnetic or optical card, a flash memory, or other type of machine-readable medium suitable for storing electronic instructions. Alternatively, the methods may be performed by a combination of hardware and software.

Claims

1. A method for determining an optimal view of a visualization object, the method comprising: receiving one or more data sets and an indication of a predetermined 3D shape; generating one or more 3D constructs that form the structure of the predetermined 3D shape; generating 3D objects corresponding to data points in the one or more data sets; arranging the 3D objects into the predetermined 3D shape such that the 3D objects also form the predetermined 3D shape; receiving a command indicating a first view of the predetermined 3D shape; determining a distance associated with the first view; and providing a display of the first view of the 3D objects together with the one or more 3D constructs representing the predetermined 3D shape, wherein the 3D objects are visually emphasized relative to the one or more 3D constructs forming the structure of the predetermined 3D shape, at least in part based on the distance associated with the first view.

2. The method according to claim 1, wherein the command indicating the first view comprises a plain English voice command.

3. The method according to claim 2, further comprising converting the plain English voice command into a predetermined command among a plurality of predetermined commands.

4. The method according to claim 3, further comprising providing the predetermined command to one or more neural networks.

5. The method according to claim 4, wherein providing a display of the first view of the predetermined 3D shape uses a view position of a virtual camera determined by an output of the one or more neural networks.

6. The method according to claim 4, wherein the one or more data sets are selected based on an output of the one or more neural networks.

7. The method according to claim 4, wherein the 3D objects and the predetermined 3D shape are selected based on an output of the one or more neural networks.

8. The method according to claim 1, wherein the first view position comprises a view that fills approximately 90% of the view area with the predetermined 3D shape.

9. The method according to claim 1, further comprising: determining that the distance associated with the first view is greater than a first threshold distance, and emphasizing the one or more 3D constructs while not emphasizing the 3D objects; and determining that the first view has been scaled to a second view position that is lower than the first threshold but higher than a second threshold, and gradually de - emphasizing the predetermined 3D shape while gradually emphasizing the 3D objects.

10. The method according to claim 1, wherein, in the display of the first view, the 3D objects are visually more emphasized relative to the one or more 3D constructs; and the method further comprises: receiving a command indicating a second view of the predetermined 3D shape; determining a distance associated with the second view, wherein the distance associated with the second view is greater than the distance associated with the first view; providing a display of the second view of the 3D objects together with the one or more 3D constructs representing the predetermined 3D shape, wherein at least in part based on the distance associated with the second view, the one or more 3D constructs representing the structure of the predetermined 3D shape are visually more emphasized relative to the 3D objects.

11. The method according to claim 10, wherein the number of the one or more 3D constructs representing the predetermined 3D shape is less than the number of the 3D objects, and wherein the one or more 3D constructs include geometric shapes of a different type from the 3D objects.

12. The method according to claim 1, wherein the 3D object is visually emphasized relative to the one or more 3D constructs by changing the transparency of the 3D object relative to the transparency of the one or more 3D constructs.

13. A non-transitory computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform operations including the following: Receiving one or more data sets and an indication of a predetermined 3D shape; Generating one or more 3D constructs forming the structure of the predetermined 3D shape; Generating 3D objects corresponding to data points in the one or more data sets; Arranging the 3D objects in the predetermined 3D shape such that the 3D objects also form the predetermined 3D shape; Receiving a command indicating a first view of the predetermined 3D shape; Determining a distance associated with the first view; And Providing a display of the first view of the 3D objects together with the one or more 3D constructs representing the predetermined 3D shape, wherein the 3D objects are visually emphasized relative to the one or more 3D constructs forming the structure of the predetermined 3D shape at least in part based on the distance associated with the first view.

14. The non-transitory computer-readable medium according to claim 13, wherein the operations further Include: Determining that the distance associated with the first view is greater than a first threshold distance and emphasizing the one or more 3D constructs without emphasizing the 3D objects.

15. The non-transitory computer-readable medium according to claim 14, wherein the operations further Include: Determining that the first view has been scaled to a second view position below a second threshold but above the first threshold and gradually de-emphasizing the predetermined 3D shape while gradually emphasizing the 3D objects.

16. The non-transitory computer-readable medium according to claim 15, wherein the operations further Include: Determining that the distance associated with the first view is less than a second threshold distance and emphasizing the 3D objects without emphasizing the one or more 3D constructs.

17. A system for determining an optimal view of a visualization object, the system Includes: One or more processors; And One or more memory devices including instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including the following: Receiving one or more data sets and an indication of a predetermined 3D shape; Generating one or more 3D constructs forming the structure of the predetermined 3D shape; Generating 3D objects corresponding to data points in the one or more data sets; Arranging the 3D objects in the predetermined 3D shape such that the 3D objects also form the predetermined 3D shape; Receiving a command indicating a first view of the predetermined 3D shape; Determining a distance associated with the first view; And Provide a display of a first view of the 3D object together with the one or more 3D constructs representing the predetermined 3D shape, wherein the 3D object is visually emphasized relative to the one or more 3D constructs of the structure forming the predetermined 3D shape, at least in part based on a distance associated with the first view.

18. The system according to claim 17, wherein the operation further comprises: Determine that the distance associated with the first view is greater than a first threshold distance, and emphasize the one or more 3D constructs without emphasizing the 3D object.

19. The system according to claim 18, wherein the operation further comprises: Determine that the first view has been scaled to a second view position that is lower than a second threshold but higher than the first threshold, and gradually de-emphasize the predetermined 3D shape while gradually emphasizing the 3D object.

20. The system according to claim 19, wherein the operation further comprises: Determine that the distance associated with the first view is less than a second threshold distance, and emphasize the 3D object without emphasizing the one or more 3D constructs.

21. A computer program product comprising instructions that, when executed by one or more processors of a computer, cause the computer to perform the method according to any one of claims 1-12.

Images