Method for displaying real-time data of three-dimensional relation graph based on augmented reality
By creating three-dimensional relationship maps in augmented reality technology and updating data in real time, the problem that traditional two-dimensional maps cannot display complex three-dimensional relationships and real-time updates is solved, and an intuitive and flexible data display method is achieved.
Patent Information
- Application Number
- CN202311792201.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-25
- Publication Date
- 2025-06-27
AI Technical Summary
Traditional two-dimensional relationship maps cannot visually display complex three-dimensional relationships and cannot update data in real time, limiting their application in real-time monitoring and analysis.
Using an augmented reality technology method, by processing the initial data into JSON data format, we create three-dimensional spatial scenes and augmented reality scenes, render nodes and edges, form a three-dimensional relationship map, and update the data in real time to reflect changes in the database.
It realizes efficient and intuitive three-dimensional relationship map display, supports multiple node formats, can update data in real time, and improves user experience and operation flexibility.
Smart Images
Figure CN120219666A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of augmented reality three-dimensional atlases, and particularly relates to a method for realizing real-time data display of three-dimensional relationship atlases by using augmented reality technology. Background Art
[0002] In modern society, data analysis and visualization have become an indispensable part of all walks of life. In data analysis, relationship atlases are a common way of data visualization, which can help people better understand the relationships between data. The traditional way of displaying relationship atlases mainly uses two-dimensional plane graphics and cannot intuitively display complex three-dimensional relationships. At the same time, the traditional atlas display method cannot update data in real time, which limits its application in real-time monitoring and analysis.
[0003] With the wide application of augmented reality (AR) technology in various fields, there are still some challenges in displaying real-time data of three-dimensional relationship atlases. At present, there have been some related researches and applications, but these methods have some problems, such as poor scalability.
[0004] In view of the problems described in the above background, the present invention provides an efficient, novel and real-time method for displaying three-dimensional relationship atlases, which can be widely applied to fields such as data analysis and education. Summary of the Invention
[0005] The present invention provides a method for displaying real-time data of three-dimensional relationship atlases based on augmented reality.
[0006] The present invention adopts the following technical solutions: A method for displaying real-time data of three-dimensional relationship atlases based on augmented reality, comprising the following steps:
[0007] Using the convertInitJsonData method to process and optimize the initial data into a JSON data format containing a node set nodes and an edge set links;
[0008] Creating a three-dimensional space scene and an augmented reality scene;
[0009] Rendering the node type nodeType of the JSON data format node set nodes according to the unique ID of each node bound to the node;
[0010] Rendering into line segments in the three-dimensional space scene according to the start node ID and the target node ID in the edge set links in the JSON data format to associate the two nodes;
[0011] Based on the node set nodes and edge set links in the JSON data format, including the start node, target node, and the relationship between the start node and the target node, a three-dimensional relationship graph is formed;
[0012] Set the position coordinates of the three-dimensional relationship graph, add it to the augmented reality scene, and render the scene;
[0013] By listening to changes in the data source in the database, when the data changes, the new changed data is pushed; the augmented reality scene processes and matches the received new data, and updates the display of the three-dimensional relationship graph.
[0014] The convertInitJsonData method includes the following steps:
[0015] Parse the input JSON data and extract the key information; the key information includes the types, sizes, and colors of the nodes nodes and edges links;
[0016] According to the key information, reconstruct a JSON data structure containing the node set nodes and edge set links; each node object contains a unique identifier ID and a data object, and the data object contains at least one attribute among the name, color, and size of the node; each link object contains the connection information between two nodes, including the start node ID, target node ID, and direction; form a JSON data format containing the node set nodes and edge set links as the output result.
[0017] For the rendering of the node type nodeType, nodes with numerical attributes need to be bound with click events; after clicking on a node, a single bar chart is generated according to the data to dynamically display the proportional data information.
[0018] The new changed data includes at least one piece of information among the attributes, positions, states, and numerical values of the nodes, and the attributes, positions, states, and numerical values of the edges.
[0019] A method for displaying real-time data of a three-dimensional relationship graph based on augmented reality further includes creating an interactive interface and adding interactive behaviors, specifically as follows:
[0020] Listen for click events through the buttons in the interface, trigger click events using functions, and execute the operation of actively obtaining requests to update data;
[0021] According to the user's trigger, zoom, rotate, or move the three-dimensional relationship graph interactively, or view information by clicking on nodes or edges.
[0022] The present invention has the following beneficial effects and advantages:
[0023] 1. An extensible data structure is adopted, which supports multiple node formats, including: data nodes, two-dimensional images / icons, three-dimensional models, and supports other information storage objects, and can more intuitively and vividly represent the data relationships in three-dimensional space.
[0024] 2. The present invention uses real-time data to update the three-dimensional relationship map, enabling users to obtain information such as the relationships, states, and values of the latest nodes and edges in a timely manner.
[0025] 3. The augmented reality technology is utilized, and the augmented reality technology and the three-dimensional relationship map can be combined and rendered into the real world. Through interactive operations such as rotation, zooming, moving, selecting nodes, and clicking on nodes to display and hide single bar charts drawn according to real-time data, users can intuitively and immersively feel the relationships between data, enhancing the user experience and operation flexibility.
[0026] 4. An incremental update method is adopted to efficiently and locally update data. The incremental update method can perform local updates according to the changed data parts, thereby improving efficiency and reducing resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 is a functional module diagram;
[0028] Figure 2 is a flowchart of specific implementation steps. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0029] In order to make the purpose, technical solutions, and advantages of the present invention clearer and more definite, actual examples are given here and further described with reference to the accompanying drawings.
[0030] A method for displaying real-time data of a three-dimensional relationship map based on augmented reality includes the following functional modules, as Figure 1 shown:
[0031] The present invention provides a method for displaying real-time data of a three-dimensional relationship map based on augmented reality. The functional modules include: data acquisition and processing, three-dimensional relationship map construction, augmented reality environment setup and application, real-time data display and update, and user interaction and operation.
[0032] Data acquisition and processing:
[0033] Obtain the unprocessed and optimized initial data from the database, and use the convertInitJsonData method to process and optimize the initial data according to the data format required by the three-dimensional relationship graph, and finally generate JSON data in the format including the nodes node set and the links edge set. The node set nodes includes information such as node unique ID, node name, node type, real-time value, etc., and the edge set links includes information such as start node ID and target node ID.
[0034] The database can be stored and managed using a graph database or other graph structure storage methods for subsequent model generation and query, such as: Neo4j graph database, etc.;
[0035] The initial data refers to the data that has not been processed and optimized;
[0036] The data structure includes: the node set nodes representing nodes, which includes information such as node unique ID, node name, node type, real-time value, user data, etc.; the edge set links representing edges, which includes information such as start node ID and target node ID;
[0037] The node types include: nodes representing data, two-dimensional images / icons, three-dimensional models, etc.;
[0038] The user data provides a storage object to store other additional information;
[0039] The processing and optimization refers to processing the obtained data into the data format required by the three-dimensional relationship graph;
[0040] Construction of the three-dimensional relationship graph:
[0041] Based on the processed and optimized data structure including the node set nodes and the edge set links, draw nodes and connections, convert the nodes and edges into 3D objects, and use the 3D graphics library Three.js to create and render the 3D objects. The final presentation form is a three-dimensional relationship graph.
[0042] The node types are extensible, supporting nodes of data, two-dimensional images / icons, and three-dimensional models. Nodes of data need to meet data information such as node shape (sphere, cylinder, cube, cone, irregular geometric body, etc.), node color, node size, node position, node name description, etc.; two-dimensional image / icon nodes need to meet data information such as node image source, node size, node position, node name description, etc.; three-dimensional model nodes need to meet data information such as model type, support (FBX, GLTF, GLB, OBJ), model position, model size, model orientation, model name description, etc.
[0043] The node attributes are scalable, providing a userOtherAttrData object for storing additional user information for extension use, and are processed according to the actual requirements of specific application scenarios.
[0044] For the described data relationship, the information of the edges in the three-dimensional relationship graph includes the starting node ID and the target node ID;
[0045] The described data attributes include the type of the node, the position of the node, the direction of the node, the shape of the node, the color of the node, the size of the node, the node name description, etc.;
[0046] For the construction of the three-dimensional relationship graph, according to the data relationship and attributes, nodes and connections are drawn in three-dimensional space, and the final presentation form is a three-dimensional relationship graph.
[0047] Augmented reality environment construction and application:
[0048] Using AR.js, an augmented reality scene is created, which can be a blank virtual space, or a plane or object in the real world; based on image recognition technology, the camera of the device scans the images in the real scene, and AR.js uses computer vision algorithms to recognize and track these images and uses the sensors of the device (such as gyroscopes and accelerometers) to determine the position and orientation of the device in space. Finally, the constructed three-dimensional relationship graph is rendered into the real scene environment in real time using display devices including smartphones, tablets, etc.
[0049] The described augmented reality technology is a technology that combines virtual information with the real-world scene. In the present invention, AR.js is actually used, which renders the three-dimensional relationship graph into the real scene by using technologies such as computer graphics, sensors, and display devices;
[0050] The described augmented reality scene is a virtual scene developed and built using AR.js technology;
[0051] The described display devices include smartphones, tablets, etc.
[0052] Real-time data display and update:
[0053] Data is requested and obtained in real time through the network, and the changes in the data source are monitored. If any changed data is found, the Websocket technology is used to start pushing and updating the data of the nodes or edges. Through the incremental update method, the data that needs to be updated can be quickly found in the three-dimensional relationship graph, and the expected changes are made to quickly update the three-dimensional relationship graph, including updating the position, attributes, and values of the nodes, adding new nodes or edges, or modifying the existing nodes and edges; otherwise, the original form of the three-dimensional relationship graph is maintained.
[0054] The specific manifestation of the network request method is to utilize two-way real-time communication via Websocket;
[0055] The monitored data source refers to a data source that will actively push updated data information if it changes;
[0056] Updating the three-dimensional relationship graph includes updating the position, attributes, and values of nodes, adding new nodes or edges, or modifying existing nodes and edges.
[0057] User interaction and operations:
[0058] Provide a user interaction interface, including operation button elements, etc., add interaction behaviors, such as rotating, zooming in, moving, selecting nodes, clicking on nodes to display or hide a single bar chart drawn based on real-time data, etc. These functions are implemented by detecting the user's gestures or touch events provided by AR.js, allowing users to operate and control the interface button elements and the three-dimensional relationship graph.
[0059] The user interaction and operations refer to operations such as rotating, zooming in, moving, selecting nodes, clicking on nodes to display or hide a single bar chart drawn based on real-time data;
[0060] The single bar chart is a single bar chart dynamically composed of two cylinders based on node data, used to dynamically display proportional data information.
[0061] A method for displaying real-time data of a three-dimensional relationship graph based on augmented reality includes the following implementation steps, as Figure 2 shown:
[0062] S1: Use the convertInitJsonData method to process and optimize the initial data into a JSON data format containing a nodes node set and a links edge set.
[0063] The convertInitJsonData method accepts an initial JSON data as input. This method will parse the input JSON data, extract the key information therein, which may include the types, sizes, colors, etc. of nodes (nodes) and edges (links). According to these key information, reconstruct a JSON data structure containing a nodes node set and a links edge set. Each node object will contain a unique identifier (ID) and a related data object, and this data object may contain attributes such as the name, color, size, etc. of the node. Each link object contains the connection information between two nodes, such as the starting node ID, target node ID, direction, etc. Finally, it is processed and optimized into a JSON data format containing a nodes node set and a links edge set as the output result.
[0064] S2: Create a Three.js 3D space scene and add a perspective camera, a light source, and a renderer.
[0065] S3: Use AR.js technology to create an augmented reality scene, set the camera parameters, and add an augmented reality scene controller.
[0066] S4: For the JSON data format node set nodes, bind the unique ID of each node according to the node type nodeType for rendering. For example, sphere is rendered as a sphere, cylinder as a cylinder, box as a cube, cone as a cone, dodecahedron as other geometric shapes, image as an image / icon according to the image path, and model as a corresponding format model according to the model path. For instance, use an image node to virtualize and represent a car brand, and a model node to represent a car model. Then, render according to other information in the JSON data format. scale includes the x-axis scaling factor (abscissa), y-axis scaling factor (ordinate), and z-axis scaling factor (top view coordinate) for node rendering, value represents the real-time data value, state represents the real-time data status, etc. Additionally, the userOtherAttrData object stores other additional information such as user data for extension use and is actually processed according to the specific application scenario requirements.
[0067] Among them, nodes with numerical attributes need to be bound with click events. After clicking on a node, a single bar chart is generated based on the data to dynamically display the proportional data information. This bar chart is composed of two cylinders spliced together. The bottom cylinder uses height to represent the value, and the calculation formula is value / 100; the top cylinder also uses height to represent the value, and the calculation formula is 100 - value / 100.
[0068] S5: According to the start node ID and target node ID in the edge set links in the JSON data format, render them as line segments in the 3D space to associate the two nodes. According to the example in S4, the edge can be understood as multiple different vehicle models under the same car brand.
[0069] S6: Combine S4 and S5 to finally present as a 3D relationship graph.
[0070] S7: Set the position coordinates of the 3D relationship graph and add it to the augmented reality scene through the add method.
[0071] S8: Render the scene through the render method.
[0072] S9: After the initial data loading and rendering are completed, introduce WebSocket to monitor changes in the data source in the database. If the data changes, push the changed data. These data changes may involve information such as the attributes, positions, states, and values of nodes or edges. In the scene, process and match the received new data. If it is determined that the matched values have changed, incrementally update the display of the three-dimensional relationship graph without re-rendering all the data, which may involve updating the positions, attributes, and values of nodes, adding new nodes or edges, or modifying existing nodes and edges to reduce the number of update operations and improve efficiency, so as to quickly and efficiently reflect these changes on the AR device and maintain the fluency and real-time nature of the AR experience. If the data does not change, maintain the original form of the three-dimensional relationship graph.
[0073] S10: Create an interactive interface and add interactive behaviors. The buttons in the interface listen for click events, and use the gesture interaction, touch interaction, etc. provided by AR.js to trigger click events and execute the operation of actively obtaining requests to update data; users can zoom in, rotate, or move the three-dimensional relationship graph through gestures, or view more information by clicking on nodes or edges.
[0074] The above has elaborated in detail on the implementation method of this method. Through functions such as data acquisition and processing, three-dimensional relationship graph construction, augmented reality environment construction and application, real-time data display and update, and user interaction and operation, users can observe and understand complex relationship structures in a more intuitive and realistic way, enhance memory and understanding, and improve interactivity by combining the AR display form, enabling users to participate in and control the display content more directly, which fully demonstrates that the present invention provides an intuitive, efficient, novel, and immersive data display method.
Claims
1. A method for displaying real-time data of a three-dimensional relationship graph based on augmented reality, characterized in that, It includes the following steps: Utilize the convertInitJsonData method to process and optimize the initial data into a JSON data format containing a node set nodes and an edge set links; Create a three-dimensional space scene and an augmented reality scene; Render the node set nodes in the JSON data format according to the unique ID bound to each node for the node type nodeType; Render the starting node ID and target node ID in the edge set links in the JSON data format into line segments in the three-dimensional space scene to associate the two nodes; Form a three-dimensional relationship graph according to the starting node, target node in the node set nodes and edge set links in the JSON data format, and the relationship between the starting node and the target node; Set the position coordinates of the three-dimensional relationship graph, add it to the augmented reality scene, and render the scene; By listening to the changes in the data source in the database, when the data changes, push the new changed data; The augmented reality scene processes and matches the received new data and updates the display of the three-dimensional relationship graph.
2. The method for displaying real-time data of a three-dimensional relationship map based on augmented reality according to claim 1, wherein, The convertInitJsonData method includes the following steps: Parse the input JSON data and extract the key information; the key information includes the types, sizes, and colors of the nodes nodes and edges links; According to the key information, reconstruct a JSON data structure containing a node set nodes and an edge set links; each node object contains a unique identifier ID and a data object, and the data object contains at least one attribute among the name, color, and size of the node; each link object contains the connection information between two nodes, including the starting node ID, target node ID, and direction; form a JSON data format containing the node set nodes and the edge set links as the output result.
3. A method for displaying real-time data of a three-dimensional relationship graph based on augmented reality according to claim 1, characterized in that, When rendering the node type nodeType, nodes with numerical attributes need to be bound with click events; After clicking on a node, generate a single bar chart according to the data to dynamically display the proportional data information.
4. A method for displaying real-time data of a three-dimensional relationship graph based on augmented reality according to claim 1, characterized in that, The new changed data includes at least one of the information such as the attributes, positions, states, and numerical values of the nodes, and the attributes, positions, states, and numerical values of the edges.
5. A method for displaying real-time data of a three-dimensional relationship graph based on augmented reality according to claim 1, characterized in that, It also includes creating an interactive interface and adding interactive behaviors, specifically as follows: Listen for click events through the buttons in the interface, trigger click events using functions, and execute the operation of actively obtaining requests to update the data; According to the user's trigger, zoom, rotate, or move the three-dimensional relationship graph interactively, or view information by clicking on nodes or edges.