Operation and maintenance data visual analysis method, system and device fusing AR device and mobile device

Abstract

The invention belongs to the technical field of visualization and human-computer interaction, and provides an operation and maintenance data visual analysis method, system and device fusing AR equipment and mobile equipment, and the method comprises the steps: constructing a multi-dimensional visual analysis framework through the interaction of the AR equipment and the mobile equipment; constructing a visual collaborative view mode based on a multi-dimensional visual analysis framework; performing data operation visualization and user scene visualization in the constructed visual collaborative view mode; the method comprises the following steps: importing data into a multi-dimensional visual analysis framework with a visual collaborative view mode through a visual tool interface, and constructing a dynamic visual framework by creating a combined view with enhanced display. According to the invention, multi-source data integration and enhanced visualization, multi-dimensional analysis and problem positioning can be realized, cooperation and interaction capabilities are improved, three-dimensional data visualization and a complex system are interpreted, and intelligent interaction and dynamic insight are realized.

Description

Technical Field

[0001] The present application belongs to the field of visualization and human-computer interaction technology, and in particular, relates to a method, system and device for visually analyzing operation and maintenance data integrating AR devices and mobile devices. Background Art

[0002] With the rapid development of modern industrial and information systems, the importance of operation and maintenance work has become increasingly prominent. In this context, how to efficiently analyze and process huge amounts of operation and maintenance data has become a key issue. Operation and maintenance data covers a variety of types, such as equipment status and sensor monitoring data. These data are usually presented in tabular form, including but not limited to temperature, humidity, CPU usage, memory occupancy, hard disk storage status, etc. These data are not only diverse and complex in format, but also mostly time series data, making it an urgent challenge to effectively display and analyze these data in terms of real-time monitoring, fault diagnosis, trend prediction, etc.

[0003] The widespread use of mobile devices, with their portability and simple operation, plays an increasingly important role in operation and maintenance data analysis. However, the portability of mobile devices also brings problems such as limited screen space and relatively low interaction accuracy, which brings certain troubles to users in the data analysis process, especially when dealing with large-scale tabular data. At the same time, augmented reality technology (AR) has made significant progress in the field of data visualization, providing users with an immersive three-dimensional visualization experience. Although AR technology has been successful in providing immersion, there are still some challenges in accurate interaction.

[0004] Augmented reality technology can provide users with a more intuitive way of interaction and immersion by superimposing virtual data on the real world, allowing users to explore and analyze data in three-dimensional space. Especially in large-scale data display, AR technology has opened up new possibilities for presenting large and complex tabular data. Through virtual reality (VR) and augmented reality technology, users can more intuitively view the structure, relationship and evolution trend of data in three-dimensional space, breaking the limitations of traditional two-dimensional display. Although theoretical work on the integration of mobile devices and augmented reality devices has achieved some important results in existing background research, previous researchers have deep insights into how to more effectively utilize the characteristics of these two types of devices to enhance the user experience in large-scale data analysis. However, current research still lacks in-depth research on how these theoretical results can be implemented in actual systems.

[0005] The visualization analysis framework that combines augmented reality technology with mobile devices can greatly improve the efficiency of operation and maintenance personnel in analyzing and making decisions on complex data, and reduce the complexity of operation and maintenance work. Operation and maintenance work usually involves monitoring and managing the operating status of various equipment and systems, including industrial production equipment, IT hardware facilities (such as servers, network equipment), and various monitoring systems that rely on sensors. Operation and maintenance data is mainly collected by sensors, mostly in the form of time series data, tabular data or multidimensional data sets. Through the real-time collection and analysis of operation and maintenance data, potential anomalies and failure risks in the system can be quickly discovered.

[0006] However, traditional monitoring methods are limited by the space and interaction mode of display devices and cannot effectively present the correlation and trend of multi-dimensional data, resulting in low efficiency in the collaborative analysis of multiple data sources. It is difficult for operation and maintenance personnel to quickly obtain in-depth insights into the global operating status of the system and specific abnormal points. In addition, most existing data analysis tools focus on two-dimensional display, which makes it difficult to simultaneously display details and contextual data in a limited screen space, further increasing the complexity of operation and maintenance work.

[0007] Therefore, developing a new visual analysis tool that can combine mobile devices and augmented reality technology to improve the efficiency and accuracy of operation and maintenance data analysis has become a technical problem that urgently needs to be solved. Summary of the invention

[0008] In response to the above problems, the present application provides a method, system and device for visual analysis of operation and maintenance data that integrates AR devices and mobile devices.

[0009] According to a first aspect of the present application, a method for visually analyzing operation and maintenance data integrating an AR device and a mobile device is provided, the method comprising:

[0010] Build a multi-dimensional visual analysis framework through the interaction between AR devices and mobile devices;

[0011] Based on the multi-dimensional visual analysis framework, a visual collaborative view mode is constructed;

[0012] Visualize data operations and user scenarios in the constructed visual collaborative view mode;

[0013] The data is imported into a multi-dimensional visualization analysis framework with a visualization collaborative view mode through a visualization tool interface, and a dynamic visualization framework is constructed by creating a combined view with enhanced display.

[0014] Optionally, in the visual analysis method of operation and maintenance data integrating AR devices and mobile devices of the present application, a multi-dimensional visual analysis framework is constructed through the interaction between the augmented reality device and the mobile device, including:

[0015] Track the position and align the AR device and the mobile device in space, and map the position coordinates of the AR device and the mobile device to the AR view by constructing a three-dimensional spatial coordinate system;

[0016] Establish a two-way communication channel between the AR device and the mobile device, manage the connection status and data update requests between the AR device and the mobile device, and send the generated AR virtual scene data to the head-mounted display device of the AR device;

[0017] The 2D view is loaded and displayed on the mobile device, and the 2D view loaded and displayed on the mobile device is associated with the AR device.

[0018] Optionally, in the visual analysis method for operation and maintenance data of the integrated AR device and the mobile device of the present application, the 2D view loaded and displayed on the mobile device is associated on the AR device, including: maintaining communication between the AR device and the mobile device during the interaction process, and associating the content displayed by the AR device with the content displayed by the mobile device according to different interaction modes.

[0019] Optionally, in the visual analysis method of operation and maintenance data integrating AR devices and mobile devices of the present application, a visual collaborative view mode is constructed based on a multi-dimensional visual analysis framework, including:

[0020] Display the detailed view of local data through the mobile device, and synchronously display the overview view of global data through the AR device, and use boundary marks in the overview view to indicate the specific range of the detailed view;

[0021] The data association of different mobile devices is displayed through the AR device, and the data sets of different mobile devices are merged in view, and the merged result is displayed through the 3D view.

[0022] Optionally, in the visual analysis method for operation and maintenance data integrating AR devices and mobile devices of the present application, data operation visualization and user scenario visualization in the constructed visual collaborative view mode include:

[0023] Respond to the user's data filtering selection, update the display range in real time according to the user's data filtering selection, and dynamically feedback and highlight the filtering selection results;

[0024] Displaying a corresponding user view according to the user operation area through the AR device, and setting corresponding viewing permissions for the displayed user view;

[0025] The user's filtering selections and interaction trajectories are dynamically marked through AR devices.

[0026] Optionally, in the visual analysis method for operation and maintenance data of the integrated AR device and the mobile device of the present application, data operation visualization and user scenario visualization are performed in the constructed visual collaborative view mode, and also includes: when multiple users coordinate operations, the corresponding public view is displayed, and the user partitions and user permissions of the corresponding operations are displayed in the public view.

[0027] Optionally, in the visual analysis method for operation and maintenance data of the integrated AR device and the mobile device of the present application, the data is imported into a multidimensional visualization analysis framework with a visualization collaborative view mode through a visualization tool interface, including: creating corresponding visualization tool interfaces for different types of data, using the created visualization tool interface to standardize the data, and importing the standardized data into a multidimensional visualization analysis framework with a visualization collaborative view mode.

[0028] Optionally, in the visual analysis method for operation and maintenance data that integrates AR devices and mobile devices of the present application, a dynamic visualization framework is constructed by creating a combined view for enhanced display, including: loading 2D view data loaded and displayed by the mobile device on the AR device, and displaying context data of the loaded 2D view data, dynamically connecting the data focus on the mobile device with the context data, and dynamically generating an overlaid 3D visualization view.

[0029] According to a second aspect of the present application, a visual analysis system for operation and maintenance data integrating an AR device and a mobile device is provided, the system comprising an analysis server, the analysis server comprising:

[0030] A multi-dimensional visualization analysis framework construction module is used to build a multi-dimensional visualization analysis framework through the interaction between AR devices and mobile devices;

[0031] A visualization collaboration view mode construction module is used to construct a visualization collaboration view mode based on a multi-dimensional visualization analysis framework;

[0032] Data operation and user scenario visualization module, used to visualize data operations and user scenarios in the constructed visualization collaboration view mode;

[0033] A dynamic visualization framework building module is used to import data into a multi-dimensional visualization analysis framework with a visualization collaborative view mode through a visualization tool interface, and to build a dynamic visualization framework by creating a combined view with enhanced display.

[0034] According to a third aspect of the present application, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect of the present application when executing the program.

[0035] The method and system of the present application have the following beneficial technical effects:

[0036] The method and system for visual analysis of operation and maintenance data integrating AR devices and mobile devices of the present application have the following beneficial technical effects:

[0037] 1. Multi-source data integration and enhanced visualization: Supports the collection, processing and visualization enhancement of multi-source operation and maintenance data to achieve efficient integration and intuitive display of data; by integrating operation and maintenance data from different sources and types, it can provide a unified data view, allowing operation and maintenance personnel to fully grasp the operating status of the system from macro to micro.

[0038] 2. Multi-dimensional analysis and problem location: Multi-dimensional analysis and detailed display of operation and maintenance data, supporting functions such as data filtering, correlation analysis, and trend identification, to help operation and maintenance personnel quickly locate potential problems, improve the accuracy and efficiency of problem diagnosis, and thus reduce the response time of system failures.

[0039] 3. Improved collaboration and interaction capabilities: Improve collaboration and interaction capabilities in operation and maintenance scenarios, support multi-user collaboration mode, display personalized views and shared views in the same augmented reality environment, avoid data conflicts through multi-user partitioning functions, and promote the collaboration efficiency of the operation and maintenance team, which not only improves work efficiency, but also enhances communication and information sharing between teams.

[0040] 4. Three-dimensional data visualization and interpretation of complex systems: A complex data interpretation system that supports operation and maintenance scenarios through three-dimensional enhanced display provides three-dimensional data visualization support in an augmented reality environment, associates the abstract features of operation and maintenance data (such as time series trends, performance anomalies) with device mapping in the real physical environment, improves data visibility and depth of understanding, and enables operation and maintenance personnel to understand and analyze complex systems more intuitively.

[0041] 5. Intelligent interaction and dynamic insight: Provide intelligent interaction and dynamic insight for operation and maintenance work, support operation and maintenance personnel to quickly switch data dimensions, adjust view parameters, and dynamically update visualization content through touch or gesture operations, improve the flexibility of operation and maintenance work, and provide a variety of interactive visualization components (such as heat maps, space-time trajectory maps, data distribution maps). Operation and maintenance personnel can customize views and explore data details in depth according to real-time needs, supporting decision optimization and problem solving.

[0042] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures indicated in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 This is an example diagram of the architecture of a visual analysis system for operation and maintenance data integrating an AR device and a mobile device according to an embodiment of the present application;

[0045] Figure 2 This is an example diagram of the architecture of an analysis server of a visual analysis system for operation and maintenance data integrating an AR device and a mobile device according to an embodiment of the present application;

[0046] Figure 3 It is a flowchart of a method for visually analyzing operation and maintenance data integrating an AR device and a mobile device according to an embodiment of the present application;

[0047] Figure 4 This is an example diagram of implementing data enhancement of a mobile device by positioning the mobile device based on an infrared optical tracking camera according to the method of an embodiment of the present application;

[0048] Figure 5 This is an example diagram of data source switching, uploading, table selection and moving functions of an enhanced interactive platform based on a mobile device according to an embodiment of the present application;

[0049] Figure 6 This is an example diagram of the statistical indicator information related to the table data displayed on the mobile device enhanced by the augmented reality device according to the method of the embodiment of the present application;

[0050] Figure 7 An example diagram of an augmented reality device being used to enhance a mobile device to display associations and links between multiple mobile devices according to a method according to an embodiment of the present application;

[0051] Figure 8 An example diagram of merging and dynamically displaying multi-device data by enhancing a mobile device through an augmented reality device according to an embodiment of the present application;

[0052] Fig. 9 An example diagram of constructing a three-dimensional visualization of tabular data by augmenting a mobile device with augmented reality according to the method of an embodiment of the present application;

[0053] Fig.10 This is an example diagram of implementing three-dimensional data visualization and gesture-controlled zooming functions according to the method of the embodiment of the present application;

[0054] Fig.11 A schematic diagram of the structure of the device provided in this application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0057] Figure 1 FIG. 1 is an example diagram of the architecture of a visual analysis system for operation and maintenance data integrating an AR device and a mobile device according to an embodiment of the present application, such as Figure 1 As shown, the system may include an analysis server 101, a communication network 102 and / or one or more analysis clients 103. Figure 1 The example shown is a plurality of analysis clients 103 .

[0058] The analysis server 101 can be any appropriate server for storing information, data, programs and / or any other suitable type of content. In some embodiments, the analysis server 101 can perform appropriate functions. For example, in some embodiments, the analysis server 101 can be used to integrate the visual analysis of operation and maintenance data of AR devices and mobile devices. As an optional example, in some embodiments, the analysis server 101 can be used to: build a multi-dimensional visualization analysis framework through the interaction between AR devices and mobile devices; build a visualization collaboration view mode based on the multi-dimensional visualization analysis framework; visualize data operations and user scenarios in the constructed visualization collaboration view mode; import data into a multi-dimensional visualization analysis framework with a visualization collaboration view mode through a visualization tool interface, and build a dynamic visualization framework by creating a combined view of enhanced display.

[0059] As another example, in some embodiments, the analysis server 101 may send a visual analysis method for operation and maintenance data integrating an AR device and a mobile device to the analysis client 103 for user use based on a request from the analysis client 103 .

[0060] Figure 2 FIG. 1 is an example diagram of the architecture of an analysis server of a visual analysis system for operation and maintenance data integrating an AR device and a mobile device according to an embodiment of the present application, such as Figure 2 As shown, the analysis server of this embodiment includes:

[0061] A multi-dimensional visualization analysis framework construction module is used to build a multi-dimensional visualization analysis framework through the interaction between AR devices and mobile devices;

[0062] A visualization collaboration view mode construction module is used to construct a visualization collaboration view mode based on a multi-dimensional visualization analysis framework;

[0063] Data operation and user scenario visualization module, used to visualize data operations and user scenarios in the constructed visualization collaboration view mode;

[0064] A dynamic visualization framework building module is used to import data into a multi-dimensional visualization analysis framework with a visualization collaborative view mode through a visualization tool interface, and to build a dynamic visualization framework by creating a combined view with enhanced display.

[0065] As an optional example, in some embodiments, the analysis client 103 is used to provide a visual analysis interface, which is used to receive a user's selection input operation for visual analysis of operation and maintenance data of an integrated AR device and a mobile device, and, in response to the selection input operation, obtain from the analysis server 101 an analysis interface corresponding to the option selected by the selection input operation and display the analysis interface, wherein the analysis interface at least displays information on the visual analysis of operation and maintenance data of the integrated AR device and the mobile device and operation options for the information on the visual analysis of operation and maintenance data of the integrated AR device and the mobile device.

[0066] In some embodiments, the communication network 102 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 102 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN), and / or any other suitable communication network. The analysis client 103 can be connected to the communication network 102 via one or more communication links (e.g., communication link 104), and the communication network 102 can be linked to the analysis server 101 via one or more communication links (e.g., communication link 105). The communication link can be any communication link suitable for transmitting data between the analysis client 103 and the analysis server 101, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link, or any suitable combination of such links.

[0067] The analysis client 103 may include any one or more clients that present an interface related to the visual analysis of the operation and maintenance data of the fused AR device and the mobile device in an appropriate form for use and operation by the user. In some embodiments, the analysis client 103 may include any suitable type of device. For example, in some embodiments, the analysis client 103 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, and / or any other suitable type of client device.

[0068] Although the analysis server 101 is illustrated as one device, in some embodiments, any suitable number of devices may be used to perform the functions performed by the analysis server 101. For example, in some embodiments, multiple devices may be used to implement the functions performed by the analysis server 101. Alternatively, the functions of the analysis server 101 may be implemented using a cloud service.

[0069] Based on the above system, an embodiment of the present application provides a method for visual analysis of operation and maintenance data that integrates AR devices and mobile devices, which is illustrated by the following embodiments.

[0070] Figure 3 FIG. 1 is a flowchart of a method for visually analyzing operation and maintenance data of an integrated AR device and a mobile device according to an embodiment of the present application. The method for visually analyzing operation and maintenance data of an integrated AR device and a mobile device in this embodiment can be executed on an analysis server, such as Figure 3 As shown, the visual analysis method of operation and maintenance data integrating AR device and mobile device includes the following steps:

[0071] Step S201: construct a multi-dimensional visualization analysis framework through the interaction between the augmented reality device and the mobile device.

[0072] As an optional example, in this embodiment, the position tracking and spatial alignment of the AR (augmented reality) device and the mobile device are performed, and the position coordinates of the AR device and the mobile device are mapped to the AR view by constructing a three-dimensional spatial coordinate system. For example, in this embodiment, the position and direction of the mobile device and the AR device head-mounted display device (hereinafter referred to as the head display) are captured in real time by infrared light tracking technology to ensure that the display content of the mobile device and the virtual view of the AR device are accurately aligned in space, and a three-dimensional spatial coordinate system is constructed by infrared reflective markers and cameras, and the position information of the mobile device and the AR device is transmitted to the server, and the coordinates are mapped to the AR view through a conversion algorithm. Figure 4 This is an example diagram of implementing data enhancement of a mobile device by positioning the mobile device based on an infrared optical tracking camera according to the method of an embodiment of the present application, as shown in FIG. Figure 4 As shown, the present application realizes the positioning of mobile devices based on infrared optical tracking technology, supports displaying overview information in augmented reality devices, displays detailed information in mobile devices and supports natural and agile interaction of users.

[0073] In this embodiment, it is also necessary to realize data synchronization and transmission between the AR device and the mobile device through a communication interface combining hardware and software. As an optional example, this embodiment establishes a two-way communication channel between the AR device and the mobile device, manages the connection status and data update requests between the AR device and the mobile device, and sends the generated AR virtual scene data to the head-mounted display device of the AR device. For example, in this embodiment, a low-latency two-way communication channel can be established between the mobile device and the AR device head display based on the JSON-RPC and WebSockets protocols to realize real-time data synchronization and operation response. The connection status and data update requests of multiple devices are managed by the server, and the generated augmented reality virtual scene data is pushed to the AR head display.

[0074] In this embodiment, a 2D view is loaded and displayed on the mobile device, and the 2D view loaded and displayed on the mobile device is associated on the AR device. For example, in this embodiment, a 2D visualization view, such as a scatter plot, a map, a bar chart, etc., is loaded on the mobile device for direct interaction between the user and the data. In actual applications, this embodiment can maintain communication between the AR device and the mobile device during the interaction process, and associate the content displayed by the AR device with the content displayed by the mobile device according to different interaction modes. Figure 5 This is an example diagram of data source switching, uploading, table selection and moving functions of an enhanced interactive platform based on a mobile device according to an embodiment of the present application, such as Figure 5 As shown, the embodiment of the present application supports users uploading table data on the mobile terminal, provides an interactive upload menu, supports the functions of selecting and uploading data sets, and users can select and move the uploaded table data.

[0075] Step S202: constructing a visual collaborative view mode based on a multi-dimensional visual analysis framework.

[0076] This embodiment implements a view mode that combines details and overviews based on the collaboration between a single device and an AR device. As an optional example, this embodiment displays a detail view of local data through a mobile device, synchronously displays an overview view of global data through an AR device, and uses boundary markers in the overview view to indicate the specific range of the detail view. For example, this embodiment displays high-resolution details of an area on a geographic map through a mobile device, and simultaneously displays a global overview, such as a thumbnail of a map, through an AR device, and uses boundary markers to indicate the specific range of the detail view, so as to achieve a separate display of details and overviews. In actual applications, this embodiment can also project AR-related data indicators around the mobile device through AR, such as edge-projected histograms around scatter plots, to show the distribution characteristics of local data. Figure 6 This is an example diagram of a mobile device displaying relevant statistical indicator information of table data enhanced by an augmented reality device according to an embodiment of the present application, such as Figure 6 As shown, the present application displays statistical information of the data in the table at corresponding positions around the mobile device based on the uploaded table data, does not occupy the limited space inside the mobile device, and assists the user in understanding the table data.

[0077] This embodiment also supports multi-device collaborative interaction and relationship visualization. As an optional example, this embodiment displays the data association of different mobile devices through AR devices, merges the data sets of different mobile devices, and displays the merged results through a 3D view. For example, when a user uses two or more mobile devices, the AR device displays the association of data points, such as using dynamic lines to represent the relationship between data points. The lines connecting the data points can adjust the width and color according to the strength of the data relationship, helping users to intuitively understand the data distribution and trends between devices, thereby realizing multi-device data linking and interaction. Figure 7 This is an example diagram of an augmented reality device being used to enhance a mobile device to display associations and links between multiple mobile devices according to a method according to an embodiment of the present application, such as Figure 7 As shown, due to the limited screen space of the mobile device, it is impossible to display all the table data information. This application associates multiple report data through an augmented reality device. According to the different positions of the table data, the table can be adaptively expanded, and an association relationship can be established between different mobile devices through the augmented device connection.

[0078] In addition, in this embodiment, it is also supported to merge data views on different devices through AR devices, such as generating a heat map between two devices to display the intersection results of two data sets. The results of the merged view can be further represented by a three-dimensional bar chart, such as showing the dimensional distribution of the data intersection, thereby realizing multi-device data merging and dynamic display. Figure 8 This is an example diagram of a method according to an embodiment of the present application to enhance the display of multiple device data merging and dynamic display on a mobile device through an augmented reality device. Figure 8 The presentation form of the multi-device expanded table of the present application is displayed. The table is expanded in different ways by placing the source table in different positions. The horizontal placement indicates that the table completes the column expansion through the connection operation; the vertical placement indicates that the table completes the row expansion by merging the items of the source table. The specific information displayed is adjusted by controlling the spatial position of the mobile device.

[0079] Step S203: Visualize the data operation and the user scenario in the constructed visualization collaboration view mode.

[0080] This embodiment can realize dynamic data filtering, selection and efficient visualization. As an optional example, in this embodiment, in response to the user's data filtering selection, the display range is updated in real time according to the user's data filtering selection, and the filtering selection results are dynamically fed back and highlighted. For example, in this embodiment, the user selects data points or adjusts the filtering conditions through gesture operations, and the mobile device updates the display range in real time, and provides real-time response to user operations through touch feedback (such as vibration or color change), thereby realizing fine operations on the mobile device. The results of the user's selection are dynamically displayed in the AR device, such as highlighting the trajectory of the selected data point, or enhancing the display of the changing trend of the time series data. In the AR device, the status of different data sets, such as outliers, key points or statistical extreme values, is displayed through color coding, so as to realize augmented reality dynamic feedback and display. Fig. 9 An example diagram of constructing a three-dimensional visualization of table data by augmenting a mobile device with augmented reality according to the method of the embodiment of the present application, such as Fig. 9 As shown, based on components such as IATK, this application constructs three-dimensional visualization forms of data in various forms of immersive environments based on reasonable input data, including stereo bar charts, scatter plots, parallel coordinate systems and stacked tables.

[0081] This embodiment also provides enhanced functions for multi-user collaborative scenarios. As an optional example, this embodiment displays the corresponding user view according to the user operation area through the AR device, and sets the corresponding viewing permissions for the displayed user view; when multiple users coordinate operations, the corresponding public view is displayed, and the user partitions and user permissions of the corresponding operations are displayed in the public view. For example, in this embodiment, the AR device headset displays the personal view according to the user operation area, so that the device operated by user A displays the personal view, but other users cannot view it; when multiple users operate collaboratively, the AR device displays the public view area, such as a jointly edited heat map or trend analysis results, thereby realizing user partition and permission display.

[0082] This embodiment can also dynamically annotate the user's filter selection and interaction trajectory through the AR device for collaborative analysis. For example, in this embodiment, the data point selected by user A on mobile device 1 points to the associated point of mobile device 2 through a virtual arrow, thereby realizing the visualization of user interaction.

[0083] Step S204: importing the data into a multi-dimensional visualization analysis framework with a visualization collaboration view mode through a visualization tool interface, and constructing a dynamic visualization framework by creating a combined view with enhanced display.

[0084] As an optional example, this embodiment creates corresponding visualization tool interfaces for different types of data, uses the created visualization tool interfaces to standardize the data, and imports the standardized data into a multi-dimensional visualization analysis framework with a visualization collaborative view mode. For example, this embodiment provides interfaces for different data types, supports data import, cleaning, and processing, and passes it to a visualization view in a standardized format. The visualization view includes scatter plots, time series plots, three-dimensional heat maps, etc., supports custom configuration and parameter adjustment, thereby supporting modular data analysis. Fig.10 This is an example diagram of implementing three-dimensional data visualization and gesture control zooming functions according to the method of the embodiment of the present application, such as Fig.10 As shown, unlike two-dimensional data visualization, users can interact more intuitively with three-dimensional visualization through augmented reality interactive gestures, thereby completing functions such as data filtering and zooming.

[0085] As an optional example, this embodiment loads the 2D view data loaded and displayed by the mobile device on the AR device, and displays the context data of the loaded 2D view data, dynamically connects the data focus on the mobile device with the context data, and dynamically generates a superimposed 3D visualization view. For example, in this embodiment, the data focus on the mobile device is dynamically connected to the context view in augmented reality, such as the seamless extension of the geographic map view, and the visual continuity is enhanced through gradient or dynamic marking in augmented reality, thereby achieving seamless extended view connection. For example, this embodiment uses augmented reality to display superimposed views, such as displaying the time change trend of spatial data (such as a three-dimensional trajectory) above the mobile device map, supporting rotation, scaling and viewing angle adjustment, and achieving a full understanding of complex three-dimensional data.

[0086] In practical applications, this embodiment is applicable to visual analysis of various cases including multi-dimensional data navigation of SPLOM (scatter plot matrix), dynamic expansion view of map, and three-dimensional time series visualization. During the visual analysis process, it is also possible to collect suggestions from experts and users on system operation, improving the interface design, interactive experience and multi-user collaboration function.

[0087] In view of the limited screen space on mobile devices and the limitations of traditional table data analysis, this application proposes a visualization method and system based on augmented reality technology. By coordinating mobile devices and augmented reality technology, a variety of visualization forms are realized to achieve a more intuitive and natural improvement in the user's experience of structured data analysis. The visual analysis method and system of operation and maintenance data that integrates AR devices and mobile devices in the embodiments of the present application are used to realize the spatial presentation and in-depth analysis of multi-dimensional data in the analysis of operation and maintenance work data, the improvement of multi-device collaborative analysis and interaction efficiency, and real-time interaction and dynamic display of data. Through the augmented reality visualization framework of this application, operation and maintenance personnel can more intuitively understand the complex relationship of multi-dimensional data, and at the same time enhance data exploration and analysis capabilities through the three-dimensional display of augmented reality expansion. In a collaborative scenario, the system can also support multi-user collaborative analysis, providing efficient and dynamic support and insights for the operation and maintenance of large-scale systems.

[0088] Specifically, the visual analysis method and system for operation and maintenance data integrating AR devices and mobile devices of the present application have the following beneficial technical effects:

[0089] 1. Multi-source data integration and enhanced visualization: Supports the collection, processing and visualization enhancement of multi-source operation and maintenance data to achieve efficient integration and intuitive display of data; by integrating operation and maintenance data from different sources and types, it can provide a unified data view, allowing operation and maintenance personnel to fully grasp the operating status of the system from macro to micro.

[0090] 2. Multi-dimensional analysis and problem location: Multi-dimensional analysis and detailed display of operation and maintenance data, supporting functions such as data filtering, correlation analysis, and trend identification, to help operation and maintenance personnel quickly locate potential problems, improve the accuracy and efficiency of problem diagnosis, and thus reduce the response time of system failures.

[0091] 3. Improved collaboration and interaction capabilities: Improve collaboration and interaction capabilities in operation and maintenance scenarios, support multi-user collaboration mode, display personalized views and shared views in the same augmented reality environment, avoid data conflicts through multi-user partitioning functions, and promote the collaboration efficiency of the operation and maintenance team, which not only improves work efficiency, but also enhances communication and information sharing between teams.

[0092] 4. Three-dimensional data visualization and interpretation of complex systems: A complex data interpretation system that supports operation and maintenance scenarios through three-dimensional enhanced display provides three-dimensional data visualization support in an augmented reality environment, associates the abstract features of operation and maintenance data (such as time series trends, performance anomalies) with device mapping in the real physical environment, improves data visibility and depth of understanding, and enables operation and maintenance personnel to understand and analyze complex systems more intuitively.

[0093] 5. Intelligent interaction and dynamic insight: Provide intelligent interaction and dynamic insight for operation and maintenance work, support operation and maintenance personnel to quickly switch data dimensions, adjust view parameters, and dynamically update visualization content through touch or gesture operations, improve the flexibility of operation and maintenance work, and provide a variety of interactive visualization components (such as heat maps, space-time trajectory maps, data distribution maps). Operation and maintenance personnel can customize views and explore data details in depth according to real-time needs, supporting decision optimization and problem solving.

[0094] like Fig.11 As shown, the present application also provides a device, including a processor 310, a communication interface 320, a memory 330 for storing a processor executable computer program, and a communication bus 340. The processor 310, the communication interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 implements the above-mentioned visual analysis method of operation and maintenance data of the integrated AR device and the mobile device by running the executable computer program.

[0095] Among them, the computer program in the memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0096] The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected based on actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0098] Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent replacements for some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A visual analysis method for operation and maintenance data integrating AR devices and mobile devices, characterized in that: The method comprises: Build a multi-dimensional visual analysis framework through the interaction between AR devices and mobile devices; Based on the multi-dimensional visual analysis framework, a visual collaborative view mode is constructed; Visualize data operations and user scenarios in the constructed visual collaborative view mode; The data is imported into a multi-dimensional visualization analysis framework with a visualization collaborative view mode through a visualization tool interface, and a dynamic visualization framework is constructed by creating a combined view with enhanced display.

2. The method for visually analyzing operation and maintenance data of integrated AR devices and mobile devices according to claim 1, characterized in that: Through the interaction between augmented reality devices and mobile devices, a multi-dimensional visual analysis framework is constructed, including: Track the position and align the AR device and the mobile device in space, and map the position coordinates of the AR device and the mobile device to the AR view by constructing a three-dimensional spatial coordinate system; Establish a two-way communication channel between the AR device and the mobile device, manage the connection status and data update requests between the AR device and the mobile device, and send the generated AR virtual scene data to the head-mounted display device of the AR device; The 2D view is loaded and displayed on the mobile device, and the 2D view loaded and displayed on the mobile device is associated with the AR device.

3. The visual analysis method for operation and maintenance data integrating AR devices and mobile devices according to claim 2 is characterized in that: Associating the 2D view loaded and displayed by the mobile device on the AR device includes: maintaining communication between the AR device and the mobile device during the interaction process, and associating the content displayed by the AR device with the content displayed by the mobile device according to different interaction modes.

4. The method for visually analyzing operation and maintenance data of integrated AR devices and mobile devices according to claim 1, characterized in that: Based on the multi-dimensional visualization analysis framework, a visualization collaboration view mode is constructed, including: Display the detailed view of local data through the mobile device, and synchronously display the overview view of global data through the AR device, and use boundary marks in the overview view to indicate the specific range of the detailed view; The data association of different mobile devices is displayed through the AR device, and the data sets of different mobile devices are merged in view, and the merged result is displayed through the 3D view.

5. The method for visually analyzing operation and maintenance data of integrated AR devices and mobile devices according to claim 1, characterized in that: Visualize data operations and user scenarios in the built-in visual collaboration view mode, including: Respond to the user's data filtering selection, update the display range in real time according to the user's data filtering selection, and dynamically feedback and highlight the filtering selection results; Displaying a corresponding user view according to the user operation area through the AR device, and setting corresponding viewing permissions for the displayed user view; The user's filtering selections and interaction trajectories are dynamically marked through AR devices.

6. The method for visually analyzing operation and maintenance data of the integrated AR device and the mobile device according to claim 5, characterized in that: The data operation and user scenario visualization are constructed in the visual collaborative view mode, and also include: when multiple users coordinate operations, the corresponding public view is displayed, and the user partitions and user permissions of the corresponding operations are displayed in the public view.

7. The visual analysis method for operation and maintenance data integrating AR devices and mobile devices according to claim 1, characterized in that: Importing data into a multi-dimensional visualization analysis framework with a visualization collaboration view mode through a visualization tool interface includes: creating corresponding visualization tool interfaces for different types of data, using the created visualization tool interfaces to standardize the data, and importing the standardized data into a multi-dimensional visualization analysis framework with a visualization collaboration view mode.

8. The visual analysis method for operation and maintenance data integrating AR devices and mobile devices according to claim 1, characterized in that: A dynamic visualization framework is constructed by creating a combined view of an augmented display, including: loading 2D view data loaded and displayed by a mobile device on an AR device, displaying context data of the loaded 2D view data, dynamically connecting a data focus on the mobile device with the context data, and dynamically generating an overlaid 3D visualization view.

9. A visual analysis system for operation and maintenance data integrating AR devices and mobile devices, characterized in that: The system includes an analysis server, which includes: A multi-dimensional visualization analysis framework construction module is used to build a multi-dimensional visualization analysis framework through the interaction between AR devices and mobile devices; A visualization collaboration view mode construction module is used to construct a visualization collaboration view mode based on a multi-dimensional visualization analysis framework; Data operation and user scenario visualization module, used to visualize data operations and user scenarios in the constructed visualization collaboration view mode; A dynamic visualization framework building module is used to import data into a multi-dimensional visualization analysis framework with a visualization collaborative view mode through a visualization tool interface, and to build a dynamic visualization framework by creating a combined view with enhanced display.

10. A computer device, characterized in that: The computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method according to any one of claims 1 to 8 when executing the program.

Images