Data interactive visualization system and method capable of being used for supercomputing platform

Abstract

The invention discloses a data interactive visualization system and method applicable to a supercomputing platform, the visualization method mainly adopts a visualization system, and the method comprises the steps of S10 system environment configuration, S20 data preprocessing, S30 GIS platform construction, S40 data visualization, S50 user interface development and S60 system test and optimization. Through the above settings, the method can directly develop and deploy numerical model products on a supercomputing platform, is suitable for processing a large amount of ocean data and corresponding map data, can improve the data processing speed, reduces the occupation of network bandwidth, improves the interactivity and real-time performance of data visualization, and is suitable for popularization and application. And meanwhile, the loading and rendering processes of the map data are optimized, so that efficient and real-time ocean data visualization is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of oceanography-related data visualization technology, and specifically relates to a data interactive visualization system and method that can be used for supercomputing platforms. Background Art

[0002] With the rapid development of data science and computational science, high-performance computing (HPC) platforms have become standard tools for handling large-scale computing tasks. These platforms are often used to perform complex simulations, analyze, and process large data sets. In fields such as physical oceanography, climate modeling, and bioinformatics, the use of supercomputers has become a key part of research. Existing data visualization methods mainly rely on downloading large amounts of data from high-performance computing (HPC) platforms to local workstations or dedicated visualization servers for processing. This traditional approach has the following technical problems:

[0003] 1) Data transmission efficiency and latency: Existing visualization technologies often require transferring large amounts of data from computing nodes to visualization terminals. This transmission process can cause significant latency, especially when network bandwidth is limited or the dataset is extremely large.

[0004] 2) Reduced interactivity and timeliness: Due to data transmission delays, users cannot obtain immediate feedback when exploring and analyzing data, which limits their ability to monitor marine phenomena in real time and provide decision support. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a system and method for interactive visualization of data on a supercomputing platform, which can increase the speed of data processing, reduce network bandwidth occupancy, and enhance the interactivity and real-time performance of data visualization.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A method for interactively visualizing data on a supercomputing platform, comprising the following steps:

[0008] S10 system environment configuration: select and set up the corresponding supercomputing platform and operating system, and install the GIS software package and data visualization software in the application support layer of the operating platform;

[0009] S20 data preprocessing: select the data source of the database and clean and format the data of the data source through data visualization software;

[0010] S30GIS platform construction: Deploy GIS servers on the supercomputing platform through the GIS software package, integrate relevant visualization engine service functions into the supercomputing platform, and further configure functions on the supercomputing platform based on the integrated visualization engine service functions to build the GIS platform;

[0011] S40 Data Visualization: Importing the pre-processed database data in S20 into the GIS platform, and displaying the pre-processed database data in the form of data, text, or graphics through the visualization management platform; setting interactive functions related to the database data in the GIS platform through the visualization management platform, and performing visualization optimization;

[0012] S50 user interface development: Develop a user-friendly user interface on the system front end, which can provide data selection and display setting functions; the system back end is connected to the GIS server and supercomputing platform.

[0013] S60 system testing and optimization: Perform functional and performance tests on the system, and set hardware requirements, network requirements, and user permissions in the system.

[0014] Furthermore, in the S10, in the process of selecting and setting the corresponding supercomputing platform and operating system, the following method is adopted: the supercomputing platform selects a supercomputer with high-performance computing capabilities, and the operating system selects an operating system of a Linux distribution; in the S10, the GIS software package uses the QGIS or ArcGIS software package, and the data visualization software uses the matplotlib or pandas plug-in of the Python software.

[0015] Furthermore, in the S20, in the process of selecting the data source and cleaning and formatting the data of the data source, the following method is adopted: marine environmental information is selected as the information source of the database, and the Python script of the data visualization software is used to clean the marine data to eliminate incomplete or erroneous marine environmental information, and the marine environmental information of the database is converted into a format supported by the GIS platform through the data quality control unit.

[0016] Furthermore, in S30, the visualization engine service function may adopt the service functions of Baidu map software, AutoNavi map software, Tencent map software and / or Apple map software; when performing function settings on the supercomputing platform, the functions set are map hierarchy, layer properties and graphic operation functions.

[0017] Furthermore, in said S40, when displaying the pre-processed database data in a graphical manner through the visualization management platform, the following method is adopted: different types of ocean data are displayed using scene diagrams and particle animations; when setting interactive functions related to database data in the GIS platform through the visualization management platform, the following method is adopted: interactive functions of zooming, roaming and navigation are set in the GIS platform; when performing visualization optimization, the following method is adopted: according to user feedback or design requirements, the color of the scene diagram is adjusted, the corresponding function icons are improved, and the movement speed of the particle animation is adjusted to enhance the visualization effect.

[0018] Furthermore, in said S60, when performing functional and performance tests on the system, the following method is adopted: checking whether the accuracy of the marine environment information selected in said S20, the operating system response time and the user interaction function correspond, and optimizing the system performance according to the test results.

[0019] Furthermore, in the S60, when setting hardware requirements, network requirements and user permissions in the system, the following method is adopted: with respect to hardware requirements, the system has sufficient storage space and data processing capabilities to support large data sets and complex computing functions; with respect to network requirements, the operating system can perform high-speed network connections to support the rapid processing of large amounts of data; with respect to user permissions, the operating system sets the user's permission to access some data and system resources.

[0020] A data interactive visualization system for a supercomputing platform, applied to a data interactive visualization method for a supercomputing platform, comprising a user layer, an application support layer, a database, a data processing module, a visualization management platform, and an infrastructure layer;

[0021] The user layer includes managers and users;

[0022] The application support layer is provided with a GIS platform, a visualization engine and an interface service;

[0023] The database includes basic information, marine environment information and background management data;

[0024] The data processing module includes a data acquisition unit, a data quality control unit and a data processing unit;

[0025] The visual management platform includes a visual platform and background management;

[0026] The infrastructure layer includes application servers, database servers, terminal PCs and network systems.

[0027] Furthermore, the visualization platform has element information, data display, layer management and toolbar; the background management has user management, authority management and system log.

[0028] The present invention has the following beneficial effects:

[0029] 1. The visualization system of the present invention provides a GIS platform, a visualization engine, and interface services at the application support layer, so that marine environmental information in a database can be visually presented on the visualization platform through the GIS platform and the visualization engine. Simultaneously, the visualization platform can display marine environmental information through data, text, and graphics, and can also display related element information, layer management, and toolbars. Through the layer management and toolbar, users can interactively process the displayed marine environmental information, such as zooming, navigating, adjusting data, and adjusting text. Compared to existing technologies that struggle with visualizing relevant data and enabling user interaction, the system of the present invention can increase data processing speed, reduce network bandwidth usage, enhance the interactivity and real-time nature of data visualization, and optimize the loading and rendering of map data to achieve efficient, real-time marine data visualization.

[0030] 2. The present invention can be used for a method for interactive visualization of data on a supercomputing platform. This method can directly develop and deploy numerical model products on the supercomputing platform and is suitable for processing large amounts of ocean data and its corresponding map data. Based on the supercomputing platform, it can increase the speed of data processing and reduce the occupancy of network bandwidth. Through the GIS platform, it can enhance the interactivity and real-time nature of data visualization and optimize the loading and rendering process of map data to achieve efficient and real-time ocean data visualization. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is the overall architecture of the visualization system of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Terms such as "upper," "inner," "middle," "left," "right," and "one" used in this specification are for ease of description and are not intended to limit the scope of the present invention. Changes or adjustments to these terms, without substantially altering the technical content, are also considered within the scope of the present invention.

[0033] Example 1

[0034] An interactive data visualization system that can be used on supercomputing platforms, such as Figure 1As shown, the method is applied to interactive visualization of data that can be used in supercomputing platforms, including a user layer, an application support layer, a database, a data processing module, a visualization management platform, and an infrastructure layer;

[0035] The user layer includes managers and users. Managers have access and management permissions to the system, and users have partial access permissions to the system.

[0036] The application support layer includes a GIS platform, a visualization engine, and interface services. The GIS platform is the platform software for the geographic information system, used for data collection and management, spatial analysis, map visualization, and decision support. The visualization engine is a tool or system that can transform complex data or information into intuitive graphical displays. It is widely used for data conversion and display, improving understanding and communication efficiency, and supporting decision-making. The visualization engine can be corresponding mapping software such as Baidu Maps, AutoNavi Maps, Tencent Maps, and Apple Maps. Interface services are used to connect other systems, subsystems, modules, or units to the supercomputing platform for functional integration.

[0037] The database includes basic information, marine environmental information, and backend management data. Basic information refers to the basic information of the visualization system, including marine environmental information such as water level, temperature, salinity, ocean currents, wind field, rainfall, air pressure, sea surface temperature, and wave field.

[0038] The data processing module includes a data acquisition unit, a data quality control unit and a data processing unit; the data acquisition unit is used to collect corresponding data, and the data quality control unit is used to uniformly format the corresponding data to improve data reliability.

[0039] The visualization management platform consists of a visualization platform and backend management. The visualization platform features element information, data display, layer management, and a toolbar; the backend management includes user management, permission management, and system logs. The visualization platform displays marine environmental information in the form of data, text, or graphics, and also displays relevant element information, layer management, and toolbars within the platform. The backend management manages and records user information, permissions, and operation history.

[0040] The infrastructure layer includes application servers, database servers, terminal PCs, and network systems.

[0041] In summary, the visualization system of the present invention facilitates the visualization of marine environmental information in a database within the visualization platform by providing a GIS platform, a visualization engine, and interface services within the application support layer. Simultaneously, the visualization platform can display marine environmental information through data, text, and graphics, and can also display related element information, layer management, and toolbars. Through the layer management and toolbars, users can interact with the displayed marine environmental information by performing graphical zooming, navigation, data adjustment, and text adjustment. Compared to existing technologies that struggle with visualizing relevant data and enabling user interaction, the system of the present invention effectively visualizes marine environmental information and interacts with the data, improving the practicality and efficiency of visualization.

[0042] Example 2

[0043] A method for interactive visualization of data on supercomputing platforms, such as Figure 1 As shown, using the data interactive visualization system for supercomputing platforms as in Example 1, the method includes the following steps:

[0044] S10 - System environment configuration: Select and set up the corresponding supercomputing platform and operating system, and install the GIS (also known as Geographic Information System) software package and data visualization software in the application support layer of the operating platform;

[0045] In S10, in the process of selecting and setting the corresponding supercomputing platform and operating system, the following method is adopted: the supercomputing platform selects a supercomputer with high-performance computing capabilities, and the operating system selects a Linux distribution operating system, such as CentOS or Ubuntu; in S10, the GIS software package uses the QGIS or ArcGIS software package, and the data visualization software uses Python software plug-ins such as matplotlib or pandas.

[0046] S20-Data preprocessing: Select the data source of the database and clean and format the data using data visualization software;

[0047] In S20, in the process of selecting the data source and cleaning and formatting the data of the data source, the following method is adopted: marine environmental information is selected as the information source of the database, such as water level, air temperature, salinity, ocean current, wind field, rainfall, air pressure, sea surface temperature and wave field; the Python script of the data visualization software of S10 is used to clean the marine data to eliminate incomplete or erroneous marine environmental information, and the marine environmental information of the database is converted into a format supported by the GIS platform through the data quality control unit.

[0048] S30-GIS platform construction: Deploy a GIS server on the supercomputing platform through the GIS software package. The GIS server can use GeoServer and integrate relevant visualization engine service functions in the supercomputing platform to integrate visualization engine data. Based on the integrated visualization engine service functions, further function configuration is performed on the supercomputing platform to build a GIS platform.

[0049] In S30, the visualization engine service function may utilize the service functions of Baidu Maps, AutoNavi, Tencent Maps, and / or Apple Maps to integrate the corresponding map software data. When setting functions on the supercomputing platform, the functions set include map hierarchy, layer properties, and graphic operations. The map hierarchy is used to divide the map hierarchy into different levels based on the resolution, detail, and information richness of the map. Layer properties refer to the characteristics and settings of a layer, which are mainly used to define the appearance, behavior, and interaction between the layer and other layers. The basic attributes of layer properties include: name, color, line type, line width, transparency, elevation, etc. Graphic operations are mainly used to edit, transform, and / or manage graphics.

[0050] S40-Data Visualization: Import the pre-processed database data from S20 into the GIS platform, and display the marine environmental information data in the database in the form of data, text or graphics through the visualization management platform; set up interactive functions related to the database data in the GIS platform through the visualization management platform, and perform visualization optimization;

[0051] In S40, when the pre-processed database data is displayed in a graphical manner through the visualization management platform, the following method is adopted: the visualization platform of the visualization management platform uses scene diagrams and particle animations to display different types of ocean data, the scene diagrams display data changes in different areas with different colors, and the particle animations dynamically display the movement of ocean currents, wind fields, and wave fields. In addition, the scene diagrams display data changes in different areas with different colors, and the particle animations dynamically display the movement of ocean currents, wind fields, and wave fields;

[0052] When setting up interactive functions related to database data in the GIS platform through the visual management platform, the following methods are adopted: setting up interactive functions of zooming, roaming and navigation related to the graphic operation process in the GIS platform;

[0053] When performing visualization optimization, the following methods are used: according to user feedback or design requirements, the color of the scene image is adjusted, the corresponding function icons are improved, and the movement speed of the particle animation is adjusted to enhance the visualization effect.

[0054] S50 - User Interface Development: Develop a user-friendly user interface on the system front end, which provides data selection and display settings. The back end of the system connects to the GIS server and supercomputing platform. This allows different users to use the visualization system of the present invention and obtain timely information and conditions on the marine environment, with the advantages of strong compatibility and ease of use.

[0055] S60-System Testing and Optimization: Perform functional and performance testing on the system, and set hardware requirements, network requirements, and user permissions in the system.

[0056] In S60, when performing functional and performance tests on the system, the following method is used: checking the accuracy of the ocean environment information selected in S20, the operating system response time, and whether the user interaction function corresponds, and optimizing the system performance according to the test results, such as improving the data processing speed and increasing the system stability.

[0057] When setting hardware requirements, network requirements, and user permissions in the system, the following approach is adopted: Regarding hardware requirements, the system has sufficient storage space and data processing capabilities to support large data sets and complex computing functions; regarding network requirements, the operating system can provide high-speed network connections to support the rapid processing of large amounts of data; regarding user permissions, the operating system sets users' permissions to access certain data and system resources.

[0058] In summary, in the field of marine science, especially physical oceanography, data visualization is crucial for understanding complex marine phenomena. Based on the above-mentioned interactive data visualization method applicable to supercomputing platforms of the present invention, this method can directly develop and deploy numerical model products on supercomputing platforms, and is suitable for processing large amounts of marine data and corresponding map data. Based on the supercomputing platform, it can increase the speed of data processing, reduce network bandwidth usage, enhance the interactivity and real-time nature of data visualization through the GIS platform, and optimize the loading and rendering process of map data to achieve efficient and real-time marine data visualization.

[0059] At the same time, the present invention also has the following beneficial effects:

[0060] 1. Innovation in data processing and visualization technology:

[0061] By directly developing and deploying the GIS platform and visualization management platform on the supercomputing platform, the present invention can process and visualize data in real time, significantly improving the efficiency and real-time performance of data processing and avoiding the data transmission delay in traditional methods.

[0062] The present invention can reduce data transmission requirements and network bandwidth requirements and costs by processing and visualizing the data sources of the database;

[0063] The present invention integrates advanced GIS technology to provide accurate visualization of geospatial data and complex geospatial analysis capabilities.

[0064] 2. Multimodal data display and user interactivity:

[0065] The present invention supports multiple types of display (data, text or graphic form) of ocean data visualization, including water level, air temperature, salinity, etc., which increases the intuitiveness and comprehensibility of data interpretation;

[0066] The present invention provides a high degree of user interactivity, allowing users to customize data display, such as adjusting map levels or layers, selecting data types, etc., thereby improving the user experience;

[0067] The present invention designs a flexible user interface to ensure good compatibility and scalability with other systems or subsystems.

[0068] 3. Promote scientific research and decision-making:

[0069] The present invention implements data processing and advanced visualization techniques to improve the efficiency and quality of data analysis, helping users to quickly identify patterns and trends;

[0070] This invention enhances the application scope of data science, especially in fields such as marine science and environmental monitoring, and provides an efficient and powerful tool to support scientific research and decision-making.

[0071] The embodiments of the present invention are not limited to these. According to the above contents of the present invention, by utilizing common technical knowledge and customary means in this field, without departing from the above basic technical ideas of the present invention, the present invention can also make other various forms of modification, replacement or combination, all of which fall within the scope of protection of the present invention.

Claims

1. A method for interactive visualization of data on a supercomputing platform, characterized in that: The following steps are involved: S10: System environment configuration Select and set up the corresponding supercomputing platform and operating system, and install the GIS software package and data visualization software on the application support layer of the operating platform; S20: Data Preprocessing Select the data source of the database and clean and format the data of the data source through data visualization software; S30: GIS platform construction Deploy GIS servers on the supercomputing platform through the GIS software package, integrate relevant visualization engine service functions in the supercomputing platform, and further configure functions on the supercomputing platform based on the integrated visualization engine service functions to build a GIS platform; S40: Data Visualization Importing the pre-processed database data in S20 into the GIS platform, and displaying the pre-processed database data in the form of data, text or graphics through the visualization management platform; setting interactive functions related to the database data in the GIS platform through the visualization management platform, and performing visualization optimization; S50: User Interface Development A user-friendly user interface is developed on the system front end, which can provide data selection and display setting functions; the system back end is connected to the GIS server and supercomputing platform. S60: System Testing and Optimization Perform functional and performance testing on the system and set hardware requirements, network requirements and user permissions in the system.

2. The method for interactive visualization of data on a supercomputing platform according to claim 1, wherein: In said S10, in the process of selecting and setting the corresponding supercomputing platform and operating system, the following method is adopted: the supercomputing platform selects a supercomputer with high-performance computing capabilities, and the operating system selects a Linux distribution operating system; In the S10, the GIS software package adopts QGIS or ArcGIS software package, and the data visualization software adopts matplotlib or pandas plug-in of Python software.

3. The method for interactive visualization of data on a supercomputing platform according to claim 2, wherein: In S20, in the process of selecting the data source and cleaning and formatting the data of the data source, the following method is adopted: marine environmental information is selected as the information source of the database, the Python script of the data visualization software is used to clean the marine data to eliminate incomplete or erroneous marine environmental information, and the marine environmental information in the database is converted into a format supported by the GIS platform through the data quality control unit.

4. The method for interactive visualization of data on a supercomputing platform according to claim 1, wherein: In S30, the visualization engine service function may adopt the service functions of Baidu map software, AutoNavi map software, Tencent map software and / or Apple map software; when setting functions on the supercomputing platform, the set functions are map hierarchy, layer properties and graphic operation functions.

5. The method for interactive visualization of data on a supercomputing platform according to claim 3, wherein: In said S40, when displaying the pre-processed database data in a graphical manner through the visualization management platform, the following method is adopted: using scene diagrams and particle animations to display different types of ocean data; When setting up interactive functions related to database data in the GIS platform through the visualization management platform, the following methods are adopted: setting interactive functions of zooming, roaming and navigation in the GIS platform; when performing visualization optimization, the following methods are adopted: according to user feedback or design requirements, the color of the scene map is adjusted, the corresponding function icons are improved, and the movement speed of the particle animation is adjusted to enhance the visualization effect.

6. The method for interactive visualization of data on a supercomputing platform according to claim 1, wherein: In said S60, when performing functional and performance tests on the system, the following method is adopted: checking whether the accuracy of the ocean environment information selected in said S20, the operating system response time and the user interaction function correspond, and optimizing the system performance according to the test results.

7. The method for interactive visualization of data on a supercomputing platform according to claim 1, wherein: In said S60 , when setting hardware requirements, network requirements, and user permissions in the system, the following approach is adopted: With respect to hardware requirements, the system has sufficient storage space and data processing capabilities to support large data sets and complex computing functions; Regarding network requirements, the operating system should be capable of high-speed network connections to support the rapid processing of large amounts of data; Regarding user permissions, the operating system sets the permissions for users to access certain data and system resources.

8. A system for interactive visualization of data on a supercomputing platform, applied to the method for interactive visualization of data on a supercomputing platform as claimed in claims 1 to 6, characterized in that: It includes user layer, application support layer, database, data processing module, visual management platform and infrastructure layer; The user layer includes managers and users; The application support layer is provided with a GIS platform, a visualization engine and an interface service; The database includes basic information, marine environment information and background management data; The data processing module includes a data acquisition unit, a data quality control unit and a data processing unit; The visual management platform includes a visual platform and background management; The infrastructure layer includes application servers, database servers, terminal PCs and network systems.

9. The data interactive visualization system applicable to a supercomputing platform according to claim 8, characterized in that: The visualization platform has element information, data display, layer management and toolbar; the background management has user management, authority management and system log.

Images