A web-based visualization method and system based on heterogeneous multi-engine cooperation

Abstract

The application provides a WEB terminal visualization method and system based on heterogeneous multi-engine cooperation, loads industrial simulation result data for analysis and processing, and constructs analysis results as general visualization data objects; obtains running parameter information, hardware resource parameter information, data feature information of the general visualization data objects, and engine feature information of each candidate visualization engine, and constructs the information as a feature vector; inputs the feature vector into a pre-trained heterogeneous visualization engine selection and discrimination model for reasoning to determine a target visualization engine; serializes the general visualization data objects according to a data format corresponding to the target visualization engine to obtain serialized byte streams; and sends the serialized byte streams to the target visualization engine according to the type of the target visualization engine for visualization presentation. The application fully gives play to the advantages of WEB terminal real-time rendering and server terminal large-scale data processing, and realizes accurate and efficient visualization for CAE, CFD and other scenes.

Description

Technical Field

[0001] This invention relates to the field of industrial simulation model visualization technology, and in particular to a web-based visualization method and a web-based visualization system based on heterogeneous multi-engine collaboration. Background Technology

[0002] With the development of modern science and technology, web browser technology has also developed rapidly. From only being able to browse text, it has evolved to viewing rich media objects such as images and videos. Around 2010, WebGL technology enabled the visualization of 3D models within web browsers. Further technological advancements led to the development of a more efficient method for 3D model visualization around 2020: WebGPU technology. These advancements have provided efficient technical solutions for visualizing 3D models on the web browser. However, with the increasing demands for 3D model quality, especially in CAE (Computer-Aided Engineering) and CFD (Computational Fluid Dynamics) fields, the number of nodes, meshes, and physical fields such as temperature, stress, and displacement have increased by orders of magnitude, from a few MB to tens to hundreds of MB, with some models even reaching tens of GB. Another method for achieving 3D visualization in web browsers involves visualizing the 3D model on the server side first, then extracting the visualization results and pushing them to the web browser in real-time, similar to an image or video stream, thus achieving the effect of 3D model visualization on the web browser.

[0003] The aforementioned general solutions are no longer sufficient to meet the requirements for real-time, efficient, and personalized analysis of simulation results. While pure web-based technologies such as WEBGPU / WEBGL can solve the problems of real-time and efficient model rendering, their acquisition of gigabyte-level data is heavily influenced by network bandwidth, hindering fast, efficient, and real-time data processing. Furthermore, limitations imposed by edge-side memory and video memory restrict the actual amount of data that can be processed, failing to truly meet the requirements of real-world industrial simulation scenarios such as CAE and CFD, where mesh data volumes typically range from tens to hundreds of gigabytes. While pure server-side rendering can partially address the visualization requirements of large-scale industrial simulation scenarios, it renders all visualizations on the server regardless of the data size, significantly impacting the real-time nature of human-computer interaction. Additionally, network speed and bandwidth limitations may lead to the discarding of some visualization results, potentially affecting the accuracy of industrial simulation data analysis. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a web-based visualization method and a corresponding web-based visualization system based on heterogeneous multi-engine collaboration to overcome or at least partially solve the above problems.

[0005] This invention discloses a web-based visualization method based on heterogeneous multi-engine collaboration, the method comprising: Load industrial simulation result data and analyze and process the industrial simulation result data, and construct the analysis results into a general visualization data object and manage the cache of the general visualization data object; Obtain the current web browser's running parameter information, hardware resource parameter information, data feature information of the general visualization data object, and engine feature information of each candidate visualization engine, and construct them into a feature vector; The feature vector is input into a pre-trained heterogeneous visualization engine selection and discrimination model for inference to obtain the selection probability of each candidate visualization engine, and the target visualization engine is determined based on the selection probability. The general visualization data object is serialized according to the data format corresponding to the target visualization engine to obtain a serialized byte stream. The serialized byte stream is sent to the target visualization engine according to the type of the target visualization engine, and the target visualization engine visualizes the serialized byte stream.

[0006] Optionally, cache management is performed on the general visualization data object, including: Based on the frequency of access to the general visualization data object, the general visualization data object is divided into hot data and cold data; The hot data and its serialized byte stream are cached in memory; When a visualization request for the hot data is received again, the cached serialized byte stream is read directly from memory and sent to the corresponding target visualization engine, skipping the reserialization process.

[0007] Optionally, the running parameter information of the current web browser, hardware resource parameter information, data feature information of the general visualization data object, and engine feature information of each candidate visualization engine are obtained and constructed into a feature vector, including: Obtain the current web browser's runtime parameter information, which includes at least whether WebAssembly, WebNN, SharedArrayBuffer, WASM-MEMORY-64-bit, WEBGPU, WebGL, WebWorker, and whether graphics acceleration is enabled. The hardware parameter information is obtained, which includes at least CPU utilization, memory usage, memory usage percentage, GPU memory usage, GPU memory usage percentage, and network bandwidth. Obtain data feature information of a general visualization data object, wherein the data feature information includes at least the number of grid nodes, the number of grid cells, the amount of scalar field data, the amount of vector field data, and the amount of tensor field data; Obtain engine feature information for each candidate visualization engine. The engine feature information includes at least whether it supports WebGPU, WebGL, WebAssembly, server-side rendering, and the hardware resource usage of the rendering object. The running parameter information, the hardware resource parameter information, the data feature information, and the engine feature information are concatenated into the feature vector.

[0008] Optionally, the feature vector is input into a pre-trained heterogeneous visualization engine selection and discrimination model for inference to obtain the selection probability of each candidate visualization engine, and the target visualization engine is determined based on the selection probability, including: The feature vector is input into the hidden layer of the heterogeneous visualization engine selection and discrimination model and subjected to multi-layer convolution operation. The extracted features are then passed to the attention layer. In the attention layer, features are enhanced through random convolution operations, and the enhanced features are then passed to the output layer. The output layer assigns weight values ​​to each candidate visualization engine based on the features obtained from the enhancement process, and uses the softmax method to normalize the weight values ​​to obtain the selection probability of each candidate visualization engine. The candidate visualization engine with the highest probability value among the selected probabilities is determined as the target visualization engine.

[0009] Optionally, before inputting the feature vector into the pre-trained heterogeneous visualization engine's selection discriminative model for inference, the method further includes: Check if the current web browser supports the WebNN interface; If the WebNN interface is supported, the heterogeneous visualization engine is loaded into the edge inference platform built by combining the WebWebNN interface with the ONNX runtime framework to select the discriminant model and accelerate inference. If the WebWebNN interface is not supported, the inference platform built using the WebAssembly interface in conjunction with the ONNX runtime framework will be used to load the heterogeneous visualization engine and select the discriminant model for inference.

[0010] Optionally, the serialized byte stream is sent to the target visualization engine according to the type of the target visualization engine, and the target visualization engine visualizes the serialized byte stream, including: Determine whether the target visualization engine is a web-based rendering engine or a server-side rendering engine. If the target visualization engine is a web-based rendering engine, the serialized byte stream is passed to the target visualization engine through local memory, and the target visualization engine deserializes and performs visualization rendering on the web browser. If the target visualization engine is a server-side rendering engine, the general visualization data object is synchronized according to the server-side rendering protocol. After the target visualization engine completes the visualization rendering on the server side, the rendering result is pushed to the web browser as an image stream for presentation.

[0011] Optionally, if the target visualization engine is a server-side rendering engine, the general visualization data object is synchronized according to the server-side rendering protocol. After the target visualization engine completes the visualization rendering on the server side, the rendering result is pushed to the web browser as an image stream for presentation, including: Send a request signal to the server-side rendering service cluster corresponding to the target visualization engine to create a rendering service instance; The server-side rendering service cluster creates an independent rendering service instance based on the request signal, and the rendering service instance establishes a one-to-one connection with the rendering client object on the web browser side. The general visualization data object is synchronized to the rendering service instance according to the server-side rendering protocol through the connection relationship; The rendering service instance performs visual rendering on the synchronized data and pushes the rendering results to the rendering client object in real time as an image stream for presentation. When a signal to close the rendering service instance is received, the rendering service instance releases the relevant cached resources and disconnects from the rendering client object. The rendering client object releases the corresponding memory object after receiving the disconnection event.

[0012] This invention also discloses a web-based visualization system based on heterogeneous multi-engine collaboration, the system comprising: The simulation data analysis service layer is used to load industrial simulation result data, analyze and process the industrial simulation result data, and construct the analysis results into a general visualization data object. A general visualization model data service layer is used for cache management of the general visualization data objects; The visualization engine intelligent scheduling module of the visualization layer is used to obtain the current web browser's running parameter information, hardware resource parameter information, data feature information of the general visualization data object, and engine feature information of each candidate visualization engine, and construct them into feature vectors. The feature vectors are then input into a pre-trained heterogeneous visualization engine selection and discrimination model for inference to obtain the selection probability of each candidate visualization engine. The target visualization engine is determined based on the selection probability. A visualization model data adaptation layer is used to serialize the general visualization data object according to the data format corresponding to the target visualization engine to obtain a serialized byte stream, and send the serialized byte stream to the target visualization engine according to the type of the target visualization engine. The visualization layer is also used to receive the serialized byte stream and visualize it through the target visualization engine.

[0013] This invention has the following advantages: This invention loads and analyzes industrial simulation results data, constructing the analysis results into a general visualization data object. It acquires the current web browser's operating parameters, hardware resource parameters, data feature information of the general visualization data object, and engine feature information of each candidate visualization engine, constructing a feature vector. This feature vector is then input into a pre-trained heterogeneous visualization engine selection and discrimination model for inference, determining the target visualization engine. The general visualization data object is serialized according to the data format corresponding to the target visualization engine to obtain a serialized byte stream. Based on the type of the target visualization engine, the serialized byte stream is sent to the target visualization engine for visualization presentation. This invention fully leverages the advantages of real-time rendering on the web and large-scale data processing on the server side, achieving accuracy and efficiency for CAE, CFD, and other scenarios. Attached Figure Description

[0014] Figure 1 This is a flowchart of a web-based visualization method based on heterogeneous multi-engine collaboration provided in an embodiment of the present invention; Figure 2 This is a structural block diagram of a web-based visualization system based on heterogeneous multi-engine collaboration provided in an embodiment of the present invention; Figure 3 This is a framework diagram of the inference platform provided in an embodiment of the present invention. Detailed Implementation

[0015] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] This invention targets large-scale scientific computing results data generated in industrial simulation analysis scenarios (such as CAE, CFD, etc.). It comprehensively utilizes the real-time high-performance rendering advantages of WEBGPU / WEBGL on the web browser side, and the hardware advantages of the server side, which has multiple computing cores, large memory space, and highly available video memory, thus supporting the visualization of ultra-large-scale grid (hundreds of millions) data. By constructing an intelligent engine selection mechanism based on AI small models and combining Web Neural Network (WebNN) technology with the ONNX fast inference execution framework, it dynamically and intelligently selects the optimal engine from multiple heterogeneous visualization engines, ultimately achieving accurate, efficient, and personalized scientific visualization presentation.

[0017] Reference Figure 1 and Figure 2 The visualization method in this embodiment runs on a visualization application platform, which may include a simulation data analysis service layer, a general visualization model data service layer, a visualization model data adaptation layer, a Trame service application, and a visualization layer. The visualization layer further includes a visualization engine intelligent scheduling module, a VTK visualization engine module, a DIME visualization engine module, a ThreeJS visualization engine module, and a Trame visualization engine module. The following sections will describe each method step in detail, considering the functional composition of each layer.

[0018] 1. Load industrial simulation result data and analyze and process the industrial simulation result data, and construct the analysis results into a general visualization data object and perform cache management on the general visualization data object.

[0019] Industrial simulation results can originate from the computational output of various CAE or CFD simulation programs, such as the .odb format result data files output by the Abaqus structural simulation analysis solver. These files are typically in the GB range and contain millions of mesh nodes and elements. After loading the simulation result file, the simulation data analysis service layer selectively applies a specific analysis module to perform relevant analysis and processing on the data. For example, for structural simulation results, a deformation analysis module can be applied to perform deformation analysis on the simulation data, thereby obtaining physically meaningful and scientifically visualized data.

[0020] After analysis and processing, the simulation data analysis service layer constructs the analysis results into a CommonVisualData object. CommonVisualData is a universal visualization data structure defined in this invention. Its core design goal is to act as a data bridge between different visualization engines, shielding them from data format differences and providing a unified data foundation for subsequent serialization and engine scheduling. Specifically, the CommonVisualData module supports methods for serializing universal visualization data to VTK byte streams, DIME byte streams, and ThreeJS Geometry object byte streams. It also supports the construction of a Trame-Server data object, which can be synchronized to the Trame-Client module via the Trame protocol.

[0021] The completed CommonVisualData data object is passed to the General Visualization Model Data Service Layer for unified cache management. The General Visualization Model Data Service Layer analyzes the different real-time requirements of the application layer and divides the CommonVisualData data into hot data and cold data. Hot data refers to data frequently used by the application, i.e., data with high access frequency; cold data refers to data used occasionally, i.e., data with relatively low usage frequency. For hot data, the General Visualization Model Data Service Layer caches it in memory. The cached content includes not only the CommonVisualData data object itself but also its serialized data byte stream. When a visualization request for hot data is received again, the cached serialized byte stream can be directly read from memory and sent to the corresponding target visualization engine, thus skipping the reserialization process and significantly optimizing the real-time response to application service requests.

[0022] 2. Obtain the current web browser's running parameter information, hardware resource parameter information, data feature information of the general visualization data object, and engine feature information of each candidate visualization engine, and construct them into feature vectors.

[0023] The visualization engine intelligent scheduling module in the visualization layer constructs feature vectors for AI inference by extracting multi-dimensional feature parameters, specifically including the following four types of feature information: The first category is web browser runtime parameter information. The visualization engine's intelligent scheduling module extracts the current web browser's runtime parameter information, which includes at least: whether it supports WebAssembly (WASM runtime), whether it supports WebNN (Web Neural Network interface), whether it supports SharedArrayBuffer (shared memory buffer), whether it supports WASM-MEMORY-64-bit (64-bit WASM memory addressing), whether it supports WEBGPU (next-generation web-side graphics API), whether it supports WebGL (current mainstream web-side graphics API), whether it supports WebWorker (multi-threaded parallel computing capability), and whether graphics acceleration is enabled. These parameters reflect the current browser environment's support for different visualization technology approaches and are important constraints for engine selection.

[0024] The second category is hardware resource parameter information. The intelligent scheduling module of the visualization engine obtains the current system's hardware parameter information, which includes at least: CPU utilization, memory usage, memory usage percentage, GPU memory usage, GPU memory usage percentage, and network bandwidth. These parameters reflect the real-time load status of the current system resources and are of crucial reference value for determining whether large-scale local rendering is suitable or whether rendering tasks should be offloaded to the server.

[0025] The third category is the data characteristic information of general visualization data objects. The intelligent scheduling module of the visualization engine obtains the data characteristic information of CommonVisualData data objects. This data characteristic information includes at least: the number of grid nodes, the number of grid cells, the amount of scalar field data, the amount of vector field data, and the amount of tensor field data. These data characteristics directly reflect the computational complexity and rendering scale of the visualization task and are one of the core bases for deciding whether to use web-based rendering or server-side rendering. For example, when the number of grid nodes and cells reaches hundreds of millions, the memory and video memory of the web-based system are usually insufficient, and in this case, a server-side rendering engine should be preferred.

[0026] The fourth category is the engine feature information of each candidate visualization engine. The intelligent scheduling module for visualization engines obtains the engine feature information of each candidate visualization engine. The engine feature information includes at least: whether it supports WebGPU, whether it supports WebGL, whether it supports WebAssembly, whether it supports server-side rendering, the memory usage of the rendering object, and the GPU memory usage. The candidate visualization engines in this invention include at least the VTK visualization engine, the DIME visualization engine, the ThreeJS visualization engine, and the Trame visualization engine. Among them, VTK is an industry-leading visualization analysis engine open-sourced by Kitware. This invention compiles the VTK rendering core into WebAssembly format using Emscripten technology and achieves efficient rendering in a web browser by mapping OpenGL ES calls to WebGL calls. The DIME visualization engine is a self-developed WebGPU-based visualization engine that supports WebGPU. The ThreeJS visualization engine is an open-source WebGL-based visualization engine open-sourced by the ThreeJS community, supporting WebGL and featuring strong versatility. The Trame visualization engine is an open-source visualization development framework from Kitware, where rendering calculations are completed on the server side.

[0027] The visualization engine's intelligent scheduling module concatenates the above four types of feature information to construct a unified feature vector, which serves as the input for subsequent AI model inference.

[0028] 3. Input the feature vector into the pre-trained heterogeneous visualization engine selection and discrimination model for inference to obtain the selection probability of each candidate visualization engine, and determine the target visualization engine based on the selection probability.

[0029] Before this step is executed, the AI ​​inference platform in the visualization engine intelligent scheduling module asynchronously loads the pre-trained heterogeneous visualization engine selection and discrimination model from the AI ​​model library during the initialization phase and initializes it.

[0030] Regarding the construction of the AI ​​inference platform, this invention utilizes the latest WebNN interface combined with ONNX Runtime's fast inference technology to build a web browser-based edge AI inference platform framework. Specifically, the visualization engine intelligent scheduling module first detects whether the current web browser supports the WebNN interface. If the current browser supports the WebNN interface, the edge inference platform built using the WebNN interface and the ONNX runtime framework loads a heterogeneous visualization engine to select a discriminant model and accelerates inference. WebNN can significantly improve inference efficiency by leveraging underlying hardware acceleration capabilities (such as GPUs and NPUs). If the current browser does not support the WebNN interface, the inference platform built using the WebAssembly interface and the ONNX runtime framework loads a heterogeneous visualization engine to select a discriminant model for inference. This degradation mechanism ensures that the AI ​​inference module can still run normally in browser environments that do not support WebNN, improving the robustness and compatibility of the inference module. This allows the invention to execute the intelligent engine selection function normally in browser environments with different capability levels.

[0031] Reference Figure 3 The heterogeneous visualization engine selects a discriminative model that employs a classic CNN (Convolutional Neural Network) multi-layer convolutional neural network architecture, including an input layer, hidden layers, attention layers, and an output layer. The specific reasoning process is as follows: First, the constructed feature vector is input into the hidden layer of the discriminative model through the heterogeneous visualization engine. Multi-layer convolution operations are performed in the hidden layer. Through the layer-by-layer feature extraction of the multi-layer convolution kernel, a high-order abstract feature representation is extracted from the original features.

[0032] Subsequently, the extracted features are passed to the attention layer. Within the attention layer, this invention incorporates a random convolutional layer to enhance the features through random convolution operations. The purpose of introducing the random convolutional layer is to increase the randomness and diversity of the model's inference results, preventing the model from producing overly deterministic outputs under specific input patterns, thereby improving the overall rationality of the inference results. This is because in real-world industrial scenarios, the selection of a visualization engine does not always have a single optimal solution. When multiple engines have similar performance, a moderate degree of randomness helps avoid resource imbalance caused by the system always choosing the same engine, making the engine selection decision more robust.

[0033] The enhanced features are passed to the output layer. The output layer assigns weight values ​​to each candidate visualization engine based on the enhanced features, and uses the softmax method to normalize each weight value to obtain the selection probability of each candidate visualization engine.

[0034] Ultimately, the candidate visualization engine with the highest probability value among the selected probabilities will be determined as the target visualization engine, which will be used for the final visualization presentation.

[0035] 4. Serialize the general visualization data object according to the data format corresponding to the target visualization engine to obtain a serialized byte stream. Send the serialized byte stream to the target visualization engine according to the type of the target visualization engine. The target visualization engine then visualizes the serialized byte stream.

[0036] The visualization model data adaptation layer is responsible for serializing CommonVisualData data objects into byte streams of the data format corresponding to the target visualization engine. Specifically, the visualization model data adaptation layer serializes CommonVisualData data objects into one of the following byte streams: VTK byte stream, DIME byte stream, ThreeJS byte stream, or Trame-Server data object. The serialized result data is distributed to the corresponding visualization engine over the network, and the corresponding data byte stream is also returned to the general visualization model data service layer for caching to optimize the real-time response to subsequent application service requests.

[0037] Before sending data to the target visualization engine, the visualization model data adaptation layer first determines whether the target visualization engine is a web-based rendering engine or a server-based rendering engine. In this invention, the VTK visualization engine, DIME visualization engine, and ThreeJS visualization engine are web-based rendering engines, while the Trame visualization engine is a server-based rendering engine.

[0038] When the target visualization engine is a web-based rendering engine, the visualization model data adaptation layer transmits the serialized byte stream to the target visualization engine via local memory. The target visualization engine then deserializes the stream and performs visualization rendering on the web browser side. For different web-based rendering engines, the specific visualization implementation methods can include: (1) If the AI ​​inference platform selects the VTK visualization engine: The visualization model data adaptation layer serializes the CommonVisualData data object according to the VTK data object format to obtain a VTK serialized byte stream, and then sends the byte stream to the VTK visualization engine module. After obtaining the serialized VTK byte stream, the VTK visualization engine module deserializes it to construct a vtkPolyData visualization data structure object, and then visualizes it through the VTK rendering mechanism. This visualization process compiles the vtk-renderer module into WASM (WebAssembly) using Emscripten technology, and achieves efficient scientific visualization of simulation data in a web browser by mapping OpenGL ES to WebGL.

[0039] (2) If the AI ​​inference platform selects the DIME visualization engine: The visualization model data adaptation layer serializes the CommonVisualData data object according to the DIME data object format to obtain a DIME serialized byte stream, and then sends the byte stream to the DIME visualization engine module. After obtaining the serialized DIME byte stream, the DIME visualization engine module deserializes it to construct a DIME visualization data object, and then visualizes it through its drawing mechanism. This visualization technology is entirely based on the latest generation of WebGPU WEB-side graphics API interface, which can make full use of the computation shaders and modern rendering pipeline capabilities provided by WebGPU.

[0040] (3) If the AI ​​inference platform chooses the ThreeJS visualization engine: The visualization model data adaptation layer serializes the CommonVisualData data object according to the ThreeJS Geometry data object format to obtain a ThreeJS Geometry serialized byte stream, and then sends the byte stream to the ThreeJS visualization engine module. After obtaining the serialized byte stream, the ThreeJS visualization engine module deserializes it to construct a threejs-Geometry object, and then visualizes it through its drawing mechanism. This visualization technology is based on WebGL technology, has the characteristics of strong universality, and can run in most browsers that support WebGL.

[0041] When the target visualization engine is a server-side rendering engine, the visualization model data adaptation layer synchronizes the serialized byte stream according to the server-side rendering protocol. After the target visualization engine completes the visualization rendering on the server side, it pushes the rendering result to the web browser as an image stream for presentation. Specifically, taking the Trame visualization engine as an example, this process includes the following steps: (a) The visualization model data adaptation layer sends a "create trame-server" request signal to the Trame service application cluster. The Trame service application cluster is used to perform lifecycle management operations such as dynamic creation and destruction of Trame-Servers as needed.

[0042] (b) After receiving the creation signal, the Trame service application cluster creates an independent Trame-Server-xxx application service instance. This Trame-Server-xxx service instance must maintain a one-to-one connection with the Trame-Client-xxx object in the web browser's visualization layer.

[0043] (c) After Trame-Server-xxx is successfully created, it establishes a connection with Trame-Client-xxx through the Trame interaction protocol. After the connection is successful, the Trame-Server data object is synchronized between Trame-Client and Trame-Server according to the Trame protocol.

[0044] (d) The Trame-Server performs visualization rendering on the synchronized data and interacts with the Trame-Client object model in real time through the Trame interaction protocol, pushing the rendering results to the Trame-Client in real time as an image stream for visualization.

[0045] (e) Upon receiving a "shut down trame-server" signal, the Trame service application cluster sends a kill signal to the corresponding Trame-Server-xxx. Upon receiving this kill signal, Trame-Server-xxx processes the relevant resources according to the set mechanism, such as releasing relevant cached data, and then autonomously disconnects from Trame-Client-xxx. Once the connection is successfully disconnected, Trame-Server-xxx executes the exit code and returns status code 0 to the Trame service application cluster. Upon receiving the disconnection event, Trame-Client-xxx releases the corresponding trame memory object.

[0046] The following is a specific application example of the present invention: This paper illustrates a typical case of visualizing static simulation analysis results in the mechanical industry. The case uses the Abaqus structural simulation analysis solver to perform calculation simulation analysis. After the simulation is completed, the results data are saved in odb format. The file size is generally in the GB range, and the number of mesh nodes and elements is generally in the millions. After the user obtains the .odb file, it loads the simulation result data module of this application platform. The data is then processed by the application analysis module, such as the deformation analysis module, to perform deformation analysis. After obtaining the application analysis results, a CommonVisualData data structure is constructed. This data is cached on this visualization application platform through a caching mechanism. Subsequently, during AI inference platform initialization, a heterogeneous visualization engine is loaded from the AI ​​model library to select a discrimination AI model. The AI ​​inference platform then constructs AI model features using parameters such as web browser runtime parameters, CPU usage, memory usage, GPU memory usage, heterogeneous visualization engine feature parameters, and CommonVisualData data features. Finally, the ONNX inference framework selects the optimal visualization engine among the heterogeneous visualization engines supported by this visualization platform. If the platform selects the VTK visualization engine, it serializes the CommonVisualData in VTK format and sends it to the visualization layer for deserialization of the byte stream by the VTK visualization engine module before final visualization. If the platform selects the DIME visualization engine, it serializes the CommonVisualData in VTK format and sends it to the visualization layer for deserialization of the byte stream before final visualization. The `mmonVisualData` is serialized in DIME format and then sent to the visualization layer. The DIME visualization engine module deserializes the byte stream for final visualization. If the platform selects the ThreeJS visualization engine, it will serialize the `CommonVisualData` in ThreeJS geometry format and then send it to the visualization layer. The ThreeJS visualization engine module deserializes the byte stream for final visualization. If the platform selects the Trame visualization engine, it will first create a `trame-server` service corresponding to the `trame-client` module in the platform's visualization layer by sending a "create trame-server" signal to the Trame service application cluster. After successful creation, the `trame-server` establishes a connection with the `trame-client` via the Trame protocol. The `trame-server` then sends the `CommonVisualData` object to the `trame-client` according to the Trame protocol, and the `trame-client` module in the platform's visualization layer performs the final visualization.

[0047] Thus, after further analysis of the simulation results through a series of application analysis modules, and by combining the characteristics of this platform, the platform's operating hardware parameters, data structure characteristics, heterogeneous visualization engines, etc., the pre-trained AI model combined with the ONNX inference framework ultimately achieves intelligent selection of the best visualization engine, enabling efficient, real-time, and intelligent visualization analysis of the results.

[0048] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0049] Reference Figure 2 This diagram illustrates a structural block diagram of a web-based visualization system based on heterogeneous multi-engine collaboration, as provided in an embodiment of the present invention. Specifically, it may include: The simulation data analysis service layer is used to load industrial simulation result data, analyze and process the industrial simulation result data, and construct the analysis results into a general visualization data object. A general visualization model data service layer is used for cache management of the general visualization data objects; The visualization engine intelligent scheduling module of the visualization layer is used to obtain the current web browser's running parameter information, hardware resource parameter information, data feature information of the general visualization data object, and engine feature information of each candidate visualization engine, and construct them into feature vectors. The feature vectors are then input into a pre-trained heterogeneous visualization engine selection and discrimination model for inference to obtain the selection probability of each candidate visualization engine. The target visualization engine is determined based on the selection probability. A visualization model data adaptation layer is used to serialize the general visualization data object according to the data format corresponding to the target visualization engine to obtain a serialized byte stream, and send the serialized byte stream to the target visualization engine according to the type of the target visualization engine. The visualization layer is also used to receive the serialized byte stream and visualize it through the target visualization engine.

[0050] In an optional embodiment of the present invention, the general visualization model data service layer is further configured to: Based on the frequency of access to the general visualization data object, the general visualization data object is divided into hot data and cold data; The hot data and its serialized byte stream are cached in memory; When a visualization request for the hot data is received again, the cached serialized byte stream is read directly from memory and sent to the corresponding target visualization engine, skipping the reserialization process.

[0051] In an optional embodiment of the present invention, the visualization engine intelligent scheduling module is further configured to: Obtain the current web browser's runtime parameter information, which includes at least whether WebAssembly, WebNN, SharedArrayBuffer, WASM-MEMORY-64-bit, WEBGPU, WebGL, WebWorker, and whether graphics acceleration is enabled. The hardware parameter information is obtained, which includes at least CPU utilization, memory usage, memory usage percentage, GPU memory usage, GPU memory usage percentage, and network bandwidth. Obtain data feature information of a general visualization data object, wherein the data feature information includes at least the number of grid nodes, the number of grid cells, the amount of scalar field data, the amount of vector field data, and the amount of tensor field data; Obtain engine feature information for each candidate visualization engine. The engine feature information includes at least whether it supports WebGPU, WebGL, WebAssembly, server-side rendering, and the hardware resource usage of the rendering object. The running parameter information, the hardware resource parameter information, the data feature information, and the engine feature information are concatenated into the feature vector.

[0052] In an optional embodiment of the present invention, the visualization engine intelligent scheduling module is further configured to: The feature vector is input into the hidden layer of the heterogeneous visualization engine selection and discrimination model and subjected to multi-layer convolution operation. The extracted features are then passed to the attention layer. In the attention layer, features are enhanced through random convolution operations, and the enhanced features are then passed to the output layer. The output layer assigns weight values ​​to each candidate visualization engine based on the features obtained from the enhancement process, and uses the softmax method to normalize the weight values ​​to obtain the selection probability of each candidate visualization engine. The candidate visualization engine with the highest probability value among the selected probabilities is determined as the target visualization engine.

[0053] In an optional embodiment of the present invention, the visualization engine intelligent scheduling module is further configured to: Check if the current web browser supports the WebNN interface; If the WebNN interface is supported, the heterogeneous visualization engine is loaded into the edge inference platform built by combining the WebWebNN interface with the ONNX runtime framework to select the discriminant model and accelerate inference. If the WebWebNN interface is not supported, the inference platform built using the WebAssembly interface in conjunction with the ONNX runtime framework will be used to load the heterogeneous visualization engine and select the discriminant model for inference.

[0054] In an optional embodiment of the present invention, the visualization layer further includes various visualization engine modules. The visualization model data adaptation layer is also used to determine whether the target visualization engine is a web-side rendering engine or a server-side rendering engine; if the target visualization engine is a web-side rendering engine, the serialized byte stream is passed to the target visualization engine through local memory. The visualization engine module corresponding to the target visualization engine is used to perform deserialization and visualization rendering on the web browser side; The visualization model data adaptation layer is also used to synchronize the general visualization data object according to the server-side rendering protocol if the target visualization engine belongs to the server-side rendering engine. The visualization engine module corresponding to the target visualization engine is used to push the rendering result to the web browser as an image stream for presentation after the visualization rendering is completed on the server side.

[0055] In an optional embodiment of the present invention, if the target visualization engine is a server-side rendering engine, the visualization engine module corresponding to the target visualization engine is further configured to: Send a request signal to the server-side rendering service cluster corresponding to the target visualization engine to create a rendering service instance; The server-side rendering service cluster creates an independent rendering service instance based on the request signal, and the rendering service instance establishes a one-to-one connection with the rendering client object on the web browser side. The general visualization data object is synchronized to the rendering service instance according to the server-side rendering protocol through the connection relationship; The rendering service instance performs visual rendering on the synchronized data and pushes the rendering results to the rendering client object in real time as an image stream for presentation. When a signal to close the rendering service instance is received, the rendering service instance releases the relevant cached resources and disconnects from the rendering client object. The rendering client object releases the corresponding memory object after receiving the disconnection event.

[0056] As the system implementation is basically similar to the method implementation, it is described in a relatively simple way. For relevant details, please refer to the description of the method implementation.

[0057] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0058] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0059] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A web-based visualization method based on heterogeneous multi-engine collaboration, characterized in that, The method includes: Load industrial simulation result data and analyze and process the industrial simulation result data, and construct the analysis results into a general visualization data object and perform cache management on the general visualization data object; Obtain the current web browser's running parameter information, hardware resource parameter information, data feature information of the general visualization data object, and engine feature information of each candidate visualization engine, and construct them into a feature vector; The feature vector is input into a pre-trained heterogeneous visualization engine selection and discrimination model for inference to obtain the selection probability of each candidate visualization engine, and the target visualization engine is determined based on the selection probability. The general visualization data object is serialized according to the data format corresponding to the target visualization engine to obtain a serialized byte stream. The serialized byte stream is sent to the target visualization engine according to the type of the target visualization engine, and the target visualization engine visualizes the serialized byte stream.

2. The method according to claim 1, characterized in that, Cache management of the general visualization data object includes: Based on the frequency of access to the general visualization data object, the general visualization data object is divided into hot data and cold data; The hot data and its serialized byte stream are cached in memory; When a visualization request for the hot data is received again, the cached serialized byte stream is read directly from memory and sent to the corresponding target visualization engine, skipping the reserialization process.

3. The method according to claim 1, characterized in that, Obtain the current web browser's runtime parameter information, hardware resource parameter information, data feature information of the general visualization data object, and engine feature information of each candidate visualization engine, and construct them into a feature vector, including: Obtain the current web browser's running parameter information, which includes at least whether WebAssembly, WebNN, SharedArrayBuffer, WASM-MEMORY-64-bit, WEBGPU, WebGL, WebWorker, and whether graphics acceleration is enabled. The hardware parameter information is obtained, which includes at least CPU utilization, memory usage, memory usage percentage, GPU memory usage, GPU memory usage percentage, and network bandwidth. Obtain data feature information of a general visualization data object, wherein the data feature information includes at least the number of grid nodes, the number of grid cells, the amount of scalar field data, the amount of vector field data, and the amount of tensor field data; Obtain engine feature information for each candidate visualization engine. The engine feature information includes at least whether it supports WebGPU, WebGL, WebAssembly, server-side rendering, and the hardware resource usage of the rendering object. The running parameter information, the hardware resource parameter information, the data feature information, and the engine feature information are concatenated into the feature vector.

4. The method according to claim 3, characterized in that, The feature vector is input into a pre-trained heterogeneous visualization engine selection and discrimination model for inference to obtain the selection probability of each candidate visualization engine. The target visualization engine is then determined based on the selection probability, including: The feature vector is input into the hidden layer of the heterogeneous visualization engine selection and discrimination model and subjected to multi-layer convolution operation. The extracted features are then passed to the attention layer. In the attention layer, features are enhanced through random convolution operations, and the enhanced features are then passed to the output layer. The output layer assigns weight values ​​to each candidate visualization engine based on the features obtained from the enhancement process, and uses the softmax method to normalize the weight values ​​to obtain the selection probability of each candidate visualization engine. The candidate visualization engine with the highest probability value among the selected probabilities is determined as the target visualization engine.

5. The method according to claim 4, characterized in that, Before inputting the feature vector into the pre-trained heterogeneous visualization engine to select a discriminative model for inference, the method further includes: Check if the current web browser supports the WebNN interface; If the WebNN interface is supported, the heterogeneous visualization engine is loaded into the edge inference platform built by combining the WebWebNN interface with the ONNX runtime framework to select the discriminant model and accelerate inference. If the WebWebNN interface is not supported, the inference platform built using the WebAssembly interface in conjunction with the ONNX runtime framework will be used to load the heterogeneous visualization engine and select the discriminant model for inference.

6. The method according to claim 2, characterized in that, The serialized byte stream is sent to the target visualization engine according to the type of the target visualization engine, and the target visualization engine visualizes the serialized byte stream, including: Determine whether the target visualization engine is a web-based rendering engine or a server-side rendering engine. If the target visualization engine is a web-based rendering engine, the serialized byte stream is passed to the target visualization engine through local memory, and the target visualization engine deserializes and performs visualization rendering on the web browser. If the target visualization engine is a server-side rendering engine, the general visualization data object is synchronized according to the server-side rendering protocol. After the target visualization engine completes the visualization rendering on the server side, the rendering result is pushed to the web browser as an image stream for presentation.

7. The method according to claim 6, characterized in that, If the target visualization engine is a server-side rendering engine, the general visualization data object is synchronized according to the server-side rendering protocol. After the target visualization engine completes the visualization rendering on the server side, the rendering result is pushed to the web browser as an image stream for presentation, including: Send a request signal to the server-side rendering service cluster corresponding to the target visualization engine to create a rendering service instance; The server-side rendering service cluster creates an independent rendering service instance based on the request signal, and the rendering service instance establishes a one-to-one connection with the rendering client object on the web browser side. The general visualization data object is synchronized to the rendering service instance according to the server-side rendering protocol through the connection relationship; The rendering service instance performs visual rendering on the synchronized data and pushes the rendering results to the rendering client object in real time as an image stream for presentation. When a signal to close the rendering service instance is received, the rendering service instance releases the relevant cached resources and disconnects from the rendering client object. The rendering client object releases the corresponding memory object after receiving the disconnection event.

8. A web-based visualization system based on heterogeneous multi-engine collaboration, characterized in that, The system includes: The simulation data analysis service layer is used to load industrial simulation result data, analyze and process the industrial simulation result data, and construct the analysis results into a general visualization data object. A general visualization model data service layer is used for cache management of the general visualization data objects; The visualization engine intelligent scheduling module of the visualization layer is used to obtain the current web browser's running parameter information, hardware resource parameter information, data feature information of the general visualization data object, and engine feature information of each candidate visualization engine, and construct them into feature vectors. The feature vectors are then input into a pre-trained heterogeneous visualization engine selection and discrimination model for inference to obtain the selection probability of each candidate visualization engine. The target visualization engine is determined based on the selection probability. A visualization model data adaptation layer is used to serialize the general visualization data object according to the data format corresponding to the target visualization engine to obtain a serialized byte stream, and send the serialized byte stream to the target visualization engine according to the type of the target visualization engine. The visualization layer is also used to receive the serialized byte stream and visualize it through the target visualization engine.

9. The system according to claim 8, characterized in that, The general visualization model data service layer is also used for: Based on the frequency of access to the general visualization data object, the general visualization data object is divided into hot data and cold data; The hot data and its serialized byte stream are cached in memory; When a visualization request for the hot data is received again, the cached serialized byte stream is read directly from memory and sent to the corresponding target visualization engine, skipping the reserialization process.

10. The system according to claim 8, characterized in that, The intelligent scheduling module of the visualization engine is also used for: Obtain the current web browser's running parameter information, which includes at least whether WebAssembly, WebNN, SharedArrayBuffer, WASM-MEMORY-64-bit, WEBGPU, WebGL, WebWorker, and whether graphics acceleration is enabled. The hardware parameter information is obtained, which includes at least CPU utilization, memory usage, memory usage percentage, GPU memory usage, GPU memory usage percentage, and network bandwidth. Obtain data feature information of a general visualization data object, wherein the data feature information includes at least the number of grid nodes, the number of grid cells, the amount of scalar field data, the amount of vector field data, and the amount of tensor field data; Obtain engine feature information for each candidate visualization engine. The engine feature information includes at least whether it supports WebGPU, WebGL, WebAssembly, server-side rendering, and the hardware resource usage of the rendering object. The running parameter information, the hardware resource parameter information, the data feature information, and the engine feature information are concatenated into the feature vector.

Images