Scientific computing visualization cloud collaboration analysis method supporting interactive exploration

Abstract

The application discloses a scientific computing visualization cloud-side collaborative analysis method supporting interactive exploration, relates to the technical field of data analysis, and solves the technical problems of analysis errors and rendering lag in processing large-scale or non-standard-format scientific computing files. The application firstly obtains file configuration information, completes file screening and distribution according to the file configuration information, separates numerical file packages and non-numerical file packages, then performs file filtering on the non-numerical file packages and lightweight transportation on the numerical file packages, introduces a spatial density detection mechanism to obtain a density state, enables a transient position cache technology throughout to intercept high-frequency interactive instructions, and finally forms a full-link closed-loop optimization system of cloud-side collaborative analysis, intelligent dirty data cleaning, lightweight and efficient transportation, adaptive security protection, dynamic rendering optimization and low-latency hardware direct-writing interaction.

Description

Technical Field

[0001] This invention belongs to the field of data analysis, specifically a cloud-based collaborative analysis method for scientific computing visualization that supports interactive exploration. Background Technology

[0002] As the volume and variety of textual information increase, the efficiency of analyzing textual information with the naked eye and brain to obtain the required information is becoming increasingly low. Therefore, text parsing programs have emerged. As long as the format, business requirements, and other information of the text to be parsed match the text parsing program, the program can be used to extract the information required for the business needs from the text.

[0003] The invention patent with publication number CN108255802B discloses a general text parsing architecture and a method and apparatus for parsing text based on the architecture. This method uses a preprocessing layer in the architecture to provide componentized preprocessing logic. After obtaining preprocessing components based on the preprocessing logic, the preprocessing components are used to preprocess the text, and the preprocessing results are transmitted to a corpus repository layer for caching. An information search algorithm layer provides information search logic that encapsulates common algorithms. After obtaining the encapsulated algorithms based on the information search logic, the algorithms are cached. The preprocessing components and / or algorithms are hot-swappable. A dimensional business logic layer is used to search for the preprocessed results cached in the corpus repository layer by calling the algorithms in the information search algorithm layer, and then processes the search results through the dimensional business logic to obtain the text parsing results. However, this method does not consider the possibility of large-scale or non-standard files, which could lead to numerical parsing errors due to high spatial density and rendering lag / deadlock.

[0004] This invention proposes a cloud-based collaborative analysis method for scientific computing visualization that supports interactive exploration. It addresses the aforementioned technical issues through a full-link collaborative design that integrates cloud-edge collaborative intelligent traffic splitting and parsing, robust data cleaning specific to Fortran, adaptive density security protection, dynamic rendering precision control, and low-latency interaction via GPU direct writing. Summary of the Invention

[0005] This invention aims to solve at least one of the technical problems existing in the prior art; to this end, this invention proposes a cloud-based collaborative analysis method for scientific computing visualization that supports interactive exploration, in order to solve the technical problems that existing visualization tools are prone to encounter when processing large-scale or non-standard format scientific computing files, such as browser memory overflow, numerical parsing errors, and rendering lag and deadlock.

[0006] To achieve the above objectives, a first aspect of the present invention provides a cloud-based collaborative analysis method for scientific computing visualization that supports interactive exploration, comprising: Step 1: Obtain the file configuration information of the processed files, filter and allocate the processed files based on the file configuration information, encapsulate and upload them, perform online parsing on the encapsulated processed files, and read, split and clean the processed files to obtain parsed file packages; Step 2: The processed parsed file package is numerically segmented using a numerical segmentation algorithm, and the non-numerical file packages are discarded; the file tags and file data of the non-numerical file packages are identified to perform file segmentation and complete file filtering; the numerical file packages are transported in a lightweight manner to complete file transportation. Step 3: Add a spatial density detection mechanism to the file filter to obtain the density status, and perform instantaneous updates based on the density status to complete the interaction.

[0007] Preferably, the step of filtering, allocating, and packaging files for processing based on file configuration information for uploading includes: Extract file configuration information; the file configuration information includes format type, file size, storage usage, and format standardization. Based on file size, the processed files are divided into local parsing files and cloud parsing files; the cloud parsing files are processed by data fragmentation, format encapsulation and security encryption, and the encapsulated files are uploaded to the cloud parsing server in a streaming manner through a preset security interface, while the local parsing files are transferred to the local parsing server.

[0008] Preferably, the step of numerically segmenting the processed parsed file package using a numerical segmentation algorithm and removing non-numerical file packages includes: Extract the parsing file package; where the parsing file package is the cloud-based parsing file uploaded to the cloud parsing server; A robust data cleaning algorithm based on regular expression look-ahead is adopted to preprocess the problem of multiple sets of floating-point number characters sticking together, which is unique to Fortran format output. Before the parsing engine reads the coordinate field, the string with the minus sign and the preceding value is automatically identified by regular expression and a separator is inserted in front of the minus sign "-" to split it. After the splitting is completed, a non-numerical file package is obtained.

[0009] Preferably, the step of identifying file tags and file data of non-numerical file packages to perform file segmentation and file filtering includes: Extract non-numeric file packages; For non-numerical file packages, identify the text type and data semantics line by line, and automatically filter and remove fixed tags, comment information, blank lines, non-physical speed lines at the end of the file and redundant end segments.

[0010] Preferably, the lightweight transport of the numerical file package to complete the file transport includes: Extract the numerical file package; Close the client-side return channel of the numerical file package, extract the coordinate array and rendering attributes from the numerical file package, and transmit them to the front end after encoding them in a binary compact format.

[0011] Preferably, the step of adding a spatial density detection mechanism to the file filtering to obtain the density status includes: Extract non-numeric file packages; The system uses a spatial hashing algorithm to quickly identify and merge atoms in non-numerical file packets, removing NaN dirty data; it also detects the number of atoms per unit space in the file packet in real time and determines whether the number of atoms per unit exceeds the set density safety threshold; the density safety threshold is set based on a combination of system hardware performance parameters and physical property parameters of the atom model.

[0012] Determine whether the number of atoms per unit exceeds the density safety threshold; if yes, mark the density state of the corresponding non-numerical file package as a dangerous density state; otherwise, mark the density state of the corresponding non-numerical file package as a normal density state.

[0013] Preferably, the density safety threshold is set based on a combination of system hardware performance parameters and atomic model physical property parameters, including: Extract system hardware performance parameters and atomic model physical property parameters from the database; among them, the system hardware performance parameters include the CPU computing power limit and GPU memory load, and the atomic model physical property parameters include the physical size of the unit lattice and the total number of atoms; The density safety threshold T is calculated using the formula T=k×(C×G) / (V×N); where k is an empirical coefficient; C is the upper limit of CPU computing power; G is the upper limit of GPU memory load; V is the physical volume of a unit lattice; and N is the total number of atoms.

[0014] Preferably, the instantaneous update interaction based on density state to complete the interaction includes: Extract density state; If the density state is detected as a dangerous density state, the "computation circuit breaker mechanism" is automatically triggered to stop the topology calculation in that area; otherwise, the geometric precision is dynamically switched according to the total number of atoms, and transient position caching technology is enabled to directly write interactive information and update the GPU rendering buffer data.

[0015] Preferably, the step of dynamically switching the geometric precision based on the total number of atoms includes: Extract the total number of atoms; Divide the scale into levels and match the level thresholds between each level; compare the total number of atoms with the level thresholds to obtain the current level, and switch the geometric precision according to the current level.

[0016] Preferably, enabling transient position caching technology to directly write interaction information and update GPU rendering buffer data includes: Intercepting high-frequency user interaction commands, enabling transient position caching technology, collecting real-time interaction pose data and caching it independently, avoiding DOM repainting and full front-end refresh; directly writing interaction transformation parameters to and synchronously updating the native video memory data of the GPU rendering buffer, relying on hardware graphics acceleration to complete view interaction refresh.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention first obtains file configuration information and uses this information to filter and allocate files, then uploads them to cloud and local parsing servers for parsing, effectively avoiding front-end browser memory overflow and large file parsing crashes; relying on a numerical segmentation algorithm, it achieves accurate numerical segmentation and separates numerical file packages from non-numerical file packages, ensuring unbiased parsing of original simulation data and error-free field segmentation; subsequently, it filters non-numerical file packages to completely eliminate problems caused by invalid and impure data; and it performs lightweight transportation of numerical file packages, significantly compressing the amount of transmitted data, reducing network latency and front-end loading pressure; The spatial density detection mechanism detects the total number of atoms in a unit space in real time and compares it with the density safety threshold to obtain the density status. In high-density areas, the calculation circuit breaker is automatically triggered. In normal density scenarios, the geometric segmentation precision is switched in stages according to the total number of atoms in the world. Transient position caching technology is used throughout the process to intercept high-frequency interaction commands. DOM repaint and full component refresh are skipped throughout the process, achieving zero-latency interaction and smooth operation under ultra-large models. Finally, a closed-loop optimization system is formed, which includes cloud-edge collaborative parsing, intelligent dirty data cleaning, lightweight and efficient transmission, adaptive security protection, dynamic rendering optimization and low-latency hardware direct writing interaction.

[0018] 2. This invention incorporates a spatial density detection mechanism to obtain the number of unit atoms and the corresponding density state in real time. Based on the density state, corresponding countermeasures are taken. By calculating the melting mechanism and dynamically switching geometric precision, and by using transient position caching technology to directly write to the GPU buffer throughout the process, it can ensure high frame rate rendering of million-level atom scenes while achieving high-frequency interaction with zero latency, thus balancing system stability and operational smoothness. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1This is a schematic diagram illustrating the working principle of the present invention; Figure 2 This is a flowchart of an embodiment of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Please see Figure 1 The first aspect of this invention provides a cloud-based collaborative analysis method for scientific computing visualization that supports interactive exploration, comprising: Step 1: Obtain the file configuration information of the processed files, filter and allocate the processed files based on the file configuration information, encapsulate and upload them, perform online parsing on the encapsulated processed files, and read, split and clean the processed files to obtain parsed file packages.

[0023] For example, after a user uploads a CONTCAR source file, the module automatically reads and archives four types of core configuration information: file format type, original file size, actual disk storage space occupied, and format standardization rating. For Fortran type calculation files, the format standardization is automatically marked as low, triggering a dedicated numerical cleaning plan in advance to avoid the problem of conventional parsing algorithm adaptation failure.

[0024] This embodiment uses the CONTCAR crystal structure file as the original processing object. This file is a typical compact Fortran encoding format, inherently containing defects such as floating-point value concatenation, dynamic constraint labels, redundant velocity data at the tail, NaN invalid coordinates, and overlapping atomic points. The original file size is 18MB, with a total of 12,800 atoms. The supporting hardware and algorithm parameters are pre-configured: CPU computing power limit C = 2.8GHz, GPU memory load limit G = 6GB, and crystal unit lattice physical volume V = 800. 3 The threshold calculation empirical coefficient k=0.5.

[0025] The system has built-in file hierarchical routing rules and adds a size threshold judgment node. In this embodiment, 5MB is used as the dividing standard. It automatically divides standard format files within 5MB that have a regular structure and no numerical concatenation into local parsing files and pushes them directly to the front-end local parsing server for parsing. The 18MB CONTCAR file, which is large in size, computationally intensive, and has format defects, is forcibly divided into cloud parsing files. Data fragmentation, format encapsulation, and security encryption are performed, and the encapsulated file is uploaded to the cloud parsing server in a streaming manner through a preset security interface, thus avoiding the risk of browser memory overflow and page freezing in advance.

[0026] It should be added that the local parsing server parses local parsing files using WASM or JS, which can protect privacy and has zero latency, while the cloud parsing server is completely free from the computing power limitations of the front-end browser's main thread and adds a dedicated data stream forwarding channel.

[0027] Step 2: The processed parsed file package is numerically segmented using a numerical segmentation algorithm, and the non-numerical file packages are discarded; the file tags and file data of the non-numerical file packages are identified to perform file segmentation and complete file filtering; the numerical file packages are transported in a lightweight manner to complete file transportation.

[0028] For example, the parsed file package output from the cloud is retrieved, and the built-in robust data cleaning algorithm based on regular lookahead is launched. This algorithm is specifically adapted to address the pain point of overlapping floating-point values ​​unique to Fortran format and performs regular segmentation. The algorithm adds a pre-scan recognition step, which globally traverses all data rows before formally parsing the coordinate fields to accurately locate the minus sign "-" character that is adjacent to the number without a separator. After the recognition is successful, a standard space separator is automatically forcibly inserted before the minus sign, accurately splitting overlapping strings such as "1.002-0.3452.111" into independent and recognizable standard floating-point arrays. After the splitting is completed, an automatic classification node is added to automatically divide the file package into two categories based on the data semantics. One category is pure data that only retains atomic coordinates, lattice size, and atomic type, which is collected into a numerical file package. The other category is content containing labels, comments, and redundant characters, which is collected into a non-numerical file package.

[0029] For the separated non-numerical file packages, a multi-level dirty data screening and filtering mechanism is added; the first layer is line-by-line semantic recognition, which automatically and accurately removes the VASP-specific dynamic fixed label "TTT", manually edited comment lines, and meaningless blank placeholder lines; the second layer is targeted cleanup of tail redundancy, which accurately locates non-physical simulation speed data, invalid closing fields, and redundant iteration logs at the end of the file; the third layer is linked to the spatial hash algorithm to assist in verification and mark suspected abnormal data segments in advance.

[0030] For the final purified numerical file package, the system adds a transmission link locking mechanism to directly close the original text full-volume return channel and prevent redundant original characters from flowing to the front end; extract the core visualization key data, including the Float32Array high-precision atomic coordinate array, custom atomic rendering color parameters, atomic radius size parameters, and lattice boundary calibration parameters; after extraction, the data is compressed and encrypted using binary compact encoding format and transmitted to the front-end server; in this embodiment, the CONTCAR crystal structure file size is 18MB, and the actual data received and transmitted by the front end after compression is only 2.1MB.

[0031] Step 3: Add a spatial density detection mechanism to the file filter to obtain the density status, and perform instantaneous updates based on the density status to complete the interaction.

[0032] For example, after the numerical file package is pushed to the front-end server, a real-time density detection mechanism is added to the entire space of the numerical file. The atomic points are calibrated twice by using a spatial hash algorithm, and duplicate and overlapping atomic points are merged and the NaN dirty data removal effect is verified to ensure that all atomic points participating in the density statistics are real and valid. Then, using the original crystal unit lattice as a fixed statistical unit, the actual number of atoms inside each unit lattice is traversed and counted in real time to obtain the unit atom count. The system hardware performance parameters and atomic model physical property parameters are extracted from the database. The density safety threshold is calculated by the formula T=k×(C×G) / (V×N). The unit atom count is compared with the density safety threshold in real time to obtain the density status. In this embodiment, the corresponding numerical values ​​are substituted to calculate the density safety threshold, which is fixed at 60 atoms per unit lattice in this scenario.

[0033] Understandably, when the number of unit atoms inside the crystal lattice exceeds the safety threshold of 60, the crystal lattice region is immediately marked as a dangerous density state; if the number of unit atoms is less than or equal to the density safety threshold, it is marked as a normal density state.

[0034] For all lattice regions marked as hazardous density states, the system automatically triggers the built-in computational shutdown mechanism, simultaneously issuing a stop command to the chemical bond topology calculation module; immediately suspending all chemical bond connection relationship deductions, atomic adjacency relationship searches, and closed-loop topology operations within the high-density region, directly cutting off O(N 2 A massive iterative computing pipeline at the 1000-level; fundamentally eliminating program dead loops, CPU overload deadlocks, browser page lag and crashes, GPU memory overflows and other faults caused by high-density complex structures, ensuring that the rest of the normal areas can still be rendered and displayed normally.

[0035] For lattice regions marked as normal density, a dynamic hierarchical detail (LOD) control strategy is initiated, with three fixed magnitude levels and thresholds pre-set. In this embodiment, the thresholds are set to 10,000 atoms and 50,000 atoms, corresponding to three fixed magnitude levels: the first level is low atomic level (total number of atoms ≤ 10,000), corresponding to 32 high-precision geometric configurations; the second level is medium atomic level (10,000 < total number of atoms ≤ 500,000), corresponding to 16 medium-precision geometric configurations; and the third level is ultra-high atomic level (total number of atoms > 500,000), corresponding to 8 low-precision geometric configurations. In this embodiment, with a total of 12,800 atoms, 16 medium-precision magnitude levels are automatically matched.

[0036] When users perform high-frequency real-time operations such as rotating the model viewpoint, dragging atoms, and selecting local areas, the system intercepts all interaction commands immediately and enables transient position caching technology. The viewpoint offset, atom pose transformation, and coordinate increment data generated by the interaction are uniformly collected into an independent high-speed cache queue, completely skipping the front-end DOM node refresh, component lifecycle redraw, and redundant data backflow from the business layer. The transformation matrix and position correction parameters in the cache are directly written to and updated in real time in the GPU's underlying rendering buffer native video memory data, relying on the graphics card hardware acceleration to complete the real-time view refresh. Ultimately, it achieves a stable high frame rate of 60 frames per second in ultra-large atom model scenes, with all interactive operations being zero-latency, smooth, and without stuttering or ghosting.

[0037] Some of the data in the above formula are calculated by removing dimensions and taking their numerical values. The formula is the closest to the real situation obtained by software simulation of a large amount of collected data. The preset parameters and preset thresholds in the formula are set by those skilled in the art according to the actual situation or obtained through simulation of a large amount of data.

[0038] Working principle of the invention: This invention first obtains file configuration information and uses it to filter and allocate files, then uploads them to cloud and local parsing servers for analysis. It relies on a numerical segmentation algorithm to achieve precise numerical segmentation and separate numerical and non-numerical file packages. Subsequently, it filters non-numerical file packages and performs lightweight transport of numerical file packages. A spatial density detection mechanism is introduced to detect the total number of atoms per unit space in real time and compare it with a density safety threshold to obtain the density status. High-density areas automatically trigger computational circuit breaking. In normal density scenarios, geometric segmentation precision is switched in stages based on the total number of atoms globally. Transient position caching technology is used throughout to intercept high-frequency interaction commands, and DOM repainting and full component refresh are skipped throughout the process. Ultimately, this forms a closed-loop optimization system encompassing cloud-edge collaborative parsing, intelligent dirty data cleaning, lightweight and efficient transmission, adaptive security protection, dynamic rendering optimization, and low-latency hardware direct-write interaction.

[0039] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A cloud-based collaborative analysis method for scientific computing visualization that supports interactive exploration, characterized in that: include: Step 1: Obtain the file configuration information of the processed files, filter and allocate the processed files based on the file configuration information, encapsulate and upload them, perform online parsing on the encapsulated processed files, and read, split and clean the processed files to obtain parsed file packages; Step 2: The processed parsed file package is numerically segmented using a numerical segmentation algorithm, and the non-numerical file packages are discarded; the file tags and file data of the non-numerical file packages are identified to perform file segmentation and complete file filtering; the numerical file packages are transported in a lightweight manner to complete file transportation. Step 3: Add a spatial density detection mechanism to the file filter to obtain the density status, and perform instantaneous updates based on the density status to complete the interaction.

2. The scientific computing visualization cloud-based collaborative analysis method supporting interactive exploration according to claim 1, characterized in that, The process of filtering, allocating, packaging, and uploading files based on file configuration information includes: Extract file configuration information; the file configuration information includes format type, file size, storage usage, and format standardization. Based on file size, the processed files are divided into local parsing files and cloud parsing files; the cloud parsing files are processed by data fragmentation, format encapsulation and security encryption, and the encapsulated files are uploaded to the cloud parsing server in a streaming manner through a preset security interface, while the local parsing files are transferred to the local parsing server.

3. The scientific computing visualization cloud-based collaborative analysis method supporting interactive exploration according to claim 1, characterized in that, The process of numerically segmenting the processed parsed file package using a numerical segmentation algorithm and then removing non-numerical file packages includes: Extract the parsing file package; where the parsing file package is the cloud-based parsing file uploaded to the cloud parsing server; A robust data cleaning algorithm based on regular expression look-ahead is adopted to preprocess the problem of multiple sets of floating-point number characters sticking together, which is unique to Fortran format output. Before the parsing engine reads the coordinate field, the string with the minus sign and the preceding value is automatically identified by regular expression and a delimiter is inserted in front of the minus sign "-" to split it. After the splitting is completed, a non-numerical file package is obtained.

4. The scientific computing visualization cloud-based collaborative analysis method supporting interactive exploration according to claim 1, characterized in that, The process of identifying file tags and file data in non-numerical file packages to perform file segmentation and filtering includes: Extract non-numeric file packages; For non-numerical file packages, identify the text type and data semantics line by line, and automatically filter and remove fixed tags, comment information, blank lines, non-physical speed lines at the end of the file and redundant end segments.

5. The scientific computing visualization cloud-based collaborative analysis method supporting interactive exploration according to claim 1, characterized in that, The lightweight transport of the numerical file package to complete the file transport includes: Extract the numerical file package; Close the client-side return channel of the numerical file package, extract the coordinate array and rendering attributes from the numerical file package, and transmit them to the front end after encoding them in a binary compact format.

6. The scientific computing visualization cloud-based collaborative analysis method supporting interactive exploration according to claim 1, characterized in that, The method of adding a spatial density detection mechanism to file filtering to obtain density status includes: Extract non-numeric file packages; The system uses a spatial hashing algorithm to quickly identify and merge atoms in non-numerical file packets, removing NaN dirty data; it also detects the number of atoms per unit space in the file packet in real time and determines whether the number of atoms per unit exceeds the set density safety threshold; the density safety threshold is set based on a combination of system hardware performance parameters and physical property parameters of the atom model. Determine whether the number of atoms per unit exceeds the density safety threshold; if yes, mark the density state of the corresponding non-numerical file package as a dangerous density state; otherwise, mark the density state of the corresponding non-numerical file package as a normal density state.

7. The scientific computing visualization cloud-based collaborative analysis method supporting interactive exploration according to claim 6, characterized in that, The density safety threshold is set based on a combination of system hardware performance parameters and atomic model physical property parameters, including: Extract system hardware performance parameters and atomic model physical property parameters from the database; among them, the system hardware performance parameters include the CPU computing power limit and GPU memory load, and the atomic model physical property parameters include the physical size of the unit lattice and the total number of atoms; The density safety threshold T is calculated using the formula T=k×(C×G) / (V×N); where k is an empirical coefficient; C is the upper limit of CPU computing power; G is the upper limit of GPU memory load; V is the physical volume of a unit lattice; and N is the total number of atoms.

8. The scientific computing visualization cloud-based collaborative analysis method supporting interactive exploration according to claim 1, characterized in that, The instantaneous update interaction based on density state to complete the interaction includes: Extract density state; If the density state is detected as a dangerous density state, the "computation circuit breaker mechanism" is automatically triggered to stop the topology calculation in that area; otherwise, the geometric precision is dynamically switched according to the total number of atoms, and transient position caching technology is enabled to directly write interactive information and update the GPU rendering buffer data.

9. The scientific computing visualization cloud-based collaborative analysis method supporting interactive exploration according to claim 8, characterized in that, The dynamic switching of geometric precision based on the total number of atoms includes: Extract the total number of atoms; Divide the scale into levels and match the level thresholds between each level; compare the total number of atoms with the level thresholds to obtain the current level, and switch the geometric precision according to the current level.

10. The scientific computing visualization cloud-based collaborative analysis method supporting interactive exploration according to claim 8, characterized in that, The enabled transient position caching technology directly writes interaction information to and updates the GPU rendering buffer data, including: Intercepting high-frequency user interaction commands, enabling transient position caching technology, collecting real-time interaction pose data and caching it independently, avoiding DOM repainting and full front-end refresh; directly writing interaction transformation parameters to and synchronously updating the native video memory data of the GPU rendering buffer, relying on hardware graphics acceleration to complete view interaction refresh.

Images