A method and device for fast loading of Tecplot model data

Abstract

This invention discloses a method and apparatus for rapidly loading Tecplot model data, belonging to the field of infrared simulation technology. The method includes: obtaining the total number of variable rows and the total number of facet rows for a single variable in the Tecplot data file based on the total number of variables and the amount of data per row; determining the data order of the Tecplot data file according to the storage format type; dividing the data in the Tecplot data file into multiple variable intervals and one facet interval according to the data order; and using a multi-threaded parallel mechanism to synchronously read and convert values ​​in each variable interval and facet interval, simultaneously obtaining multiple variable arrays and facet index arrays for loading the Tecplot data file. This invention significantly improves the loading speed and processing efficiency of Tecplot model files.

Description

Technical Field

[0001] This invention relates to the field of infrared simulation technology, and in particular to a method and apparatus for rapidly loading Tecplot model data. Background Technology

[0002] In the fields of infrared simulation and scientific computing visualization, Tecplot format decimal data files are an important carrier for storing and transmitting geometric and physical property information of models. With the continuous improvement of simulation accuracy and complexity, the size of models has increased dramatically, often containing millions or even tens of millions of vertex and surface data elements. This results in significantly longer loading times for model files, becoming a key bottleneck restricting the efficiency of simulation analysis.

[0003] Currently, the common approach for loading this type of data is a sequential line-by-line parsing method. This involves starting from the beginning of the file, reading and converting decimal strings line by line in a single thread, and then processing each variable block and element index data sequentially. While this method is simple to implement, its serial processing mode cannot effectively utilize the parallel computing capabilities of modern multi-core processors, resulting in low loading efficiency for large-scale Tecplot models, with loading times often reaching unacceptable levels.

[0004] Therefore, there is an urgent need to provide a method and device for quickly loading Tecplot model data. Summary of the Invention

[0005] This invention provides a method and apparatus for fast loading of Tecplot model data, which significantly improves the loading speed and processing efficiency of Tecplot model files. The technical solution is as follows: On the one hand, this invention provides a method for fast loading of Tecplot model data, including: Based on the total number of variables and the number of data per row in the Tecplot data file, obtain the total number of rows of variables and the total number of cells for a single variable in the Tecplot data file; The data order of the Tecplot data file is determined according to the storage format type. The data in the Tecplot data file is divided into multiple variable intervals and one facet interval according to the data order. The number of rows in each variable interval is the total number of rows of the variable, so that the data in each variable interval is the same variable data. The number of rows in the facet interval is the total number of rows of the facet. A multi-threaded parallel mechanism is used to synchronously read and transform values ​​for each variable range and surface range, resulting in multiple variable arrays and surface index arrays for loading Tecplot data files.

[0006] Optionally, the amount of data in a single row is obtained by reading the starting row of the Tecplot data file and dividing it into statistical values ​​using a preset delimiter.

[0007] Optionally, the total number of rows of the variable is calculated as follows: H = ceil(Nodes / K) Where H is the total number of rows of the variable, Nodes is the total number of variables, and K is the amount of data in a single row.

[0008] Optionally, the variable interval is calculated as follows: BlockRange(i) = [L0+ 1 + (i-1)·H, L0+ i·H] Where BlockRange(i) is the range of the i-th variable, L0 is the starting line of the Tecplot data file, i takes the value of (1,2,...,n) the total number of n variables, and H is the total number of rows of the variables.

[0009] Optionally, the step of synchronously reading and converting values ​​for each variable range and surface cell range using a multi-threaded parallel mechanism includes: A dynamic thread pool is used for thread management, and the number of concurrent threads is dynamically adjusted according to the number of intervals to be parsed and the number of available CPU cores.

[0010] Optionally, mutexes and condition variables are set during the parallel execution of the multithreaded processes.

[0011] On the other hand, a fast loading device for Tecplot model data is provided, the device comprising: The parameter calculation module is used to obtain the total number of variable rows and the total number of facet rows for a single variable in the Tecplot data file based on the total number of variables and the amount of data per row in the Tecplot data file. The interval division module is used to determine the data order of the Tecplot data file according to the storage format type, and divide the data in the Tecplot data file into multiple variable intervals and one facet interval according to the data order; wherein, the number of rows in each variable interval is the total number of rows of the variable, so that the data in each variable interval is the same variable data, and the number of rows in the facet interval is the total number of rows of the facet. The parallel parsing module is used to synchronously read and transform values ​​for each variable range and surface range using a multi-threaded parallel mechanism, and simultaneously obtain multiple variable arrays and surface index arrays to load Tecplot data files.

[0012] On the other hand, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in any one of the present specification.

[0013] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in any one of the present specification.

[0014] On the other hand, the present invention provides a computer program product, characterized in that it includes a computer program, which, when executed by a processor, implements the steps of any of the methods described in this specification.

[0015] This invention provides a method and apparatus for fast loading of Tecplot model data. The method divides the Tecplot data file into multiple intervals by calculating partitioning parameters based on the total number of vertices and the amount of data per row. It then utilizes a multi-threaded parallel mechanism to synchronously read and convert values ​​from each interval, generating multiple variable data arrays and facet index arrays in parallel. Thus, this invention effectively overcomes the problem of low loading efficiency caused by single-threaded processing when the model data volume increases in traditional sequential parsing methods. Through parallel parsing and precise data localization, it significantly improves the loading speed and processing efficiency of Tecplot model files. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of a method for quickly loading Tecplot model data according to an embodiment of the present invention; Figure 2 This is a hardware architecture diagram of a computer device provided in an embodiment of the present invention; Figure 3 This is a structural diagram of a Tecplot model data fast loading device provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] The following describes the specific implementation of the above concept.

[0020] Please refer to Figure 1 This invention provides a method for fast loading of Tecplot model data, the method comprising: Step 100: Based on the total number of variables and the number of data per row in the Tecplot data file, obtain the total number of rows for a single variable and the total number of rows for each cell in the Tecplot data file. Step 102: Determine the data order of the Tecplot data file according to the storage format type. Divide the data in the Tecplot data file into multiple variable intervals and one facet interval according to the data order. The number of rows in each variable interval is the same as the total number of rows of the variable, so that the data in each variable interval is the same variable data. The number of rows in the facet interval is the same as the total number of rows of the facet. Step 104: Use a multi-threaded parallel mechanism to read and convert values ​​for each variable range and surface range simultaneously, and obtain multiple variable arrays and surface index arrays to load the Tecplot data file.

[0021] In this invention, the partitioning parameters for each interval are first calculated based on the total number of vertices and the amount of data per row, dividing the Tecplot data file into multiple intervals. A multi-threaded parallel mechanism is then used to synchronously read and convert values ​​from each interval, generating multiple variable data arrays and facet index arrays in parallel. Thus, this invention effectively overcomes the problem of low loading efficiency caused by single-threaded processing when the model data volume increases in traditional sequential parsing methods. Through parallel parsing and precise data localization, the loading speed and processing efficiency of Tecplot model files are significantly improved.

[0022] The following description Figure 1 The execution method for each step is shown.

[0023] First, in step 100, the amount of data in a single row is obtained by reading the starting row of the Tecplot data file and dividing it into statistics using a preset delimiter.

[0024] In one embodiment of the present invention, an empty character is used as a delimiter. Each time an empty character is read, the data size of a single line is incremented by 1. The final data size of a single line is used as the input parameter for calculating the total number of lines.

[0025] The total number of rows for a variable is calculated as follows: H = ceil(Nodes / K) Where H is the total number of rows of variables, Nodes is the total number of variables, and K is the number of data per row.

[0026] In step 102, the variable interval is calculated as follows: BlockRange(i) = [L0+ 1 + (i-1)·H, L0+ i·H] Where BlockRange(i) is the range of the i-th variable, L0 is the starting line of the Tecplot data file, i takes the value of (1,2,...,n) the total number of n variables, and H is the total number of rows of variables.

[0027] In one embodiment of the present invention, step 102 further includes: Once the variable intervals are divided, the face interval begins at the next line after all variable intervals have ended, and ends at the position of the starting line plus one less than the total number of face interval rows.

[0028] In step 104, a multi-threaded parallel mechanism is used to synchronously read and convert values ​​for each variable range and surface cell range, including: A dynamic thread pool is used for thread management, and the number of concurrent threads is dynamically adjusted according to the number of intervals to be parsed and the number of available CPU cores.

[0029] In this invention, the dynamic thread pool automatically calculates the optimal number of concurrent threads by monitoring the number of intervals to be parsed and the currently available CPU core resources in real time: when there are many data intervals, the number of threads is appropriately increased to make full use of multi-core performance; when the system load is high, the number of threads is reduced to avoid excessive competition for resources.

[0030] In step 104, mutexes and condition variables are set during the parallel execution of multithreaded processes.

[0031] In this invention, the mutex lock establishes a critical section protection mechanism to prevent multiple threads from accessing or modifying shared resources (such as task queues, parsing status flags, etc.) simultaneously, thereby avoiding memory errors or distorted parsing results caused by data races; while the condition variable achieves efficient scheduling and coordination of parsing tasks through a status notification mechanism between threads—when a thread completes its task, it can promptly wake up the waiting associated threads, which not only eliminates the waste of CPU resources caused by busy waiting, but also ensures the orderly connection between each parsing stage.

[0032] like Figure 2 , Figure 3 As shown, this embodiment of the invention provides a fast loading device for Tecplot model data. This device can be implemented in software, hardware, or a combination of both. From a hardware perspective, as... Figure 2 The diagram shown is a hardware architecture diagram of a computing device containing a test target electromagnetic scattering data determination device provided in an embodiment of the present invention. (Except for...) Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown, the computing device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing packets. Taking software implementation as an example, such as... Figure 3 As shown, as a logical device, it is formed by the CPU of its computing device reading the corresponding computer program from non-volatile memory into memory and running it. The electromagnetic scattering data determination device for the test target provided in this embodiment includes: The parameter calculation module 300 is used to obtain the total number of variable rows and the total number of cells for a single variable in the Tecplot data file based on the total number of variables and the amount of data in a single row in the Tecplot data file. The interval division module 302 is used to determine the data order of the Tecplot data file according to the storage format type, and divide the data in the Tecplot data file into multiple variable intervals and one facet interval according to the data order; wherein, the number of rows in each variable interval is the total number of rows of the variable, so that the data in each variable interval is the same variable data, and the number of rows in the facet interval is the total number of rows of the facet. The parallel parsing module 304 is used to synchronously read and convert values ​​for each variable range and surface range using a multi-threaded parallel mechanism, and obtain multiple variable arrays and surface index arrays to load the Tecplot data file.

[0033] In some specific implementations, the parameter calculation module 300 can be used to perform the above step 100, the interval partitioning module 302 can be used to perform the above step 102, and the parallel parsing module 304 can be used to perform the above step 104.

[0034] In some specific implementations, the parameter calculation module 300 is used to perform the following operations: The data volume of a single row is obtained by reading the starting row of the Tecplot data file and dividing it by a preset delimiter.

[0035] In some specific implementations, the parameter calculation module 300 is also used to perform the following operations: The total number of rows for a variable is calculated as follows: H = ceil(Nodes / K) Where H is the total number of rows of variables, Nodes is the total number of variables, and K is the number of data per row.

[0036] In some specific implementations, the interval division module 302 is used to perform the following operations: The method for calculating the variable interval is as follows: BlockRange(i) = [L0+ 1 + (i-1)·H, L0+ i·H] Where BlockRange(i) is the range of the i-th variable, L0 is the starting line of the Tecplot data file, i takes the value of (1,2,...,n) the total number of n variables, and H is the total number of rows of variables.

[0037] In some specific implementations, the parallel parsing module 304 is also used to perform the following operations: A multi-threaded parallel mechanism is used to synchronously read and convert values ​​for each variable range and surface element range, including: A dynamic thread pool is used for thread management, and the number of concurrent threads is dynamically adjusted according to the number of intervals to be parsed and the number of available CPU cores.

[0038] In some specific implementations, the parallel parsing module 304 is also used to perform the following operations: Mutexes and condition variables are set during the parallel execution of multithreaded processes.

[0039] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the device for quickly loading Tecplot model data. In other embodiments of the present invention, the device for quickly loading Tecplot model data may include more or fewer components than illustrated, or combine some components, split some components, or have different component arrangements. The components illustrated may be implemented in hardware, software, or a combination of software and hardware.

[0040] The information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description of the method embodiment of the present invention, and will not be repeated here.

[0041] This invention also provides a computing device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the method for fast loading of Tecplot model data in any embodiment of this invention.

[0042] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the Tecplot model data fast loading method according to any embodiment of this invention.

[0043] Embodiments of this application also provide a computer program product comprising a computer program, wherein a processor of a computer device reads the computer program from a computer-readable storage medium, and the processor executes the computer program, causing the computer device to perform any of the above embodiments of the fast loading method for testing Tecplot model data.

[0044] Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage medium.

[0045] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0046] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0047] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0048] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion module connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion module execute some and all of the actual operations, thereby realizing the function of any of the above embodiments.

[0049] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0050] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for fast loading of Tecplot model data, characterized in that, include: Based on the total number of variables and the number of data per row in the Tecplot data file, obtain the total number of rows of variables and the total number of cells for a single variable in the Tecplot data file; The data order of the Tecplot data file is determined according to the storage format type. The data in the Tecplot data file is divided into multiple variable intervals and one facet interval according to the data order. The number of rows in each variable interval is the total number of rows of the variable, so that the data in each variable interval is the same variable data. The number of rows in the facet interval is the total number of rows of the facet. A multi-threaded parallel mechanism is used to synchronously read and transform values ​​for each variable range and surface range, resulting in multiple variable arrays and surface index arrays for loading Tecplot data files.

2. The method as described in claim 1, characterized in that, The amount of data in a single row is obtained by reading the starting row of the Tecplot data file and dividing it by a preset delimiter.

3. The method as described in claim 1, characterized in that, The total number of rows for the variable is calculated as follows: H = ceil(Nodes / K) Where H is the total number of rows of the variable, Nodes is the total number of variables, and K is the amount of data in a single row.

4. The method as described in claim 1, characterized in that, The method for calculating the variable interval is as follows: BlockRange(i) = [L0+ 1 + (i-1)·H, L0+ i·H] Where BlockRange(i) is the range of the i-th variable, L0 is the starting line of the Tecplot data file, i is the sequential index of the variable in the variable range, and H is the total number of rows of the variable.

5. The method as described in claim 1, characterized in that, The method of using a multi-threaded parallel mechanism to synchronously read and convert values ​​for each variable range and surface element range includes: A dynamic thread pool is used for thread management, and the number of concurrent threads is dynamically adjusted according to the number of intervals to be parsed and the number of available CPU cores.

6. The method as described in claim 1, characterized in that, The multi-threaded parallel execution process is equipped with mutexes and condition variables.

7. A device for rapidly loading Tecplot model data, characterized in that, The device includes: The parameter calculation module is used to obtain the total number of variable rows and the total number of facet rows for a single variable in the Tecplot data file based on the total number of variables and the amount of data per row in the Tecplot data file. The interval division module is used to determine the data order of the Tecplot data file according to the storage format type, and divide the data in the Tecplot data file into multiple variable intervals and one facet interval according to the data order; wherein, the number of rows in each variable interval is the total number of rows of the variable, so that the data in each variable interval is the same variable data, and the number of rows in the facet interval is the total number of rows of the facet. The parallel parsing module is used to synchronously read and transform values ​​for each variable range and surface range using a multi-threaded parallel mechanism, and simultaneously obtain multiple variable arrays and surface index arrays to load Tecplot data files.

8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images