Rapid loading method and device for Tecplot model data

Abstract

The invention discloses a rapid loading method and device for Tecplot model data, and belongs to the technical field of infrared simulation. The method comprises the following steps: acquiring the total variable row number and the total surface element row number of a single variable in a Tecplot data file according to the total variable number and the single-row data volume of the Tecplot data file; determining a data sequence of the Tecplot data file according to the storage format type, and dividing data in the Tecplot data file into a plurality of variable intervals and a surface element interval according to the data sequence; and synchronously performing reading and numerical conversion on each variable interval and each surface element interval by utilizing a multi-thread parallel mechanism, and simultaneously obtaining a plurality of variable arrays and a plurality of surface element index arrays so as to load the Tecplot data file. According to the method, the loading speed and the processing efficiency of the Tecplot model file are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of infrared simulation technology, and in particular to a fast loading method and device for Tecplot model data. BACKGROUND

[0002] In the field of infrared simulation and scientific computing visualization, the Tecplot format decimal data file is an important carrier for storing and transmitting model geometry and physical property information. With the continuous improvement of simulation accuracy and complexity, the model size has increased dramatically, often containing millions or even tens of millions of vertex and surface element data, resulting in a significant time-consuming loading process, which has become a key bottleneck restricting the efficiency of simulation analysis.

[0003] Currently, the loading of such data generally uses a sequential line-by-line parsing method, which starts from the beginning of the file and reads and converts decimal strings line by line in a single-threaded manner, processing each variable block and surface element index data in turn. Although this method is simple to implement, its serial processing mode cannot effectively utilize the parallel computing capabilities of modern multi-core processors, resulting in low loading efficiency of large-scale Tecplot models, and the loading time is often unacceptable.

[0004] Therefore, there is an urgent need to provide a fast loading method and device for Tecplot model data. SUMMARY

[0005] The present application provides a fast loading method and device for Tecplot model data, which significantly improves the loading speed and processing efficiency of Tecplot model files. The technical solution is as follows: On the one hand, the present application provides a fast loading method for Tecplot model data, comprising: According to the total number of variables and the amount of data in a single line of the Tecplot data file, the total number of lines of a single variable and the total number of surface elements in the Tecplot data file are obtained; According to the storage format type, the data order of the Tecplot data file is determined, and the data in the Tecplot data file is divided into multiple variable intervals and a surface element interval according to the data order; wherein the number of lines in each variable interval is the total number of lines, so that the data in each variable interval is the same variable data, and the number of lines in the surface element interval is the total number of surface elements; Using a multi-threaded parallel mechanism, each variable interval and surface element interval is read and converted simultaneously to obtain multiple variable arrays and surface element index arrays, thereby loading the Tecplot data file.

[0006] Optionally, the single-row data amount is obtained by reading the starting line of the Tecplot data file and splitting statistics according to a preset delimiter.

[0007] Optionally, the variable total row number is calculated as follows: H = ceil(Nodes / K) wherein H is the variable total row number, Nodes is the variable total number, and K is the single-row data amount.

[0008] Optionally, the variable interval is calculated as follows: BlockRange(i) = [L0+ 1 + (i-1)·H, L0+ i·H] wherein BlockRange(i) is the interval of the ith variable, L0 is the starting line of the Tecplot data file, i takes values of (1, 2, …, n), n is the total number of variables, and H is the variable total row number.

[0009] Optionally, the reading and numerical conversion of each variable interval and the element interval are performed synchronously using a multi-thread parallel mechanism, including: Dynamic thread pool is used for thread management, and the number of concurrent threads is dynamically adjusted according to the number of intervals to be parsed and the number of available CPU cores.

[0010] Optionally, a mutex and a condition variable are provided during the multi-thread parallel execution process.

[0011] In another aspect, a device for quickly loading Tecplot model data is provided, and the device includes: a parameter calculation module configured to obtain the variable total row number and the element total row number of a single variable in a Tecplot data file according to the variable total number and the single-row data amount of the Tecplot data file; an interval division module configured to determine the data order of the Tecplot data file according to the storage format type, and divide the data in the Tecplot data file into a plurality of variable intervals and an element interval according to the data order; wherein the number of rows of each variable interval is the variable total row number, so that the data in each variable interval is the data of the same variable, and the number of rows of the element interval is the element total row number; a parallel analysis module configured to synchronously read and convert each variable interval and the element interval using a multi-thread parallel mechanism, and obtain a plurality of variable arrays and an element index array, so as to load the Tecplot data file.

[0012] In another aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method of any one of the present application.

[0013] In another aspect, the present application provides a computer readable storage medium storing a computer program, wherein the computer program, when executed in a computer, causes the computer to perform the method of any one of the present application.

[0014] In another aspect, the present application provides a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of the method of any one of the present application.

[0015] The present application provides a fast loading method and device for Tecplot model data, which divides the Tecplot data file into multiple intervals according to the total number of vertices and the amount of single-row data to calculate the division parameters of each interval, reads and converts the values of each interval synchronously by using the multi-thread parallel mechanism, and generates multiple variable data arrays and face element index arrays in parallel. In this way, the present application effectively overcomes the problem of low loading efficiency caused by single-thread processing when the amount of model data increases in the traditional sequential parsing method, and significantly improves the loading speed and processing efficiency of the Tecplot model file through parallel parsing and accurate data positioning. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0017] Figure 1 is a fast loading method flowchart of Tecplot model data provided by an embodiment of the present application; Figure 2 is a hardware architecture diagram of a computer device provided by an embodiment of the present application; Figure 3 is a fast loading device structure diagram of Tecplot model data provided by an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings of the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0019] The following describes a specific implementation of the above concept.

[0020] Referring to Figure 1 The embodiments of the present application provide a fast loading method of a Tecplot model data, which comprises the following steps. In step 100, the total number of variable rows and the total number of face rows of a single variable in the Tecplot data file are obtained according to the total number of variables in the Tecplot data file and the single-row data amount. In step 102, the data order of the Tecplot data file is determined according to the storage format type, and the data in the Tecplot data file is divided into multiple variable intervals and a face interval according to the data order; wherein the number of rows of each variable interval is the total number of variable rows, so that the data in each variable interval is the data of the same variable, and the number of rows of the face interval is the total number of face rows. In step 104, the multiple-thread parallel mechanism is used to synchronously read and convert the value of each variable interval and the face interval, and multiple variable arrays and face index arrays are obtained, so as to load the Tecplot data file.

[0021] In the present application, first, the division parameters of each interval are calculated according to the total number of vertices and the single-row data amount, the Tecplot data file is divided into multiple intervals, the multiple-thread parallel mechanism is used to synchronously read and convert the value of each interval, and multiple variable data arrays and face index arrays are generated in parallel. In this way, the present application effectively overcomes the problem of low loading efficiency caused by single-thread processing when the model data amount increases in the traditional sequential parsing method, and significantly improves the loading speed and processing efficiency of the Tecplot model file through parallel parsing and accurate data positioning.

[0022] The following describes Figure 1 The execution mode of each step shown.

[0023] First, in step 100, the single-row data amount is obtained by reading the starting row of the Tecplot data file and being segmented by a preset delimiter.

[0024] In one embodiment of the present application, the empty character is used as a delimiter, and each time an empty character is read, the single-row data amount is increased by 1, and the final single-row data amount is used as an input parameter for calculating the total number of rows.

[0025] The calculation method of the total number of rows is as follows: H = ceil(Nodes / K) wherein H is the total number of rows, Nodes is the total number of variables, and K is the single-row data amount.

[0026] In step 102, the calculation method of the variable interval is as follows: BlockRange(i) = [L0+ 1 + (i-1)·H, L0+ i·H] wherein BlockRange(i) is the interval of the ith variable, L0 is the starting row of the Tecplot data file, i takes values of (1, 2, …, n), n is the total number of variables, and H is the total number of rows.

[0027] In one embodiment of the present application, step 102 further comprises: After the variable interval is divided, the cell interval starts from the next row after the end of all variable intervals and ends at the position of the starting row plus the total number of cells minus one.

[0028] In step 104, the multi-thread parallel mechanism is used to synchronously read and convert the value of each variable interval and cell interval, including: A dynamic thread pool is used for thread management, and the number of concurrent threads is dynamically adjusted according to the number of intervals to be parsed and the number of available CPU cores.

[0029] In the present application, the dynamic thread pool automatically calculates the optimal number of concurrent threads by real-time monitoring of the number of intervals to be parsed and the current available CPU core resources: when there are many data intervals, the number of threads is appropriately increased to fully utilize the multi-core performance; when the system load is high, the number of threads is reduced to avoid excessive resource contention.

[0030] In step 104, a mutex and a condition variable are set during the multi-thread parallel execution process.

[0031] In the present application, the mutex establishes a critical section protection mechanism to prevent multiple threads from simultaneously accessing or modifying shared resources (such as task queues, parsing state flags, etc.), thereby avoiding memory errors or parsing result errors caused by data competition; and the condition variable uses a state notification mechanism between threads to achieve efficient scheduling and coordination of parsing tasks: when a thread completes a task, it can wake up the associated threads waiting in time, which not only eliminates the CPU resource waste caused by busy waiting, but also ensures the orderly connection between the parsing stages.

[0032] As Figure 2 , Figure 3 indicated, the embodiment of the present application provides a fast loading device for Tecplot model data, and the device embodiment can be realized by software, or realized by hardware or a combination of software and hardware. From the hardware layer, as Figure 2 indicated, a hardware architecture diagram of a computing device where the device for determining electromagnetic scattering data of a test target is located is provided in the embodiment of the present application. In addition to the processor, the memory, the network interface, and the non-volatile memory shown in Figure 2 , the computing device where the device is located in the embodiment can usually also include other hardware, such as a forwarding chip responsible for processing packets, and the like. Taking software implementation as an example, as Figure 3 indicated, as a device in a logical sense, it is formed by the CPU of the computing device where the device is located reading the corresponding computer program in the non-volatile memory into the memory and running. The device for determining electromagnetic scattering data of a test target provided in the embodiment includes: The parameter calculation module 300 is configured to obtain the total number of rows of a single variable and the total number of rows of a cell in the Tecplot data file according to the total number of variables of the Tecplot data file and the single-row data amount. The interval division module 302 is configured to determine the data order of the Tecplot data file according to the storage format type, and divide a plurality of variable intervals and a cell interval in the Tecplot data file according to the data order. The number of rows of each variable interval is the total number of rows of the variable, so that the data in each variable interval is the data of the same variable, and the number of rows of the cell interval is the total number of rows of the cell. The parallel analysis module 304 is configured to simultaneously read and convert values of each variable interval and the cell interval by using a multi-thread parallel mechanism, and simultaneously obtain a plurality of variable arrays and a cell index array, so as to load the Tecplot data file.

[0033] In some specific embodiments, the parameter calculation module 300 can be configured to execute the above step 100, the interval division module 302 can be configured to execute the above step 102, and the parallel analysis module 304 can be configured to execute the above step 104.

[0034] In some specific embodiments, the parameter calculation module 300 is configured to perform the following operations: The single-row data amount is obtained by reading the starting row of the Tecplot data file and splitting by a preset delimiter.

[0035] In some specific embodiments, the parameter calculation module 300 is further configured to perform the following operations: The calculation method of the total number of rows of the variable is as follows: H = ceil(Nodes / K) Wherein, H is the total number of variable rows, Nodes is the total number of variables, and K is the amount of single-row data.

[0036] In some specific embodiments, the interval division module 302 is configured to perform the following operations: The calculation method of the variable interval is as follows: BlockRange(i) = [L0+ 1 + (i-1)·H, L0+ i·H] Wherein, BlockRange(i) is the interval of the ith variable, L0 is the starting row of the Tecplot data file, i takes values of (1, 2, …, n), n is the total number of variables, and H is the total number of variable rows.

[0037] In some specific embodiments, the parallel parsing module 304 is further configured to perform the following operations: The multi-thread parallel mechanism is used to synchronize the reading and numerical conversion of each variable interval and the element interval, including: A dynamic thread pool is used for thread management, and the number of concurrent threads is dynamically adjusted according to the number of intervals to be parsed and the number of available CPU cores.

[0038] In some specific embodiments, the parallel parsing module 304 is further configured to perform the following operations: The mutual exclusion lock and the condition variable are set in the multi-thread parallel execution process.

[0039] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the fast loading device for Tecplot model data. In other embodiments of the present application, the fast loading device for Tecplot model data can include more or fewer components than the illustration, or combine some components, or split some components, or different component arrangement. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0040] The information interaction, execution process, and the like between the modules in the above device are based on the same concept as the method embodiments of the present application, and the specific content can be referred to the description in the method embodiments of the present application, which will not be described here.

[0041] The embodiments of the present application also provide a computing device including a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the fast loading method for Tecplot model data in any of the embodiments of the present application.

[0042] The embodiment of the present application also provides a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program enables a processor to execute the fast loading method of Tecplot model data in any of the embodiments of the present application when the processor executes the computer program.

[0043] The embodiment of the present application also provides a computer program product, and the computer program product comprises a computer program, and a processor of a computer device reads the computer program from a computer readable storage medium, and the processor executes the computer program, so that the computer device executes the fast loading method of testing Tecplot model data in any of the embodiments.

[0044] Specifically, a system or device provided with a storage medium can be provided, and the storage medium stores software program codes for realizing the functions of any of the embodiments, and a computer (or CPU or MPU) of the system or device reads and executes the program codes stored in the storage medium.

[0045] In this case, the program codes read from the storage medium can realize the functions of any of the embodiments, and therefore the program codes and the storage medium storing the program codes constitute a part of the present application.

[0046] The embodiments of the storage medium for storing the program codes include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, nonvolatile memory cards and ROMs. Alternatively, the program codes can be downloaded from a server computer through a communication network.

[0047] In addition, it should be clear that not only the program codes read by the computer can be executed, but also part or all of the actual operations can be completed by operating systems and the like operating on the computer based on the instructions of the program codes, so as to realize the functions of any of the embodiments.

[0048] In addition, it can be understood that the program codes read from the storage medium can be written into the memory provided in the expansion board inserted into the computer or the memory provided in the expansion module connected to the computer, and then part or all of the actual operations can be executed by the CPU and the like installed on the expansion board or the expansion module based on the instructions of the program codes, so as to realize the functions of any of the embodiments.

[0049] It should be noted that, in the present document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0050] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program performs the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes various storage media that can store program codes, such as ROM, RAM, magnetic disk or optical disk.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for fast loading of Tecplot model data, characterized in that, include: Based on the total number of variables and the number of data per row in the Tecplot data file, obtain the total number of rows of variables and the total number of cells for a single variable in the Tecplot data file; The data order of the Tecplot data file is determined according to the storage format type. The data in the Tecplot data file is divided into multiple variable intervals and one facet interval according to the data order. The number of rows in each variable interval is the total number of rows of the variable, so that the data in each variable interval is the same variable data. The number of rows in the facet interval is the total number of rows of the facet. A multi-threaded parallel mechanism is used to synchronously read and transform values ​​for each variable range and surface range, resulting in multiple variable arrays and surface index arrays for loading Tecplot data files.

2. The method as described in claim 1, characterized in that, The amount of data in a single row is obtained by reading the starting row of the Tecplot data file and dividing it by a preset delimiter.

3. The method as described in claim 1, characterized in that, The total number of rows for the variable is calculated as follows: H = ceil(Nodes / K) Where H is the total number of rows of the variable, Nodes is the total number of variables, and K is the amount of data in a single row.

4. The method as described in claim 1, characterized in that, The method for calculating the variable interval is as follows: BlockRange(i) = [L0+ 1 + (i-1)·H, L0+ i·H] Where BlockRange(i) is the range of the i-th variable, L0 is the starting line of the Tecplot data file, i is the sequential index of the variable in the variable range, and H is the total number of rows of the variable.

5. The method as described in claim 1, characterized in that, The method of using a multi-threaded parallel mechanism to synchronously read and convert values ​​for each variable range and surface element range includes: A dynamic thread pool is used for thread management, and the number of concurrent threads is dynamically adjusted according to the number of intervals to be parsed and the number of available CPU cores.

6. The method as described in claim 1, characterized in that, The multi-threaded parallel execution process is equipped with mutexes and condition variables.

7. A device for rapidly loading Tecplot model data, characterized in that, The device includes: The parameter calculation module is used to obtain the total number of variable rows and the total number of facet rows for a single variable in the Tecplot data file based on the total number of variables and the amount of data per row in the Tecplot data file. The interval division module is used to determine the data order of the Tecplot data file according to the storage format type, and divide the data in the Tecplot data file into multiple variable intervals and one facet interval according to the data order; wherein, the number of rows in each variable interval is the total number of rows of the variable, so that the data in each variable interval is the same variable data, and the number of rows in the facet interval is the total number of rows of the facet. The parallel parsing module is used to synchronously read and transform values ​​for each variable range and surface range using a multi-threaded parallel mechanism, and simultaneously obtain multiple variable arrays and surface index arrays to load Tecplot data files.

8. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images