An INP file parsing method

Abstract

The application discloses an INP file analysis method. Wherein, the method comprises: fusing the first block size predicted by the machine learning model and the second block size calculated based on the feedback formula through the trust factor to obtain the third block size; dividing the first INP file into the third data block and the second INP file, identifying the complete data block and the separated data from the third data block according to the lightweight boundary identification model, splicing the separated data and the second INP file to form a new first INP file; analyzing the complete data block in parallel and storing the final analysis result; repeating the above process until the size of the first INP file is less than or equal to the third block size, at this time, the first INP file as a whole is analyzed as a complete data block; the final analysis results of all complete data blocks are merged into the complete analysis result of the INP file for storage. The application solves the problems of low analysis efficiency and poor analysis accuracy of the existing INP file analysis technology.

Description

Technical Field

[0001] This application relates to the field of simulation software data processing technology, and more specifically, to a method for parsing INP files. Background Technology

[0002] INP files, a standard input file format widely used in finite element analysis (FEA), typically contain model information such as nodal coordinates, element connectivity, material properties, and boundary conditions. As engineering models become increasingly complex, INP file sizes can reach GB or even TB levels, making efficient and stable parsing a critical factor in simulation analysis.

[0003] However, existing INP file parsing technologies primarily rely on static block partitioning, which pre-determines fixed-size data blocks based on experience. This method cannot adapt to the complexity of the file's internal structure, easily leading to unreasonable block sizes and low efficiency. Furthermore, it ignores dynamically changing system environments, making it prone to memory overflows or parsing stutters under resource constraints. In addition, this method lacks an online feedback mechanism, failing to dynamically adjust the strategy based on actual parsing performance. In other words, existing INP file parsing technologies face the problems of low parsing efficiency and poor parsing accuracy.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides an INP file parsing method to at least solve the problems of low parsing efficiency and poor parsing accuracy faced by existing INP file parsing technologies.

[0006] According to one aspect of the embodiments of this application, an INP file parsing method is provided, comprising: Step 1: obtaining a first INP file and extracting a first attribute feature from the first INP file; Step 2: inputting the first attribute feature into a trained machine learning model to predict a first block size; Step 3: calculating a second block size based on a feedback formula; Step 4: fusing the first block size and the second block size using a confidence factor to obtain a third block size, wherein the confidence factor is dynamically updated based on the prediction bias rate of the machine learning model and the prediction bias rate of the feedback formula; Step 5: if the size of the first INP file is larger than the size of the third block, dividing the first INP file into a third data block and a second INP file, identifying complete data blocks and separated data from the third data block using a lightweight boundary recognition model, and concatenating the separated data with the second INP file to form an updated first INP file, wherein the third data block... The size is the same as the third block size mentioned above; Step 6: Divide the complete data block into N complete data units according to the identifier, start M parallel execution threads, assign the N complete data units to the M parallel execution threads for parsing to obtain N parsing results, and concatenate the N parsing results according to the order of the corresponding complete data units in the complete data block to generate the final parsing result of the complete data block, where N is an integer greater than or equal to 1, M is less than or equal to N, and one parallel execution thread parses at least one complete data unit; Step 7: Repeat steps S1 to S6 until the size of the first INP file is less than or equal to the third block size; Step 8: When the size of the first INP file is less than or equal to the third block size, determine the first INP file as a complete data block and execute step S6 for parsing; Step 9: Merge the final parsing results of all the complete data blocks into the complete parsing result of the INP file and store it.

[0007] Optionally, in this embodiment, step 1 includes: obtaining the first INP file; querying the size of the first INP file and the amount of available physical memory in the system; determining the ratio of the total number of rows in the first INP file to the size of the first INP file as data density; obtaining the parsing difficulty level based on a lightweight classifier and obtaining resource stability through a time series model; and determining the size of the first INP file, the amount of available physical memory in the system, the data density, the parsing difficulty level, and the resource stability as the first attribute feature.

[0008] Optionally, in this embodiment, obtaining the parsing difficulty level based on the lightweight classifier includes: acquiring the second attribute features of the first INP file, wherein the second attribute features include data density, unit type proportion, and line spanning rate, wherein the data density represents the proportion of valid data in a single first INP file, the unit type proportion represents the proportion of each type of unit in a single first INP file, and the line spanning rate represents the ratio of the number of data rows of the line spanning unit to the total number of rows; inputting the second attribute features into the lightweight classifier that has been fine-tuned and trained; and the lightweight classifier outputting the parsing difficulty level of the first INP file, wherein the parsing difficulty level is divided into three categories: high, medium, and low, and quantified as values ​​of 1.2, 1.0, and 0.8, respectively.

[0009] Optionally, in this embodiment of the application, obtaining resource stability through a time series model includes: collecting the CPU utilization rate of the system within a preset time period at a specified interval to form a first time series; collecting the memory occupancy rate of the system within the preset time period at the specified interval to form a second time series; inputting the first time series and the second time series into the time series model after fine-tuning and training; and outputting the resource stability from the time series model, wherein the resource stability is quantified as a score of 0-10.

[0010] Optionally, in this embodiment of the application, step 3 above includes: Wherein, FB_Size_i represents the size of the second block of the i-th third data block of the INP file, FinalSize_{i-1} is the size of the third block of the (i-1)-th third data block of the INP file, T_{i-1} is the actual time spent parsing the (i-1)-th third data block of the INP file, and T_target is the preset target time spent.

[0011] Optionally, in this embodiment of the application, step 4 above includes: calculating the size of the third block using the collaborative formula of the collaborative model: FinalSize=λ×PredictSize_ML+(1-λ)×FB_Size, where FinalSize represents the size of the third block, PredictSize_ML represents the size of the first block, FB_Size represents the size of the second block, and λ is the trust factor.

[0012] Optionally, in the embodiments of the present application, the update rule of the above-mentioned confidence factor λ is as follows: when d_ml < d_fb: λ_new = λ_old + μ×(1 - d_ml / d_fb); when d_ml ≥ d_fb: λ_new = λ_old - μ×(d_ml / d_fb); where d_ml represents the deviation rate between the actual time taken to parse the first data block and the above-mentioned preset target time, the size of the above-mentioned first data block is the above-mentioned first block size, d_fb represents the deviation rate between the actual time taken to parse the second data block and the above-mentioned preset target time, the size of the above-mentioned second data block is the above-mentioned second block size, λ_old is the current above-mentioned confidence factor, λ_new is the updated above-mentioned confidence factor, μ is the increment step size, and λ_new is between the preset minimum value and the preset maximum value.

[0013] Optionally, in the embodiments of the present application, the above-mentioned step 5 includes: when the size of the above-mentioned first INP file is greater than the above-mentioned third block size, extract a third data block with a size of the above-mentioned third block size from the starting position of the above-mentioned first INP file, and determine the remaining part of the above-mentioned first INP file after removing the above-mentioned third data block as the above-mentioned second INP file, where the above-mentioned third block size includes at least one complete data unit;

[0014] Calculate the adaptive safety distance based on the following formula:

[0015] δ = base_δ × type_weight × bias_coeff

[0016] where δ represents the above-mentioned adaptive safety distance, base_δ represents the base distance, the above-mentioned base distance is determined based on the average line length, type_weight represents the type weight, the above-mentioned type weight is calculated based on the proportion of each unit type in the above-mentioned third data block and the predefined line number coefficient, bias_coeff represents the deviation correction coefficient, and the above-mentioned deviation correction coefficient is dynamically adjusted based on the historical missed detection rate; input the above-mentioned adaptive safety distance of the above-mentioned third data block into the above-mentioned lightweight boundary recognition model after fine-tuning training, and the above-mentioned lightweight boundary recognition model identifies the end position of the last complete data unit in the above-mentioned third data block; when the end position of the last complete data unit in the above-mentioned third data block is different from the end position of the above-mentioned third data block, separate the data between the end position of the last complete data unit in the above-mentioned third data block and the end position of the above-mentioned third data block from the above-mentioned third data block to obtain the above-mentioned complete data block and the above-mentioned separated data, where the above-mentioned complete data block and the above-mentioned separated data are concatenated to form the above-mentioned third data block; splice the above-mentioned separated data to the starting position of the above-mentioned second INP file to form the updated above-mentioned first INP file.

[0017] Optionally, in this embodiment, the machine learning model is also periodically updated: when the complete data block is parsed, the parsing performance of the complete data block is evaluated; when the parsing performance of the complete data block is higher than the expected threshold, the parsing data of the complete data block is recorded, wherein the parsing data includes the size of the first INP file, the amount of available physical memory in the system, the data density, the parsing difficulty level, the resource stability, and the size of the third block; the parsing data is identified as a new sample for the machine learning model and stored in the replay buffer; when the parsing data in the replay buffer reaches a preset threshold, the machine learning model is updated with small-step parameters based on the parsing data in the replay buffer; the parsing data in the replay buffer is cleared so that the machine learning model is updated based on non-repeating parsing data each time.

[0018] In this embodiment, a first INP file is obtained, and a first attribute feature of the first INP file is extracted; the first attribute feature is input into a trained machine learning model to predict the first block size; the second block size is calculated based on a feedback formula; the first block size and the second block size are fused using a confidence factor to obtain a third block size; if the size of the first INP file is larger than the size of the third block, the first INP file is divided into a third data block and a second INP file; a lightweight boundary recognition model is used to identify complete data blocks and separated data from the third data block; the separated data is then concatenated with the second INP file to form an updated... The first INP file is divided into N complete data units. M parallel execution threads are started to parse the N complete data units, resulting in N parsing results. The N parsing results are then concatenated according to the order of their corresponding complete data units in the complete data block to generate the final parsing result for storage. This process is repeated until the size of the first INP file is less than or equal to the size of the third block. If the size of the first INP file is less than or equal to the size of the third block, the first INP file is determined as a complete data block and parsed. The final parsing results of all complete data blocks are merged into the complete parsing result of the INP file for storage. In other words, this application dynamically adjusts the size of the third data block online through a machine learning model, feedback formula, and trust factor, so that the block size of the INP file takes into account file complexity and system resource environment. At the same time, the third data block is adjusted to a complete data block containing complete data units based on a lightweight boundary recognition model. In addition, multiple parallel execution threads are used to parse the complete data block in parallel. This improves the parsing accuracy and efficiency of INP files, thereby solving the problems of poor parsing accuracy and low parsing efficiency faced by existing INP file parsing methods. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0020] Figure 1 This is a schematic diagram of an optional INP file parsing method provided according to an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of another optional INP file parsing method according to an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of another optional INP file parsing method according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of another optional INP file parsing method according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of another optional INP file parsing method according to an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of another optional INP file parsing method according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] Alternatively, as an optional implementation scheme, such as Figure 1 As shown, the above-mentioned INP file parsing method includes:

[0029] S102, Obtain the first INP file and extract the first attribute feature of the first INP file;

[0030] S104, Input the first attribute feature into the trained machine learning model to predict the size of the first block;

[0031] S106, Calculate the size of the second block based on the feedback formula;

[0032] S108, the size of the first block and the size of the second block are merged using a trust factor to obtain the size of the third block;

[0033] S110, Determine whether the size of the first INP file is greater than the size of the third block;

[0034] S110-1, if the size of the first INP file is larger than the size of the third block, the first INP file is divided into the third data block and the second INP file. The complete data block and the separated data are identified from the third data block according to the lightweight boundary recognition model. The separated data is concatenated with the second INP file to form the updated first INP file.

[0035] S110-1-1: Divide the complete data block into N complete data units according to the identifier, start M parallel execution threads, assign the N complete data units to the M parallel execution threads for parsing to obtain N parsing results, and concatenate the N parsing results according to the order of the corresponding complete data units in the complete data block to generate the final parsing result of the complete data block.

[0036] S110-2, if the size of the first INP file is less than or equal to the size of the third block, the entire first INP file is determined as a complete data block, and S110-1-1 is executed for parsing;

[0037] S112, merge the final parsing results of all complete data blocks into a complete parsing result of the INP file and store it.

[0038] Optionally, in this embodiment, the INP file can be an INP file exported by software such as ABAQUS or HyperMesh, and the INP file can come from different engineering scenarios. The INP file size refers to the amount of space occupied by the file on the storage medium, usually expressed in bytes (Byte) and its derivative units (KB, MB, GB, etc.). The available physical memory refers to the amount of unused physical memory (RAM) that the operating system can directly allocate to applications or system processes at a specific time. A complete data unit refers to a data block or record that cannot be further divided during the parsing process and has independent semantics. Data density is defined as the total number of data rows in the INP file divided by the size of the INP file, used to measure the parsing complexity within a unit of data. When the INP file is large, the data density can be estimated based on samples from the beginning of the INP file, such as reading the first 1000 rows and calculating the number of rows divided by the amount of data. The identifier can be a delimiter. A complete data block refers to a data block in which all data units are complete data units, such as a Node or Element. A complete parsing result refers to the parsing result containing all data in the INP file.

[0039] For example, the complete data block is divided into 9 complete data units according to the identifier, and 3 parallel execution threads (parallel execution thread A, parallel execution thread B, and parallel execution thread C) are started. The 1st, 4th, and 7th complete data units are assigned to parallel execution thread A for parsing, the 2nd, 5th, and 8th complete data units are assigned to parallel execution thread B for parsing, and the 3rd, 6th, and 9th complete data units are assigned to parallel execution thread C for parsing. At the start of parsing, parallel execution threads A, B, and C simultaneously perform the first round of parsing, meaning the 1st, 2nd, and 3rd complete data units are parsed concurrently (thread A parses the 1st complete data unit, thread B parses the 2nd, and thread C parses the 3rd). After thread A completes parsing the 1st complete data unit, thread B completes parsing the 2nd, and thread C completes parsing the 3rd, the second round of parsing begins, meaning the 4th, 5th, and 6th complete data units are parsed concurrently (thread A parses the 4th, thread B parses the 5th, and thread C parses the 6th). After thread A completes parsing the 3rd complete data unit, thread B completes parsing the 4th, and thread C completes parsing the 5th, and thread C completes parsing the 6th, the second round of parsing begins. After parsing the 5th complete data unit, the C parallel execution thread begins the third round of parsing, that is, the 7th, 8th and 9th complete data units are parsed simultaneously (the A parallel execution thread parses the 7th complete data unit, the B parallel execution thread parses the 8th complete data unit, and the C parallel execution thread parses the 9th complete data unit). After the A parallel execution thread parses the 7th complete data unit, the B parallel execution thread parses the 8th complete data unit, and the C parallel execution thread parses the 9th complete data unit, the nine parsing results are concatenated according to the order of the corresponding complete data units in the complete data block. That is, the parsing result corresponding to the 1st complete data unit is concatenated with the parsing result corresponding to the 2nd complete data unit, the parsing result of the 2nd complete data unit is concatenated with the parsing result corresponding to the 3rd complete data unit, and so on, to obtain the final parsing result of the complete data block.

[0040] If the complete data block is divided into 8 complete data units based on the identifier, and 3 parallel execution threads (thread A, thread B, and thread C) are started, the 1st, 4th, and 7th complete data units are assigned to thread A for parsing, the 2nd, 5th, and 8th complete data units are assigned to thread B for parsing, and the 3rd and 6th complete data units are assigned to thread C for parsing. The parsing method and process for the first two rounds are consistent with the above. In the last round (i.e., the third round) of parsing, thread C has no complete data units to parse, so thread C remains idle. The 7th and 8th complete data units begin parsing simultaneously (thread A parses the 7th complete data unit, and thread B parses the 8th complete data unit).

[0041] For example, the process involves obtaining the INP file to be parsed and using it as the initial value for the first INP file. The first attribute feature of the first INP file is input into a machine learning model trained offline. The parameters of this model have already been obtained through offline training. The machine learning model predicts the size of the first block. Then, the size of the second block is calculated based on a feedback formula. The first and second block sizes are weighted and fused using a confidence factor to obtain the size of the third block. If the size of the first INP file is larger than the size of the third block, the first INP file is divided into a third data block and a second INP file. The size of the third data block is the size of the third block, and the size of the second INP file is the size of the first INP file minus the size of the third block. Based on the lightweight boundary recognition model trained offline, complete data blocks containing complete data units and separated data (incomplete data units) are identified from the third data block. The separated data is then concatenated to the beginning of the second INP file to form the updated first INP file. The complete data block is divided into N complete data units. M parallel execution threads are used to parse these N complete data units, where M is less than or equal to N. Each parallel execution thread parses at least one complete data unit. The parsing results of each parallel execution thread are then merged into the final parsing result of the complete data block and stored. This process is repeated until the size of the first INP file is less than or equal to the size of the third block. If the size of the first INP file is less than or equal to the size of the third block, the entire first INP file is considered a complete data block, and the parsing process with the M parallel execution threads is executed again. Finally, the final parsing results of all complete data blocks are merged into the final parsing result of the remaining INP file and stored.

[0042] Optionally, in this embodiment of the application, when the final parsing results of all complete data blocks are finally merged into the complete parsing results of the INP file for storage, the merging order of the final parsing results is based on the order of the complete data blocks in the INP file.

[0043] In this embodiment, a first INP file is obtained, and first attribute features of the first INP file (first INP file size, available physical memory, data density, parsing difficulty level, and resource stability) are extracted. These first attribute features are then input into a trained machine learning model to predict the first block size. The second block size is calculated based on a feedback formula. The first and second block sizes are fused using a confidence factor to obtain a third block size. If the first INP file size is larger than the third block size, the first INP file is divided into a third data block and a second INP file. A lightweight boundary recognition model is used to identify complete data blocks and separated data from the third data block. The separated data is then combined with… The second INP file is concatenated to form the updated first INP file; the complete data block is divided into N complete data units, and M parallel execution threads are used to parse the N complete data units to obtain N parsing results. The N parsing results are concatenated according to the order of their corresponding complete data units in the complete data block to generate the final parsing result of the complete data block; the above process is repeated until the size of the first INP file is less than or equal to the size of the third block; if the size of the first INP file is less than or equal to the size of the third block, the first INP file as a whole is determined as a complete data block, and parsing is performed by M parallel execution threads; the final parsing results of all complete data blocks are merged into the complete parsing result of the INP file and stored. In other words, the INP file parsing method proposed in this application not only considers the complexity of the file's internal structure (such as data density and parsing difficulty level), but also the current system resource environment (such as the amount of available physical memory and resource stability). It dynamically adjusts the block size according to the feedback formula and the trust factor, so that the block size combines the actual situation of the INP file and the actual situation of the resource environment. At the same time, it uses M parallel execution threads to parse the third data block, which ensures both efficiency and accuracy when parsing the data blocks divided from the INP file.

[0044] Based on one or more of the above embodiments, this application further improves upon them, such as... Figure 2 As shown, step 1 includes:

[0045] S202, Obtain the first INP file;

[0046] S204, query the size of the first INP file and the amount of physical memory available in the system;

[0047] S206, The ratio of the total number of lines in the first INP file to the size of the first INP file is determined as the data density;

[0048] S208: The parsing difficulty level is obtained based on a lightweight classifier, and the resource stability is obtained through a time series model.

[0049] S210 defines the size of the first INP file, the amount of available physical memory in the system, the data density, the parsing difficulty level, and the resource stability as the first attribute features.

[0050] Optionally, in this embodiment, the parsing difficulty level of the first INP file is obtained based on the lightweight classifier after fine-tuning training; the resource stability of the first INP file is obtained based on the time series model after fine-tuning training; the ratio of the total number of lines in the first INP file to the size of the first INP file is determined as the data density of the first INP file; and the available physical memory of the system and the size of the first INP file are checked according to specified operation steps. The first INP file size, available physical memory of the system, data density, parsing difficulty level, and resource stability obtained above are used as the first attribute features.

[0051] Based on one or more of the above embodiments, this application further improves upon them, such as... Figure 3 As shown, the parsing difficulty levels obtained from the lightweight classifier include:

[0052] S302, Obtain the second attribute characteristics of the first INP file;

[0053] S304, input the second attribute feature into the lightweight classifier that has been fine-tuned and trained;

[0054] S306, the difficulty level of parsing the first INP file output by the lightweight classifier.

[0055] Optionally, in this embodiment, data density is defined as the proportion of valid data in the first INP file or the amount of valid data per unit storage / transmission space. For example, in an INP data block storing 1000 node coordinates, if only 200 lines are coordinate values ​​and the remaining 800 lines are comments and blank lines, the data density is considered low. The higher the data density, the less redundant content needs to be skipped during parsing, resulting in lower memory usage and read time. The unit type proportion refers to the proportion of various types of units (such as C3D8, C3D20, S4R) in the current first INP file, represented by a vector (such as [0.6, 0.3, 0.1]). Because the parsing complexity (number of rows, number of fields) of different units varies significantly, it directly affects the block partitioning strategy. The row span rate is defined as "the number of data rows of the row spanning unit / the total number of data rows", specifically characterizing the row spanning degree of variable-length units, solving the defect of traditional DataDensity in not being able to distinguish between "dense single-row units" and "sparse row spanning units". In addition, the second attribute feature can also include field complexity, which is the standard deviation of the number of fields in each data row (e.g., the number of fields in a Node unit is fixed, and the standard deviation is 0; the number of fields in an Element unit fluctuates greatly, and the standard deviation is large), reflecting the field parsing overhead during data parsing. The parsing difficulty level Complexity is divided into three categories: high, medium, and low (high = 1.2, medium = 1.0, low = 0.8).

[0056] Based on one or more of the above embodiments, this application further improves upon them, such as... Figure 4 As shown, obtaining resource stability through time series models includes:

[0057] S402-1, Collect the CPU utilization rate of the system within a preset time period according to the specified interval time to form a first time series;

[0058] S402-2, Collect the system's memory usage rate within a preset time period according to a specified interval to form a second time series;

[0059] S404, Input the first and second time series into the time series model that has been fine-tuned and trained;

[0060] S406, Time Series Model Outputs Resource Stability.

[0061] Optionally, in this embodiment, resource stability (Stability) is a quantitative score of the stability of computing resources (CPU and memory), with a score range of 0-10, where a higher score indicates greater stability.

[0062] Based on one or more of the above embodiments, this application further improves upon the previous one by including step 3 as follows:

[0063] (2)

[0064] Wherein, FB_Size_i represents the size of the second block of the i-th third data block of the INP file, FinalSize_{i-1} is the size of the third block of the (i-1)-th third data block of the INP file, T_{i-1} is the actual time taken to parse the (i-1)-th third data block of the INP file (e.g., the time to achieve 90% CPU utilization), and T_target is the preset target time (e.g., the time to achieve 90% CPU utilization).

[0065] Based on one or more of the above embodiments, this application further improves upon the previous one by including step 4 as follows:

[0066] The size of the third block mentioned above is calculated using the collaborative formula (3) of the collaborative model:

[0067] FinalSize=λ×PredictSize_ML+(1-λ)×FB_Size (3)

[0068] Where FinalSize represents the size of the third block, PredictSize_ML represents the size of the first block, FB_Size represents the size of the second block, and λ is the confidence factor.

[0069] Based on one or more of the above embodiments, this application further improves upon the previous one by defining the update rule for the trust factor λ as follows:

[0070] d_ml <d_fb:λ_new=λ_old + μ×(1-d_ml / d_fb);

[0071] d_ml≥d_fb: λ_new=λ_old - μ×(d_ml / d_fb);

[0072] Where d_ml represents the deviation rate between the actual time spent parsing the first data block and the preset target time T_target, the size of the first data block is the size of the first block, d_fb represents the deviation rate between the actual time spent parsing the second data block and the preset target time T_target, the size of the second data block is the size of the second block, λ_old is the current confidence factor, λ_new is the updated confidence factor, μ is the increment step size, and λ_new is between the preset minimum and the preset maximum value. The preset minimum and the preset maximum value can be values ​​defined based on experience.

[0073] Based on one or more of the above embodiments, this application further improves upon them, such as... Figure 5 As shown, step 5 includes:

[0074] S502, if the size of the first INP file is greater than the size of the third block, extract a third data block with a size of the third block from the beginning of the first INP file, and determine the remaining part of the first INP file after removing the third data block as the second INP file;

[0075] S504, Calculate the adaptive safety distance based on formula (4):

[0076] δ = base_δ × type_weight × bias_coeff (4)

[0077] Where δ represents the adaptive safety distance, base_δ represents the base distance, which is determined based on the average row length, type_weight represents the type weight, which is calculated based on the proportion of each cell type in the third data block and the predefined row number coefficient, and bias_coeff represents the bias correction coefficient, which is dynamically adjusted based on the false negative rate of historical detections.

[0078] S506, the adaptive safety distance of the third data block is input into the fine-tuned lightweight boundary recognition model, and the lightweight boundary recognition model identifies the end position of the last complete data unit in the third data block.

[0079] S508, if the end position of the last complete data unit in the third data block is different from the end position of the third data block, the data between the end position of the last complete data unit in the third data block and the end position of the third data block is separated from the third data block to obtain the complete data block and the separated data.

[0080] S510, the separated data is concatenated to the beginning of the second INP file to form the updated first INP file.

[0081] Optionally, in this embodiment, the starting position of the first INP file refers to the very beginning of the data in the first INP file, and the third data block is a data block of a third size divided from the starting position of the data in the first INP file. The data in the third data block and the data in the second INP file together constitute the data of the first INP file. The average line length refers to the average number of characters (including spaces, punctuation, etc.) in all lines of the first INP file. For large first INP files, the average line length can be estimated based on 5% of the first INP file. The ending position of the data block refers to the very end of the data block.

[0082] Based on one or more of the above embodiments, this application further improves upon them, such as... Figure 6 As shown, the machine learning model is also updated regularly:

[0083] S602, Evaluate the parsing performance of the complete data block after it has been parsed;

[0084] S604: If the parsing performance of the complete data block is higher than the expected threshold, record the parsing data of the complete data block.

[0085] S606, The parsed data is identified as a new sample for the machine learning model and stored in the replay buffer;

[0086] S608, when the parsed data in the replay buffer reaches a preset threshold, the machine learning model is updated with small steps based on the parsed data in the replay buffer.

[0087] S610, clear the parsed data in the replay buffer so that the machine learning model is updated based on non-repeating parsed data each time.

[0088] Optionally, as an alternative implementation, the above method is explained and illustrated by the following steps:

[0089] Part 1: Data Preparation

[0090] Obtain INP files of any size and type, and use these INP files as the initial first INP file. Query the first INP file's file size (FileSize) and the current system's available physical memory (AvailableMemory). Determine the first INP file's data density (DataDensity) by the ratio of the total number of rows to its size. Obtain the first INP file's parsing difficulty level (Complexity) and the current system's resource stability (Stability). Use these five dimensions—first INP file size, available physical memory, data density, parsing difficulty level, and resource stability—as the first attribute feature. Use the optimal block size manually labeled for the first INP file as the first true label (TrueSize). Use these first attribute features as the first input features of the machine learning model. Construct a first dataset based on the first input features and the first true label of the first INP file. Each first INP file corresponds to one first input feature and one first true label. The first dataset includes the first input features and first true labels corresponding to multiple first INP files.

[0091] The parsing difficulty level of the first INP file is determined by a finely tuned lightweight classifier. Features such as data density, cell type ratio, line span rate, and field complexity of the first INP file are input into the trained lightweight classifier, which outputs whether the parsing difficulty level of the first INP file is high, medium, or low. The fine-tuning training process of the lightweight classifier includes: using the data density, cell type ratio, line span rate, and field complexity of each first INP file as the second input features of the lightweight classifier; labeling each first INP file with its parsing difficulty level using annotation tools or manually as the second true label; using the second input features and the second true label as the second dataset for the lightweight classifier; and fine-tuning the lightweight classifier based on this second dataset. Training stops when the number of iterations is reached or the metric reaches a preset threshold, thus outputting the finely tuned lightweight classifier. The lightweight classifier can be a neural network model such as logistic regression, MobileNet series, ShuffleNet series, SqueezeNet, GhostNet, etc.

[0092] The resource stability of the first INP file is determined using a finely tuned time series model. The time series model (such as LSTM or GRU) analyzes the fluctuation trends of CPU utilization and memory usage over the past 10 seconds, outputting a resource stability score (0-10). Two time series datasets are generated: CPU utilization (the percentage of CPU usage collected at intervals (e.g., 0.1 seconds) and memory usage (the percentage of memory usage collected at the same time intervals). Each dataset contains approximately 100 data points (10 seconds ÷ 0.1 seconds). These are then input into the finely tuned time series model (such as an LSTM recurrent neural network), and the LSTM will output a resource stability score (e.g., 6). The training process of the time series model includes: using two time series formed by CPU utilization and memory usage as the third input features; assigning a third true label to the system resources in that time period (e.g., 10 seconds) based on annotation tools or manually; the third true label is the resource stability score; using the third input features and the third true label as the third dataset of the time series model; fine-tuning the time series model based on this third dataset; stopping training when the number of iterations is reached or the index reaches a preset threshold; and thus outputting the fine-tuned time series model.

[0093] Part Two: Model Building

[0094] 1. Building machine learning models (ML models)

[0095] Build a machine learning model to predict an optimal initial block size (i.e., the first block size) before parsing the first INP file.

[0096] 2. Establish a feedback formula

[0097] The calculation of the feedback value FB_Size_i is based on control theory, and the calculation method is as shown in formula (2).

[0098] For example: Suppose the system's preset target time is T_target = 5 seconds, and the size of the third block of the (i-1)th third data block in the INP file is FinalSize_{i-1} = 100MB.

[0099] Scenario A: The system takes approximately 5 seconds to process a 100MB third data block (CPU maintains 90% utilization). Therefore, the actual processing time T_{i-1}_A ≈ 5 seconds.

[0100] Scenario B: The system takes approximately 10 seconds to process a 100MB third data block. Therefore, the actual processing time is T_{i-1}_B ≈ 10 seconds.

[0101] For scenario A (T_{i-1}_A ≈ T_target):

[0102] Formula calculation: FB_Size_i = 100MB * (5 / 5) = 100MB

[0103] Conclusion: The block size is appropriate, and it should remain unchanged at 100MB.

[0104] For scenario B (T_{i-1}_B = 10 seconds > T_target)

[0105] Formula calculation: FB_Size_i = 100MB * (5 / 10) = 50MB,

[0106] Conclusion: The feedback indicates that the actual block size used for the (i-1)th third data block of the INP file is too large, causing slow processing. The block size of the i-th third data block of the INP file should be reduced. That is, the actual processing time of 10 seconds is greater than the target processing time of 5 seconds. It is recommended to reduce the block size of the i-th third data block of the INP file to 50MB. Explanation: The feedback formula "discovered" that the previous processing took too long (10 seconds). It believes this is because the blocks are too large, resulting in insufficient work completed per unit time (despite CPU activity). To complete the task within the target time, it needs to improve "efficiency." Here, it attempts to achieve this by reducing the block size, as smaller blocks generally mean better load balancing and faster single-block processing speed (although this may increase some management overhead). The feedback formula aims to find a better block size (i.e., the second block size) so that when processing future third data blocks, regardless of their size, the actual processing time is close to T_target.

[0107] Without historical parsing records, the feedback formula cannot provide effective suggestions. Therefore, the initial state of FB_Size_1 can be set to a conservative default value (e.g., 512KB).

[0108] 3. Construct a collaborative model

[0109] The ML model provides the optimal initial value (first block size), but in actual parsing, fine-tuning is performed by combining online feedback, forming a two-layer optimization structure of "ML coarse-tuning + feedback fine-tuning".

[0110] The size of the first block of the i-th third data block is PredictSize_ML_i, which is predicted by the ML model. The adjusted value calculated by the feedback formula after parsing is the size of the second block, FB_Size_i. Then, according to formula (3), the optimized block size, i.e., the size of the third block, can be obtained.

[0111] The update rules for λ are as follows:

[0112] T_ml_test = Actual time spent parsing the first data block # The size of the first data block is the size of the first chunk

[0113] T_fb_test = Actual time spent parsing the second data block. # The size of the second data block is the size of the second sub-block. When the first data block and / or the second data block are relatively large, only 5% of the data in the first data block and 5% of the data in the second data block can be parsed for calculation and comparison.

[0114] d_ml = |T_ml_test - T_target| / T_target # Bias rate of the machine learning model

[0115] d_fb = |T_fb_test - T_target| / T_target # Prediction bias rate of the feedback formula

[0116] If d_ml < d_fb: this means the ML prediction is closer to the target than the feedback prediction.

[0117] λ_new = λ_old + μ * (1 - d_ml / d_fb) # The better the ML, the more you add.

[0118] If d_ml ≥ d_fb: this means the feedback prediction is more accurate than the ML prediction.

[0119] λ_new = λ_old - μ * (d_ml / d_fb) # The worse the ML, the more it is reduced.

[0120] Constraints: λ_new ∈ [λ_min, λ_max], typically [0.1, 0.95], maintaining a minimum confidence level (e.g., 0.1) to prevent system crashes when ML completely fails, and limiting the maximum confidence level (e.g., 0.95) to maintain the fine-tuning capability of the feedback mechanism. λ can initially be set to a low value (e.g., 0.1), indicating that the size of the third block initially relies more on conservative default rules or simple trial and error than on the predictions of the machine learning model. As the system stabilizes, the confidence in the ML model's predictions can be gradually increased. The closer λ value is to 1.0, the more the system trusts the ML model's predictions.

[0121] When the confidence factor λ stabilizes above 0.9, it indicates that the PredictSize_ML output by the ML model is very close to the theoretical optimal value. The role of the feedback value FB_Size changes from "large-scale correction" to "fine-tuning and calibration" to cope with instantaneous fluctuations in system load (such as other programs suddenly occupying CPU / memory). μ is a small increment (such as 0.05).

[0122] Synergistic effect:

[0123] When λ is small, decisions are primarily based on feedback values ​​calculated using the feedback formula, and the system behaves like a sensitive controller, quickly responding to real-time performance fluctuations. As λ increases, decisions become primarily based on machine learning (ML) predictions, and the system behaves like an experienced expert, making near-optimal decisions in advance.

[0124] Feedback formulas always play the role of "corrective guardians". Even if ML predictions occasionally go wrong, feedback formulas can bring them back on track through real-time feedback, preventing serious performance degradation.

[0125] Three: Model Training

[0126] 1. Initialization of machine learning model parameters

[0127] The initialization of ML model parameters adopts a hierarchical initialization strategy to ensure the stability and rapid convergence of the training process.

[0128] (1) Experience-guided initialization: Based on historical data analysis, set reasonable initial value ranges for each parameter.

[0129] (2) Numerical stability guarantee: The weight coefficients are initialized with a uniform distribution of Xavier to ensure stable gradient propagation.

[0130] 2. Machine Learning Model Training Methods

[0131] The first dataset was divided into training, validation, and test sets in a 6:2:2 ratio. The Huber loss function and Adam optimizer were used to train the ML model.

[0132] 3. Machine learning model updates

[0133] After successfully parsing a complete data block, evaluate the parsing performance (e.g., parsing speed). If the parsing performance exceeds the expected threshold, record the parsed data of the complete data block as a six-tuple (FileSize, AvailableMemory, DataDensity, ...).

[0134] Complexity, Stability, FinalSize). New samples identified from the parsed data for use in machine learning models are stored in a fixed-size replay buffer within the system.

[0135] Every so often (e.g., after parsing 10 complete data blocks), the system evaluates recent records. If the parsing performance of a particular complete data block is below the expected threshold, the block size used in that parsing (i.e., the third block size, FinalSize) is considered suboptimal. By performing local performance interpolation or fine-tuning search near the features of that complete data block, a new block size is estimated as a correction label. The size of the first INP file, the amount of available physical memory in the system when parsing the first INP file, the data density of the first INP file, the parsing difficulty level of the first INP file, the resource stability when parsing the first INP file, and the new block size are stored as new samples for the machine learning model in a fixed-size replay buffer.

[0136] Periodically (e.g., after collecting N new samples or when the amount of new sample data reaches a specified threshold), sample a specified proportion of data from the replay buffer. Update the machine learning model using one or more steps of gradient descent based on the sampled data, setting the learning rate to 1 / 10 of the offline training learning rate (small step size to prevent oscillation). Use the remaining sample data in the replay buffer as the validation and test sets to evaluate the performance of the updated machine learning model. If performance improves, replace the original machine learning model; otherwise, retain the original model. After the update, the buffer can be cleared or some high-quality samples can be retained for parameter learning using new samples or mostly new samples in the next machine learning model update.

[0137] 4: Boundary Detection

[0138] To address the issue of data unit truncation in INP files that may result from dynamic segmentation, this application proposes an intelligent boundary detection algorithm.

[0139] 1. Adaptive safety distance δ design:

[0140] In this application, δ is no longer a single value, but a dynamic value that integrates multi-dimensional features. The formula for calculating δ is as follows:

[0141] δ=base_δ×type_weight×bias_coeff

[0142] The base distance is calculated as: base_δ = average row length × 2. Type weight is calculated based on the proportion of cell types in the third data block. The core logic is that the higher the proportion of complex types, the greater the weight, i.e., type_weight = Σ(type proportion × type row number coefficient), where the "type row number coefficient" is predefined as NODE=1, C3D8=2, C3D20=3, S4R=2, C3D4=1, C3D6=2, C3D8R=2, C3D10=3 (the more rows, the larger the coefficient). The bias correction coefficient is dynamically adjusted based on historical detection bias. The core logic is that the more missed detections, the larger the coefficient, bias_coeff = 1 + (cumulative missed detections / total detections) × 0.5 (the cumulative missed detections are the number of times incomplete data appears outside δ in the last 100 detections; the upper limit of the coefficient is set to 2 to avoid δ becoming too large).

[0143] 2. Train a lightweight boundary recognition model:

[0144] (1) Data preparation

[0145] We collected INP files from various engineering scenarios, covering core data unit types such as NODE, C3D8, C3D20, S4R, C3D4, C3D6, C3D8R, and C3D10, with file sizes ranging from 100MB to 10GB to ensure data diversity. We simulated a real parsing process, dividing the collected INP files using a dynamic block-splitting strategy (random blocks from 1KB to 100KB) to obtain a large number of data block samples. For each data block sample, we extracted the last character sequence as the input sample for a lightweight boundary recognition model. The sequence length covered the possible range of the adaptive safety distance δ (50-500 characters) to ensure coverage of detection requirements under different δ scenarios.

[0146] For each extracted sequence of last characters, a label is manually applied. The label is defined as "the offset of the end position of the last complete data unit relative to the last character sequence", where:

[0147] NODE type: A single NODE occupies 1 line, and the label is the offset corresponding to the last newline character in the last character sequence;

[0148] C3D8 type: A single C3D8 occupies 2 lines, and the label is the offset corresponding to the last "non-comma-ending line" in the last character sequence (the first line of C3D8 ends with a comma, and the last line does not end with a comma).

[0149] C3D20 type: A single C3D20 occupies 3 lines, and the label is the offset corresponding to the last "non-comma-ending line" in the last character sequence;

[0150] C3D4 type: A single C3D4 occupies 1 line, and the label is the offset corresponding to the last newline character in the last character sequence;

[0151] C3D6 type: A single C3D6 occupies 1 line, and the label is the offset corresponding to the last newline character in the last character sequence;

[0152] C3D8R type: A single C3D8R occupies 2 lines, and the label is the offset corresponding to the last "non-comma-ending line" in the last character sequence (the first line of C3D8R ends with a comma, and the last line does not end with a comma).

[0153] C3D10 type: A single C3D10 occupies 3 lines, and the label is the offset corresponding to the last "non-comma-ending line" in the last character sequence (the first and second lines of C3D10 end with a comma, and the last line does not end with a comma).

[0154] The collected end-character sequence samples and their corresponding labels were standardized to ensure compatibility with the input requirements of the lightweight boundary recognition model. First, character encoding was performed. Since the core characters in the INP file only include a limited number of types such as numbers, commas, newlines, and spaces, a character-to-number mapping table (e.g., ASCII code mapping) was constructed to convert the character sequences into quantifiable number sequences, reducing input dimensionality. Then, sequence standardization was performed, unifying the length of all input samples to 500 characters (covering the maximum possible δ value). Samples shorter than 500 characters were padded with zeros at the end (empty character mapping value), and samples longer than 500 characters had the last 500 characters truncated. Next, label normalization was performed, dividing the boundary offset by the sequence length to convert it into floating-point labels of 0-1, adapting to the model's output range and reducing training difficulty. Finally, the data was partitioned, with the preprocessed samples divided into training, validation, and test sets in an 8:1:1 ratio. The training set was used for model parameter learning, the validation set for hyperparameter tuning during training, and the test set for final model performance evaluation.

[0155] (2) Lightweight boundary recognition model architecture design

[0156] The lightweight boundary recognition model adopts a minimalist architecture of "embedding layer + lightweight convolutional layer + pooling layer + minimal fully connected layer". The core idea is to extract local features of character sequences (such as line ending characters and line break patterns) with minimal network parameters to predict boundary positions, as detailed below:

[0157] Embedding layer: Maps the encoded number sequence into a low-dimensional dense vector, aiming to capture the semantic relationships between different characters (such as the relationship between commas, newlines, and boundaries). The input is a 500-dimensional number sequence, and the output is a 500×8 low-dimensional vector (the embedding dimension is set to 8 to balance feature expressiveness and model size).

[0158] Convolutional layers: 1D convolutional kernels are used to extract local features of the character sequence (such as boundary-related feature patterns like "number + comma" and "newline + number"). Sixteen 3×8 convolutional kernels are set (8 input channels, 16 output channels), and a padding strategy is used to ensure that the sequence length remains unchanged after convolution, avoiding feature loss.

[0159] Activation function: The ReLU activation function is adopted to introduce non-linear feature expression capabilities and enhance the model's ability to fit complex sequence patterns;

[0160] Pooling layer: Max pooling is used with a kernel size of 2 to compress the length of the convolutional feature sequence to 250, reducing subsequent computation while retaining key features;

[0161] Fully connected layer: Flattens the pooled 250×16-dimensional feature vector into a 4000-dimensional vector, and maps it to a floating-point number of 0~1 through one output node, corresponding to the normalized boundary offset;

[0162] Output layer: The Sigmoid activation function is used to ensure that the output value is in the range of 0 to 1, which matches the normalized label.

[0163] When designing the model architecture, the following techniques were used to reduce the parameter size of the lightweight boundary recognition model: First, parameter quantization, converting the lightweight boundary recognition model parameters from 32-bit floating-point numbers to 16-bit floating-point numbers. Second, pruning optimization, pruning redundant weights in convolutional and fully connected layers (pruning ratio 10%), removing weights that contribute very little to feature extraction, while ensuring that the accuracy of the lightweight boundary recognition model is not affected.

[0164] (3) Training of lightweight boundary recognition model

[0165] During training, the mean squared error loss function is used, which is suitable for regression prediction tasks with boundary offsets and can effectively measure the deviation between predicted values ​​and true labels. The Adam optimizer is used with a learning rate of 0.001, adaptively adjusting the learning rate to accelerate training convergence and avoid local optima. The batch size is set to 32 to balance training efficiency and gradient stability. Ten training epochs are used, and the model performance is evaluated using a validation set after each epoch. If the loss on the validation set does not decrease for two consecutive epochs, training is stopped early to avoid overfitting. The weights of convolutional and fully connected layers are initialized using a random normal distribution, while the weights of embedding layers are initialized using a uniform distribution, thus initializing the model parameters.

[0166] (4) Model reasoning

[0167] Obtain the adaptive safety distance δ of the third data block, and truncate the last character sequence of length δ from the end of the third data block (if the data block length is less than δ, truncate the entire data block). Perform the same preprocessing operations (character encoding, sequence normalization) as in the training phase on the truncated character sequence, transforming it into an input format acceptable to the lightweight boundary recognition model. Input the preprocessed input sequence into the lightweight boundary recognition model to obtain normalized boundary offset prediction values. Multiply the normalized boundary offset prediction value by the actual length of the truncated character sequence to obtain the position of the boundary offset within the character sequence, and finally transform it into the boundary position relative to the third data block, outputting the end position of the last complete data unit in the third data block for subsequent incomplete data stripping and splicing. If the end position of the last complete data unit in the third data block differs from the end position of the third data block, separate the data between the end position of the last complete data unit and the end position of the third data block from the third data block to obtain complete data blocks and separated data. Splice the separated data to the beginning position of the second INP file to form the updated first INP file. Boundary recognition was implemented for the last character sequence of the third data block.

[0168] 5. Parallel Analysis

[0169] For the complete data block containing several complete Node / Element data units after boundary detection, M parallel execution threads are used to parse N complete data units in the complete data block, with each parallel execution thread parsing at least one complete data unit. If the complete data block contains only one type of complete data unit, the following three-level parallel architecture can be used for parsing:

[0170] Level 1: Pipeline Producer. This involves simply splitting and extracting the string containing information about a complete data unit from a complete data block, and placing it into an ordered container (e.g., a std::vector). <std::string>`lines` contains all the data in a complete data block. Each `std::string` in the container contains information about a complete data unit (such as a Node or Element, and only one). Each piece of data (`std::string`) in the container forms a "task package".

[0171] Level 2: Consumer Parallel Processing. In this stage, based on the "task packages" from Level 1, each task package is parsed and constructed into structured in-memory data. The following optimization strategies are used to optimize this parallel program:

[0172] 1. Use a static scheduling strategy. Since each task package has a consistent data structure and uniform processing time, static scheduling is used. The task allocation is determined at compile time, and the loop iterations are evenly distributed to each thread, eliminating the need for dynamic allocation at runtime, thereby reducing the overhead of thread scheduling and thread switching.

[0173] 2. Set a reasonable number of threads. Setting a reasonable number of threads avoids exceeding the number of CPU cores, preventing excessive threading that leads to context switching and improving cache hit rate. It avoids "thread explosion," which causes "CPU resource contention and a precipitous performance drop," while also avoiding too few threads, which wastes resources. For example, a reasonable number of threads can be set based on the number of available processors in the system.

[0174] 3. Use thread-private data structures. Create a private data structure for each thread to store the structured in-memory data constructed by that thread, avoiding critical sections. Thread-private data structures can fully utilize caching, avoiding cache invalidation issues. They also avoid frequent lock acquisition when adding data due to critical section contention, as well as frequent thread waiting and frequent thread context switching during lock contention.

[0175] 4. Optimize string processing. Avoid time-consuming operations in parallel loops. For example, use a more efficient string splitting method.

[0176] 5. Memory pre-allocation. Since each task package contains the same amount of data, the size of its storage structure can be pre-allocated to avoid multiple memory allocations.

[0177] Level 3: Safe Reduction and Merging. Each parallel execution thread temporarily stores the parsed data in its own thread-private data structure. When the task package is parsed, the thread-private data is merged into the global data structure in batches outside the parallel processing program. This "local first, global later" reduction strategy greatly reduces the contention overhead in critical sections and improves data writing efficiency.

[0178] In other words, this patent application first divides a complete data block from the INP file for parallel parsing (using parallel execution threads). After parsing, another complete data block is divided from the remaining data in the INP file for parallel parsing, and this process is repeated until all data in the INP file has been parsed. The complete data block is identified from the third data block using a lightweight boundary recognition model. The size of the third data block is the size of the third sub-block, which may vary each time it is divided, as it is dynamically adjusted based on factors such as the complexity of the current data, the system environment, and the parsing performance of the previous third sub-block. Because each generated complete data block is processed sequentially, and a new complete data block is generated only after the previous one has been parsed, the process repeats. This ensures that only one data block exists at any given time, preventing data block accumulation and further increasing the upper limit of the INP file size that the program can process.

[0179] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application. In other words, the implementation methods or embodiments of this disclosure are not exhaustive, but merely illustrative of some implementation methods or embodiments, and are not intended to limit the scope of protection of this disclosure. Where there is no contradiction, each step in a certain implementation method or embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, the solution after removing some steps in a certain implementation method or embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain implementation method or embodiment can be arbitrarily interchanged. Furthermore, optional methods or examples in a certain implementation method or embodiment can be arbitrarily combined; in addition, various implementation methods or embodiments can be arbitrarily combined. For example, some or all steps of different implementation methods or embodiments can be arbitrarily combined, and a certain implementation method or embodiment can be arbitrarily combined with optional methods or examples of other implementation methods or embodiments.

[0180] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the INP file parsing method provided in any of the foregoing embodiments when it is run.

[0181] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the INP file parsing method provided in any of the foregoing embodiments through the computer program.

[0182] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.< / std::string>

Claims

1. A method for parsing INP files, characterized in that, include: Step 1: Obtain the first INP file and extract the first attribute features of the first INP file. The first attribute features include the size of the first INP file, the amount of available physical memory in the system, the data density, the parsing difficulty level, and the resource stability. Step 2: Input the first attribute feature into the trained machine learning model to predict the size of the first block; Step 3: Calculate the size of the second block based on the feedback formula, which is as follows: Wherein, FB_Size_i represents the second block size of the i-th third data block of the INP file, FinalSize_{i-1} is the third block size of the (i-1)-th third data block of the INP file, T_{i-1} is the actual time spent parsing the (i-1)-th third data block of the INP file, and T_target is the preset target time. Step 4: Merge the first block size and the second block size using a trust factor to obtain the third block size. The trust factor is dynamically updated based on the prediction bias rate of the machine learning model and the prediction bias rate of the feedback formula. The prediction bias rate of the machine learning model is the deviation rate between the actual time taken to parse the first data block and the preset target time. The prediction bias rate of the feedback formula is the deviation rate between the actual time taken to parse the second data block and the preset target time. Step 5: If the size of the first INP file is larger than the size of the third block, the first INP file is divided into a third data block and a second INP file. The end position of the last complete data unit in the third data block is identified according to the lightweight boundary recognition model to identify the complete data block and the separated data from the third data block. The separated data is concatenated with the second INP file to form the updated first INP file, wherein the size of the third data block is the size of the third block. Step 6: Divide the complete data block into N complete data units according to the identifier, start M parallel execution threads, assign the N complete data units to the M parallel execution threads for parsing to obtain N parsing results, and concatenate the N parsing results according to the order of the corresponding complete data units in the complete data block to generate the final parsing result of the complete data block, where N is an integer greater than or equal to 1, M is less than or equal to N, and each parallel execution thread parses at least one complete data unit; Step 7: Repeat steps 1 to 6 until the size of the first INP file is less than or equal to the size of the third block; Step 8: If the size of the first INP file is less than or equal to the size of the third block, the entire first INP file is determined as the complete data block, and step 6 is executed for parsing; Step 9: Merge the final parsing results of all the complete data blocks into the complete parsing result of the INP file and store it.

2. The method according to claim 1, characterized in that, Step 1 includes: Obtain the first INP file; Query the size of the first INP file and the amount of available physical memory in the system; The ratio of the total number of lines in the first INP file to the size of the first INP file is determined as the data density. The difficulty level of the analysis is obtained based on a lightweight classifier, and the resource stability is obtained through a time series model. The size of the first INP file, the amount of available physical memory in the system, the data density, the parsing difficulty level, and the resource stability are determined as the first attribute feature.

3. The method according to claim 2, characterized in that, The method of obtaining the parsing difficulty level based on the lightweight classifier includes: Obtain the second attribute features of the first INP file, wherein the second attribute features include data density, unit type proportion, and line spanning rate. The data density is used to represent the proportion of valid data in a single first INP file, the unit type proportion is used to represent the proportion of each type of unit in a single first INP file, and the line spanning rate is used to represent the ratio of the number of data rows of the line spanning unit to the total number of rows. The second attribute feature is input into the lightweight classifier that has been fine-tuned and trained. The lightweight classifier outputs the parsing difficulty level of the first INP file, wherein the parsing difficulty level is divided into three categories: high, medium, and low, and quantified as values ​​of 1.2, 1.0, and 0.8, respectively.

4. The method according to claim 2, characterized in that, The method of obtaining resource stability through time series models includes: The CPU utilization rate of the system within a preset time period is collected at specified intervals to form a first time series. The second time series is formed by collecting the memory usage rate of the system within the preset time period based on the specified interval time. The first time series and the second time series are input into the time series model that has been fine-tuned and trained; The time series model outputs the resource stability, wherein the resource stability is quantified as a score of 0-10.

5. The method according to claim 1, characterized in that, Step 4 includes: The size of the third block is calculated using the collaborative formula of the collaborative model: Where FinalSize represents the size of the third block, PredictSize_ML represents the size of the first block, FB_Size represents the size of the second block, and λ is the trust factor.

6. The method according to claim 5, characterized in that, The update rule for the trust factor λ is as follows: d_ml <d_fb:λ_new=λ_old + μ×(1-d_ml / d_fb); d_ml≥d_fb: λ_new=λ_old - μ×(d_ml / d_fb); Where d_ml represents the deviation rate between the actual time spent parsing the first data block and the preset target time spent, the size of the first data block is the size of the first sub-block, d_fb represents the deviation rate between the actual time spent parsing the second data block and the preset target time spent, the size of the second data block is the size of the second sub-block, λ_old is the current trust factor, λ_new is the updated trust factor, μ is the increment step size, and λ_new is between the preset minimum value and the preset maximum value.

7. The method according to claim 1, characterized in that, Step 5 includes: If the size of the first INP file is greater than the size of the third block, a third data block of the size of the third block is extracted from the starting position of the first INP file, and the remaining part of the first INP file after removing the third data block is determined as the second INP file, wherein the size of the third block includes at least one complete data unit; The adaptive safety distance is calculated based on the following formula: δ = base_δ × type_weight × bias_coeff Wherein, δ represents the adaptive safety distance, base_δ represents the base distance, which is determined based on the average row length, type_weight represents the type weight, which is calculated based on the proportion of each unit type in the third data block and a predefined row number coefficient, and bias_coeff represents the bias correction coefficient, which is dynamically adjusted based on the historical false negative rate. The lightweight boundary recognition model, which is fine-tuned by inputting the adaptive safety distance of the third data block into the trained model, identifies the end position of the last complete data unit in the third data block. If the end position of the last complete data unit in the third data block is different from the end position of the third data block, the data between the end position of the last complete data unit in the third data block and the end position of the third data block is separated from the third data block to obtain the complete data block and the separated data. The complete data block and the separated data are then concatenated to form the third data block. The separated data is then appended to the beginning of the second INP file to form the updated first INP file.

8. The method according to claim 1, characterized in that, The machine learning model is also updated regularly: Once the complete data block has been parsed, evaluate the parsing performance of the complete data block; If the parsing performance of the complete data block is higher than the expected threshold, the parsing data of the complete data block is recorded, wherein the parsing data includes the size of the first INP file, the amount of available physical memory in the system, the data density, the parsing difficulty level, the resource stability, and the size of the third block; The parsed data is identified as a new sample for the machine learning model and stored in the replay buffer. When the parsed data in the replay buffer reaches a preset threshold, the machine learning model is updated with small-step parameters based on the parsed data in the replay buffer. The parsed data in the playback buffer is cleared so that the machine learning model is updated based on non-repeating parsed data each time.

Images