Data processing optimization method for an archival management system

Abstract

This invention relates to the field of archival data management technology, specifically disclosing a data processing optimization method for archival management systems. First, proprietary format files are collected and converted to a standard format in a runnable native environment to form paired training data. Next, a semantic understanding network composed of feature extraction units and structure generation units is constructed and trained, enabling it to learn to identify semantic units from binary streams and reconstruct structured data. Then, this network is encapsulated as a parsing program independent of the native software environment, capable of directly reading proprietary format files and outputting standard files containing complete geometry, topology, and attributes. The confidence level is calculated by comparing the output with the verification file, and low-confidence samples are fed back to the training set for incremental network training, continuously optimizing parsing capabilities. This invention liberates data from technically locked archives and forms a self-evolving closed-loop optimization system, ensuring the long-term reusability of industrial digital assets.

Description

Technical Field

[0001] This invention relates to the field of archival data management technology, and more specifically to a data processing optimization method for archival management systems. Background Technology

[0002] In professional fields such as engineering, manufacturing, and design, a large amount of core knowledge assets are stored in proprietary electronic file formats generated by specific commercial software (such as CAD and CAE). These archives are key carriers for recording product design, process parameters, and R&D processes, and have extremely high value for long-term preservation and reuse. However, existing document management systems face severe technical challenges and long-term risks when dealing with such proprietary file formats.

[0003] The existing technology has the following shortcomings: When proprietary format files that rely on specific hardware licenses (such as parallel port dongles) and complex software ecosystems become completely obsolete due to the complete demise of their native commercial environment (software obsolescence, hardware lock-in, vendor bankruptcy) and cannot be correctly interpreted by any existing simulation or migration technologies, a problem arises where core engineering data suffers from "logical permanent loss." Traditional methods completely fail in this scenario because their prerequisite (the original, runnable software) no longer exists. Summary of the Invention

[0004] The purpose of this invention is to provide a data processing optimization method for archive management systems to solve the problems mentioned above.

[0005] The objective of this invention can be achieved through the following technical solutions: The data processing optimization method for the archives management system includes the following steps: S1: Collect multiple proprietary format files generated by the target software, and convert each proprietary format file into a corresponding standardized representation file in the runtime environment to form a paired data set consisting of proprietary format files and standardized representation files; S2: Construct a semantic understanding network including a feature extraction unit and a structure generation unit. Train the semantic understanding network using a paired dataset so that the feature extraction unit can identify semantic units from the binary data of the proprietary format file and drive the structure generation unit to reconstruct structured data consistent with the content of the standardized representation file based on the semantic units. S3: The trained semantic understanding network is encapsulated into an independent parsing program. The independent parsing program directly reads the proprietary format file to be processed and outputs the corresponding target structured data file. The target structured data file contains complete geometric elements, topological relationships and engineering attributes. S4: Calculate the processing confidence of the independent parser based on the differences between the target structured data file and the verification file obtained by converting the proprietary format file to be processed through the runtime environment; S5: Process proprietary format files and verification files with confidence levels below a preset threshold as new paired data to supplement the paired data set, and incrementally train the semantic understanding network to update the independent parsing program.

[0006] As a further aspect of the present invention: S2 specifically includes: The feature extraction unit uses a context-aware sliding window to scan the binary data of the proprietary format file. Based on the statistical dependency between adjacent bytes, it dynamically divides and labels continuous data blocks with engineering semantics as semantic units. The structure generation unit receives a sequence of semantic units, maps different types of semantic units to predefined geometric primitive generation rules or parameter constraint generation rules, and synthesizes the initial structured data according to the contextual order between semantic units. A cyclic verification process is performed on the initial structured data. The separable geometric primitives and their parameter constraints are reverse encoded into simulated binary fragments and compared with the original semantic units that triggered the generation of the corresponding data. Based on this, the partitioning strategy of the feature extraction unit and the mapping rules of the structure generation unit are adjusted until the reconstructed structured data is consistent with the content of the standardized representation file.

[0007] As a further aspect of the present invention: the synthesis of initial structured data specifically includes: Receive a sequence of semantic units, and create and maintain an intermediate structure that describes the temporary topological associations between primitives based on the engineering semantic label of each semantic unit and its position in the sequence. Traverse the intermediate structure and match the associated primitive groups with engineering semantic tags to the predefined composite feature generation template. The composite feature generation template defines the derivation logic of the precise geometric relationship and parameter constraints of the primitives in the group. The composite feature generation template is executed. By parsing the geometric relationships and parameter constraints defined in the template, the final spatial coordinates and constraint values ​​of each primitive are calculated and filled in, generating structured data consistent with the content of the standardized representation file.

[0008] As a further aspect of the present invention: S3 specifically includes: The feature parsing is performed on the beginning part of the proprietary format file, and based on the parsed identifiers, a semantic unit extraction strategy and a data reconstruction strategy that match it are selected from a preset strategy set. Based on the selected semantic unit extraction strategy, the binary data of the proprietary format file is read sequentially, the function of the feature extraction unit is dynamically executed, and the semantic unit stream is generated and output in real time. Based on the selected data reconstruction strategy, the driving structure generation unit synchronously receives and sequentially reconstructs the semantic unit stream to generate an initial set of geometric and parametric data. The system performs topological relationship inference and completion on the initial set of geometric and parametric data. Based on the spatial adjacency relationships between geometric features and the logical association of parametric constraints, it reconstructs the missing topological connections and generates a target structured data file that includes complete geometric features, topological relationships, and engineering attributes.

[0009] As a further aspect of the present invention: the generation and output of the semantic unit stream specifically includes: A parsing context is established based on the semantic unit extraction strategy. The parsing context includes the current semantic type identifier, the expected data length, and the temporary data cache. Binary data is read in a stream using fixed-length data blocks. At the same time, the parsing context is used to perform real-time semantic pattern matching and boundary recognition on the data blocks. The successfully matched data segments are labeled with the corresponding semantic type identifier to form the original semantic fragment. When a pattern indicating the end of a semantic unit is identified or when the temporary data cache reaches the expected data length, multiple cached original semantic fragments are merged according to preset rules to generate a complete semantic unit. Perform on-the-fly syntax validation on complete semantic units, check the internal data structure logic of semantic units according to predefined engineering semantic specifications, and immediately add the semantic unit to the semantic unit stream for output after the validation is passed, and reset the parsing context to process subsequent data.

[0010] As a further aspect of the present invention: the formation of the original semantic fragment specifically includes: An adaptive matching window is set in the parsing context, and the length of the adaptive matching window is dynamically adjusted based on the historical data patterns corresponding to the current semantic type identifier. A fixed-length data block is placed into an adaptive matching window, and the matching degree between each data segment in the data block and the feature template corresponding to the preset semantic type identifier is calculated by bit offset. When the matching degree exceeds the dynamic threshold, the data segment boundary is determined, and the validity of the boundary is verified by combining the characteristics of the data bits before and after the data segment boundary. The verified data segments and their start and end positions are marked, and the prediction information about the start position of subsequent semantic units in the parsing context is updated to form the original semantic fragment.

[0011] As a further aspect of the present invention: S4 specifically includes: Geometric features, topological relationships, and engineering attributes are extracted from the target structured data file and the validation file, respectively, to generate the corresponding first feature set and second feature set; A multi-level difference comparison analysis is performed on the first feature set and the second feature set. The spatial position deviation between geometric elements, the connection consistency of topological relationships, and the numerical consistency of engineering attributes are calculated in sequence to generate a set of difference measurement values. Based on the preset complexity level of the proprietary format file, an adaptive weight is assigned to each item in a set of difference measures, and a comprehensive difference index is calculated by weighted fusion. The comprehensive difference index is mapped to a preset confidence interval to obtain the final processing confidence of the independent parsing program for the current file.

[0012] As a further aspect of the present invention: the generation of a set of difference metrics specifically includes: Fast matching based on spatial bounding boxes is performed on the geometric features in the first feature set and the second feature set. For each matched feature pair, the average spatial distance of its surface sampling points is calculated as the spatial position deviation. Based on the matched element pairs, topological adjacency matrices are constructed for the target structured data file and the validation file, respectively. The connection consistency is calculated by comparing the consistency ratio of the connection relationships between corresponding nodes in the two matrices. Based on the matched feature pairs, the engineering attribute values ​​of the corresponding features in the first feature set and the second feature set are compared one by one. The relative error of each attribute value is calculated using a predefined tolerance range, and the average of the relative errors of all attributes is calculated as the numerical consistency. By combining spatial location deviation, connectivity consistency, and numerical consistency, a set of difference measures is generated.

[0013] As a further aspect of the present invention: S5 specifically includes: Analyze and process a set of difference measures corresponding to proprietary format files with confidence scores below a preset threshold, identify the main types of differences, and classify the main types as feature extraction bias or structure generation bias. Based on the classification results, corresponding problem data fragments are extracted from the new paired data, and a targeted fine-tuning training sample set is constructed. The sample set focuses on enhancing the semantic understanding network's ability to correct identified bias types. The semantic understanding network is trained in a targeted manner using a fine-tuned training sample set. Priority is given to adjusting the internal parameters of the feature extraction unit or structure generation unit that are directly related to the type of bias. After training, an updated independent parsing program is generated.

[0014] The beneficial effects of this invention are: (1) This invention constructs a novel data processing path that does not rely on original commercial software and its operating environment (such as dongles or older operating systems) by training a semantic understanding network and encapsulating it into an independent parsing program. The program can directly read the binary data of proprietary format files, automatically and accurately extract and reconstruct the complete geometric elements, topological relationships, and engineering attributes contained therein, and output a standardized structured data file. This fundamentally solves the problem of "readable but unusable" historical archives being lost due to software obsolescence, hardware lock-in, or license expiration, completely releasing the value of archives from the "black box" bound by a specific commercial ecosystem, and ensuring the long-term, independent, and reusable access and utilization capability of core engineering data far beyond the life cycle of software and hardware.

[0015] (2) This invention introduces a feedback optimization mechanism based on confidence assessment and incremental training. The system automatically diagnoses the weaknesses of the current parsing capability (such as feature extraction or structure generation deviations) by quantitatively comparing the differences between the parsing results and the verification samples, and generates targeted training samples accordingly to enhance the semantic understanding network. This closed-loop process enables the independent parsing program to learn autonomously from errors and continuously adapt to the parsing challenges of different versions and types of proprietary format files. Its processing accuracy and robustness can be continuously improved iteratively with the use of the program, realizing the leap from a static "one-time conversion tool" to a dynamic "intelligent growth service", reducing the labor costs and uncertainties of long-term system operation and maintenance and technical adaptation. Attached Figure Description

[0016] The invention will now be further described with reference to the accompanying drawings.

[0017] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1 As shown, this invention is a data processing optimization method for an archive management system, comprising the following steps: S1: Collect multiple proprietary format files generated by the target software, and convert each proprietary format file into a corresponding standardized representation file in the runtime environment to form a paired data set consisting of proprietary format files and standardized representation files; S2: Construct a semantic understanding network including a feature extraction unit and a structure generation unit. Train the semantic understanding network using a paired dataset so that the feature extraction unit can identify semantic units from the binary data of the proprietary format file and drive the structure generation unit to reconstruct structured data consistent with the content of the standardized representation file based on the semantic units. S3: The trained semantic understanding network is encapsulated into an independent parsing program. The independent parsing program directly reads the proprietary format file to be processed and outputs the corresponding target structured data file. The target structured data file contains complete geometric elements, topological relationships and engineering attributes. S4: Calculate the processing confidence of the independent parser based on the differences between the target structured data file and the verification file obtained by converting the proprietary format file to be processed through the runtime environment; S5: Process proprietary format files and verification files with confidence levels below a preset threshold as new paired data to supplement the paired data set, and incrementally train the semantic understanding network to update the independent parsing program.

[0020] In S1, multiple proprietary format files generated by the target software are collected, and each proprietary format file is converted into a corresponding standardized representation file in the runtime environment, forming a paired data set consisting of proprietary format files and standardized representation files, specifically including: The original engineering files generated by the target software are retrieved and extracted from the internal historical data backup system of the organization using the target software. These files are in a proprietary format, the format of which is not publicly available and depends on a specific commercial software environment. The acquisition process must ensure the integrity of the files and, as far as possible, cover different historical versions of the software as well as representative samples created using its different functional modules (such as part design, assembly, engineering drawings, etc.) to build a library of original files with diversity in version and type.

[0021] Subsequently, format conversion is performed in a runtime environment. This runtime environment refers to a legally authorized independent computer system with the corresponding (or compatible) version of the target software and its necessary plugins fully installed. In this environment, each proprietary format file is opened manually or in batches via scripts. Using the software's built-in "Save As" or "Export" functions, its content is completely converted into one or more open, standard intermediate format files. For example, 3D models are exported as STEP-compliant files, engineering drawings are exported as PDF files, and metadata is exported as XML files. These exported files collectively constitute a standardized representation file, designed to retain all geometric, topological, and engineering semantic information of the original file without loss.

[0022] Finally, a paired dataset is formed. Based on the preceding steps, a precise association is systematically established between proprietary format files and their corresponding standardized representation files. Each proprietary format file is uniquely paired with a set of standardized representation files generated from its conversion, and managed through a unified naming convention or index database, thus forming a structured paired dataset for subsequent network training. To ensure data quality, the paired data needs to be sampled and verified before being stored in the database to ensure the correctness of the conversion process and the consistency of the data.

[0023] In S2, a semantic understanding network comprising a feature extraction unit and a structure generation unit is constructed. The semantic understanding network is trained using a paired dataset, enabling the feature extraction unit to identify semantic units from the binary data of a proprietary format file, and driving the structure generation unit to reconstruct structured data consistent with the content of the standardized representation file based on the semantic units. Specifically, this includes: The semantic understanding network consists of a feature extraction unit and a structure generation unit. The feature extraction unit is responsible for parsing segments with independent engineering meaning from the original binary sequence of a proprietary format file. Its specific operation is as follows: the unit sequentially scans the binary data stream using a window of variable length. The initial length of the window is set to a preset value, such as 128 bytes. During the scanning process, the unit dynamically analyzes the statistical characteristics of the byte sequence within the window. These statistical characteristics are characterized by calculating the conditional probability of a byte value at a specific position within the current window appearing in a paired data set with its adjacent byte values. When the conditional probability between multiple consecutive bytes is higher than a preset first threshold, these bytes are determined to belong to the same data entity; when the probability is lower than the threshold, they are considered potential boundaries. Based on this analysis, the unit divides the continuous binary data into blocks and labels each data block with an engineering semantic tag. This tag is assigned from a predefined tag set, such as "file header," "length parameter," "coordinate array start," and "arc definition," thereby generating a sequence of semantic units. The determination of the first threshold is based on offline analysis of the statistical features at the boundaries of known semantic units in the paired dataset, typically taking a quantile of the distribution of these feature values.

[0024] The structure generation unit receives the sequence of semantic units as input. Internally, it stores a set of rules, called the generation rule set, which defines the mapping relationship between different engineering semantic labels and the construction methods of specific geometric primitives (such as points, lines, and arcs) or the assignment methods of engineering parameters (such as length constraints and angle constraints). The unit applies the corresponding generation rules sequentially according to the order of the semantic units in the sequence. For example, when it receives a semantic unit labeled "starting point of a line" and its following "coordinate array" semantic unit containing three floating-point numbers, the unit calls the "create point" rule to generate a vertex; subsequently, when it receives "ending point of a line" and its corresponding coordinate array, it generates another vertex and finally calls the "connect two points to form a line" rule to generate a line primitive and its geometric data. This process progressively generates a structured data draft containing basic primitives and preliminary parameters.

[0025] After obtaining the structured data draft, a cyclical verification process is executed to improve reconstruction accuracy. This process begins by reverse-engineering from the draft: a generated geometric primitive (such as a cylinder defined by two vertices) is selected, and based on its type and parameters, the encoding rules of the original proprietary format are simulated to generate a simulated binary data fragment. Next, this simulated fragment is compared byte-by-byte with the proprietary format binary data corresponding to the original semantic unit that initially triggered the generation of the primitive. The difference between the two is calculated as the proportion of mismatched bytes to the total number of bytes. If the difference exceeds a preset second threshold, it indicates that the current feature extraction partitioning or structure generation mapping may be inaccurate. In this case, the training process adjusts the statistical dependency weights used to partition the data in the feature extraction unit, or corrects the generation rule details corresponding to specific semantic labels in the structure generation unit. Afterward, the adjusted unit is used to reprocess the proprietary format file, generating new structured data, and reverse-encoding and comparison are performed again. This iterative process continues until the difference between the reverse-generated simulated binary fragment and the original data is lower than the second threshold, or the preset maximum number of iterations is reached. At this point, it is considered that the structured data reconstructed by the current semantic understanding network is consistent with the content of the standardized representation file, and the training cycle is complete. The second threshold is set according to the acceptable tolerance for engineering errors, for example, 0.5%.

[0026] In S3, the trained semantic understanding network is encapsulated into an independent parser. This parser directly reads the proprietary format file to be processed and outputs the corresponding target structured data file. The target structured data file contains complete geometric features, topological relationships, and engineering attributes, specifically including: In step S3, feature parsing and strategy selection are performed first. When processing the input proprietary format file, the independent parsing program first reads a preset length of binary data from the beginning of the file, such as the first 512 bytes. This data is parsed to identify whether a specific identifier sequence exists, such as a specific magic number or version number. The program internally has a pre-defined set of strategies, which is a database or list where each record associates a specific identifier sequence with a set of specific processing strategies. This set of strategies includes semantic unit extraction strategies and data reconstruction strategies. By comparing the parsed identifiers with the records in the strategy set, the program selects a uniquely matching extraction and reconstruction strategy to guide the subsequent parsing process. If no exact match is found, a general, statistically based strategy is selected by default.

[0027] After selecting a strategy, the program establishes an initial parsing context based on the semantic unit extraction strategy. The parsing context is a dynamically maintained data structure in memory used to store state information during the parsing process. It mainly includes the identifier of the semantic type currently being parsed (e.g., "coordinates of the center point of an arc"), the expected data length obtained from the strategy based on this type, and a buffer for temporarily accumulating data. The program streams subsequent binary data from the proprietary format file in fixed-length data blocks, with each block's length set, for example, to 102 bytes. Simultaneously with reading each data block, the program initiates a real-time semantic pattern matching and boundary recognition process. The core of this process is the use of an adaptive matching window. The length of this window is not fixed but dynamically adjusted based on the historical data patterns corresponding to the current semantic type identifier. For example, if the variance of the historical data length for the current type is small, the window length is set to 1.2 times the expected data length; if the variance is large, it is set to 2.5 times the expected data length to accommodate data fluctuations.

[0028] When calculating the matching degree within the window, the program uses a bit-by-bit offset scanning method. For a data block entering the window, the program slides a pre-set feature template (a standard binary pattern) corresponding to the current semantic type identifier along the data block for comparison. The matching degree MM is calculated using the following formula: ; in, Indicates the length of the feature template (in bits). This indicates the number of data blocks after offset from the current position. Each bit value. The first feature template Each bit value. It is a comparison function, when equal The function returns 1 if the condition is met, and zero otherwise. It sums the number of bits with the same value at all corresponding positions, then divides by the total template length. , obtain matching degree Its value is between 0 and 1.

[0029] The program sets a dynamic threshold. Used to determine valid matches and boundaries. This dynamic threshold is not a fixed value; its calculation takes into account the local features of the current data block. ;in, This represents the average matching degree calculated from the previous offset positions of the current data block. The standard deviation represents these matching degrees. and These are preset weighting coefficients, typically set to 0.7 and 0.3 respectively. The matching degree calculated at a certain offset position... Exceeding the dynamic threshold at this time At that time, the program initially determined that the location might be the boundary starting point of a semantic unit.

[0030] After initial assessment, the program performs boundary validity verification. The verification method checks whether the data characteristics of several bytes before and after the candidate boundary (e.g., four bytes before and after) conform to common feature patterns of this type of semantic unit boundary, such as specific delimiters or checksums. If they conform, the boundary is deemed valid. The program marks the continuous data from this valid boundary to the point where the expected data length or the next valid ending pattern is identified as a raw semantic segment, records its start and end position information in a file, and stores the segment in a temporary data cache. The program also updates the prediction information for the starting position of the next semantic unit in the parsing context based on the identified boundary position and length.

[0031] When the program recognizes a specific binary pattern (such as a specific end-of-line character sequence) indicating the end of a complete semantic unit, or when the total length of data accumulated in the temporary data cache reaches the expected data length, the program considers that a complete semantic unit data has been collected. At this point, it merges all the original semantic fragments in the cache according to the preset merging rules obtained from the strategy (e.g., directly concatenating them according to positional order, or reassembling them after decoding specific fields) to generate a complete semantic unit. Before outputting, the program performs on-the-fly syntax validation on the complete semantic unit, that is, checks whether its internal data structure logic is reasonable according to the predefined engineering semantic specifications. For example, does a "circular arc" semantic unit contain necessary fields such as the center, radius, starting angle, and ending angle? After successful validation, the program immediately adds the semantic unit as a data item to the semantic unit stream being constructed for output, and resets the parsing context (clears the cache and resets the state) to prepare for processing subsequent binary data.

[0032] Simultaneously with the generation of the semantic unit stream, the structure generation unit receives the data from the stream according to the selected data reconstruction strategy. The reconstruction strategy defines how semantic units of different sequences are combined into geometric primitives and parametric constraints. Following the order in which the semantic units arrive, this unit applies predefined rules in real time to map the semantic units into basic geometric elements such as points, lines, and surfaces, or engineering parameters such as length and angle, gradually constructing an initial set of geometric and parametric data.

[0033] Finally, the program performs topological relationship inference and completion on the initial dataset. Since streaming parsing and sequential reconstruction may not immediately handle all cross-references between features, some topological connections (such as adjacency between faces and loop closure) may be missing. The completion process is based on the spatial adjacency relationships between geometric features and the logical associations of parametric constraints. For example, the bounding boxes of all geometric features are calculated; if the bounding boxes of two features intersect and the average spatial distance between their surface sampling points is less than a tolerance value (set according to 1 / 1000 of the overall model size), a topological connection is inferred between them. For features associated through parametric constraints (such as a hole feature and its location sketch), the association is reconstructed by parsing the logical chain of parametric constraints. Through this process, the program reconstructs all missing topological connections and integrates geometric features, complete topological relationships, and engineering attributes, ultimately generating a target structured data file. This file is encapsulated in standard formats such as STEP and BREP to ensure the integrity and interchangeability of its content.

[0034] In S4, the processing confidence of the independent parser is calculated based on the differences between the target structured data file and the verification file obtained by converting the proprietary format file to be processed through the runtime environment. Specifically, this includes: First, the feature set is extracted and generated. The target structured data file output by the independent parser and the verification file obtained through conversion from the original executable environment are typically standard format files. The program reads these two files respectively, parses their internal structure, and extracts three core types of information: geometric elements, topological relationships, and engineering attributes. Geometric element extraction involves identifying and listing all basic geometric entities, such as vertices, edges, surfaces, and solids, and recording their identifiers and defined parameters (such as point coordinates, edge endpoint references, and surface mathematical expression coefficients). Topological relationship extraction involves parsing the connection and containment relationships between these geometric entities, such as which two vertices connect an edge, which edge loops constitute a surface, and which surfaces enclose a solid, and constructing a relationship list. Engineering attribute extraction involves reading the non-geometric information attached to the geometric elements, such as material name, color code, manufacturing tolerance, and surface roughness. The sum of all the above information extracted from the target structured data file is called the first feature set; the sum of the corresponding information extracted from the verification file is called the second feature set.

[0035] Subsequently, the program performs multi-level difference comparison analysis on the first and second feature sets to generate a set of quantified difference metrics. The first level of analysis is calculating the spatial positional deviation between geometric features. Specifically, the program performs fast matching of geometric features in the two sets based on their bounding boxes. That is, for each geometric feature, a minimum cuboid (bounding box) that can contain all its points is calculated. By comparing the center positions and size similarities of the bounding boxes of geometric features in the two sets, the program finds the most likely corresponding feature in the other set for each geometric feature, forming a matching feature pair. For each pair of matched geometric features, the program regularly samples a certain number of points on its surface (e.g., parameterized uniform sampling of 100 points on a curved surface), calculates the three-dimensional Euclidean distance between the corresponding sampled points on the two features, and then calculates the average of all these distances. This average is recorded as the spatial positional deviation of this pair of features. Finally, the average of the spatial positional deviations of all matching pairs is taken as the overall spatial positional deviation metric.

[0036] The second level of analysis involves calculating the connectivity consistency of topological relationships. Based on the matched feature pairs mentioned above, the program treats the geometric features in the two feature sets as nodes in a graph, and the topological connections between them (such as "an edge connecting vertices A and B") as edges. The program constructs a topological adjacency matrix for each set. The rows and columns of the matrix represent geometric features (nodes). If there is a direct topological connection between two features, the corresponding matrix element is marked with a value of 1; otherwise, it is marked with a value of 0. When constructing the matrix, matched feature pairs use the same node index. The connectivity consistency is calculated by iterating through all corresponding node pairs (i.e., matched feature pairs) in both matrices and counting the number of times they share the same connection state (connected or unconnected). This number is then divided by the total number of comparisons (i.e., the number of node pairs multiplied by the total number of nodes). The resulting ratio is the connectivity consistency metric.

[0037] The third level of analysis involves calculating the numerical consistency of engineering attributes. Based on matched feature pairs, the program compares the engineering attribute values ​​of corresponding features in the first and second feature sets one by one. For each comparable attribute (such as length, angle, density), the program uses a predefined tolerance range. The absolute value of the numerical difference between the attribute in the two sets is calculated and then divided by the attribute's value in the verification file to obtain a relative error. If the relative error is less than or equal to the tolerance range, the attribute is considered consistent; otherwise, the actual relative error is included. After processing all comparable attributes, the program calculates the arithmetic mean of the relative errors of all compared attributes; this mean is used as the numerical consistency measure. If an attribute exists on one side but is missing on the other, it is considered inconsistent, and its error is counted at the maximum value (e.g., 1). Finally, the calculated spatial location deviation measure, connectivity consistency measure, and numerical consistency measure are combined to form the aforementioned set of difference measures.

[0038] Next, the program assigns adaptive weights to the three difference measures based on the preset complexity level of the proprietary format file to be processed, and then performs weighted fusion. The complexity level of the file is pre-defined in step S1 or at the beginning of parsing based on factors such as its size, depth of internal structural hierarchy, and number of feature types, for example, into three levels: "simple," "medium," and "complex." Each level corresponds to a set of preset weight coefficients. For example, for the "simple" level, the weights for spatial position deviation, connectivity consistency, and numerical consistency might be set to 0.5, 0.3, and 0.2, respectively; for the "complex" level, they might be set to 0.3, 0.5, and 0.2, respectively, to emphasize the accuracy of the topological structure. The program selects the corresponding weight coefficients based on the current file's level, multiplies them by the three difference measures calculated in the previous step, and then adds the three products together to obtain a comprehensive difference index. This process is the weighted fusion calculation.

[0039] Finally, the program maps the overall difference index to a preset confidence interval to obtain the final processing confidence score. The preset confidence interval is a numerical range from 0 to 100. The mapping relationship is typically defined by a piecewise linear function. For example, when the overall difference index is less than or equal to a lower threshold (e.g., 0.01), the processing confidence score is set to 100; when the overall difference index is greater than or equal to an upper threshold (e.g., 0.1), the processing confidence score is set to 0; when the overall difference index is between the lower and upper thresholds, the processing confidence score is calculated using linear interpolation. The specific formula is: processing confidence score = 100 minus the overall difference index minus 0.01 (within parentheses), then divided by 0.1 minus 0.01 (within parentheses), and finally multiplied by 100. Through this calculation, the program outputs a specific value between 0 and 100, which serves as the final processing confidence score for the current proprietary format file processed by the independent parsing program. This value is used to quantitatively evaluate the reliability of the parsing result.

[0040] In S5, proprietary format files and verification files with confidence levels below a preset threshold are added to the pairing dataset as new pairing data. The semantic understanding network is then incrementally trained to update the independent parsing program. Specifically, this includes: First, a bias type analysis is performed on the low-confidence results. When the confidence level of the independent parsing program in processing a proprietary format file is lower than a preset threshold (e.g., 80), the program retrieves a set of difference metrics corresponding to the file calculated in step S4, namely, spatial location bias metric, connectivity consistency metric, and numerical consistency metric. The program analyzes the specific values ​​and relative magnitudes of these three metrics to identify the main types of overall differences. The specific judgment rule is as follows: if the spatial location bias metric is significantly higher than the connectivity consistency metric and the numerical consistency metric (e.g., more than twice the average of the latter two), the main difference type is determined to be feature extraction bias, indicating that the parsing program has a systematic error in recognizing basic data such as geometric coordinates in the original binary data. If the connectivity consistency metric is lower than the other two metrics (e.g., lower than 0.85), the main difference type is determined to be structure generation bias, indicating that the parsing program has a logical error in assembling semantic units into a correct topological structure. If the numerical consistency metric is abnormally low, it may involve both types of bias simultaneously, requiring further analysis based on the specific attribute type.

[0041] Based on the classification of the bias types, the program then constructs a targeted fine-tuning training sample set. From the new paired data—specifically, the proprietary format files with low confidence in this processing and the verification files converted from the original runnable environment—the program extracts local data fragments directly related to the identified biases. If classified as feature extraction bias, the program locates the original binary data segments corresponding to the geometric elements in the proprietary format files that are judged to have large spatial location deviations, and simultaneously extracts the precise geometric definitions of these elements from the verification files, collectively forming one or more new "proprietary format fragment-standard geometric definition" paired samples. If classified as structure generation bias, the program extracts the relevant semantic unit sequences that cause inconsistencies in topological connectivity and their correct topological relationship descriptions that should exist in the verification files, forming new "semantic sequence-topological relationship" paired samples. These newly extracted samples are specially labeled and added to the original paired data set, forming a fine-tuning training sample subset focused on correcting the identified bias types. In this subset, the density of samples with similar problems is intentionally increased, for example, by copying or generating similar variants, to enhance the targeting of subsequent training.

[0042] Finally, the program uses the constructed fine-tuned training sample set to perform targeted incremental training on the semantic understanding network. The training process does not start from scratch but builds upon the network's existing internal parameters. The training focus and parameter update strategy depend on the type of bias. For feature extraction bias, the training process keeps the parameters of the structure generation units relatively unchanged, primarily using new samples to adjust the internal weight parameters responsible for coordinate recognition and data boundary determination within the feature extraction units. Its loss function primarily penalizes the difference between the geometric data fields in the semantic units output by the network and the standard values ​​in the validation file. For structure generation bias, the program tends to keep the parameters of the feature extraction units relatively stable, focusing on adjusting the internal logical parameters of the structure generation units responsible for semantic unit sequence parsing and topological relationship derivation. Its loss function primarily penalizes the inconsistency between the reconstructed topological connections and the correct connections recorded in the validation file. This targeted training continues until the prediction error on this fine-tuned training sample subset falls below a preset convergence criterion (e.g., the average error decreases to within 10% of the initial value) or reaches a preset number of iterations (e.g., 1000 times). After training is complete, the program repackages all the parameters of the updated semantic understanding network to generate a new version of the independent parsing program, thereby completing an optimization iteration and improving its processing accuracy and reliability for similar proprietary format files.

[0043] The working principle of this invention is as follows: First, historical proprietary format files generated by the target software are collected and converted into standard format files in batches within the still-running original software environment, forming a "proprietary format-standard format" paired data set. Next, a semantic understanding network composed of feature extraction units and structure generation units is constructed and trained. This network can identify unit sequences with engineering semantics from the binary stream of the proprietary format files and reconstruct structured data consistent with the standard format content based on the sequences. After training, the network is encapsulated as an independent parser. This program can directly read any proprietary format file, bypassing dependence on the original software and its hardware, and output a target structured data file containing complete geometry, topology, and attributes. Subsequently, by comparing the output file with the verification file obtained from the original environment, the processing confidence of the parser is calculated, quantifying its accuracy. Finally, low-confidence files and their verification files are fed back as new samples to the paired data set for targeted incremental training of the semantic understanding network, thereby iteratively optimizing the parsing capability of the independent parser and forming a self-improving closed-loop system. This fundamentally ensures the long-term, accurate, and reusable data value of technical archives after they are removed from their original commercial ecosystem.

[0044] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A data processing optimization method for an archive management system, characterized in that, Includes the following steps: S1: Collect multiple proprietary format files generated by the target software, and convert each proprietary format file into a corresponding standardized representation file in the runtime environment to form a paired data set consisting of proprietary format files and standardized representation files; S2: Construction A semantic understanding network, including a feature extraction unit and a structure generation unit, is trained using a paired dataset. This enables the feature extraction unit to identify semantic units from the binary data of a proprietary format file and drives the structure generation unit to reconstruct structured data consistent with the content of the standardized representation file based on the semantic units. S3: The trained semantic understanding network is encapsulated into an independent parsing program. The independent parsing program directly reads the proprietary format file to be processed and outputs the corresponding target structured data file. The target structured data file contains complete geometric elements, topological relationships and engineering attributes. S4: Calculate the processing confidence of the independent parser based on the differences between the target structured data file and the verification file obtained by converting the proprietary format file to be processed through the runtime environment; S5: Process proprietary format files and verification files with confidence levels below a preset threshold as new paired data to supplement the paired data set, and incrementally train the semantic understanding network to update the independent parsing program.

2. The data processing optimization method for an archive management system according to claim 1, characterized in that, S2 specifically includes: The feature extraction unit uses a context-aware sliding window to scan the binary data of the proprietary format file. Based on the statistical dependency between adjacent bytes, it dynamically divides and labels continuous data blocks with engineering semantics as semantic units. The structure generation unit receives a sequence of semantic units, maps different types of semantic units to predefined geometric primitive generation rules or parameter constraint generation rules, and synthesizes the initial structured data according to the contextual order between semantic units. A cyclic verification process is performed on the initial structured data. The separable geometric primitives and their parameter constraints are reverse encoded into simulated binary fragments and compared with the original semantic units that triggered the generation of the corresponding data. Based on this, the partitioning strategy of the feature extraction unit and the mapping rules of the structure generation unit are adjusted until the reconstructed structured data is consistent with the content of the standardized representation file.

3. The data processing optimization method for an archive management system according to claim 2, characterized in that, The initial structured data for synthesis specifically includes: Receive a sequence of semantic units, and create and maintain an intermediate structure that describes the temporary topological associations between primitives based on the engineering semantic label of each semantic unit and its position in the sequence. Traverse the intermediate structure and match the associated primitive groups with engineering semantic tags to the predefined composite feature generation template. The composite feature generation template defines the derivation logic of the precise geometric relationship and parameter constraints of the primitives in the group. The composite feature generation template is executed. By parsing the geometric relationships and parameter constraints defined in the template, the final spatial coordinates and constraint values ​​of each primitive are calculated and filled in, generating structured data consistent with the content of the standardized representation file.

4. The data processing optimization method for an archive management system according to claim 1, characterized in that, S3 specifically includes: The feature parsing is performed on the beginning part of the proprietary format file, and based on the parsed identifiers, a semantic unit extraction strategy and a data reconstruction strategy that match it are selected from a preset strategy set. Based on the selected semantic unit extraction strategy, the binary data of the proprietary format file is read sequentially, the function of the feature extraction unit is dynamically executed, and the semantic unit stream is generated and output in real time. Based on the selected data reconstruction strategy, the driving structure generation unit synchronously receives and sequentially reconstructs the semantic unit stream to generate an initial set of geometric and parametric data. The system performs topological relationship inference and completion on the initial set of geometric and parametric data. Based on the spatial adjacency relationships between geometric features and the logical association of parametric constraints, it reconstructs the missing topological connections and generates a target structured data file that includes complete geometric features, topological relationships, and engineering attributes.

5. The data processing optimization method for an archive management system according to claim 4, characterized in that, The generation and output of the semantic unit stream specifically includes: A parsing context is established based on the semantic unit extraction strategy. The parsing context includes the current semantic type identifier, the expected data length, and the temporary data cache. Binary data is read in a stream using fixed-length data blocks. At the same time, the parsing context is used to perform real-time semantic pattern matching and boundary recognition on the data blocks. The successfully matched data segments are labeled with the corresponding semantic type identifier to form the original semantic fragment. When a pattern indicating the end of a semantic unit is identified or when the temporary data cache reaches the expected data length, multiple cached original semantic fragments are merged according to preset rules to generate a complete semantic unit. Perform on-the-fly syntax validation on complete semantic units, check the internal data structure logic of semantic units according to predefined engineering semantic specifications, and immediately add the semantic unit to the semantic unit stream for output after the validation is passed, and reset the parsing context to process subsequent data.

6. The data processing optimization method for an archive management system according to claim 5, characterized in that, The formation of the original semantic fragment specifically includes: An adaptive matching window is set in the parsing context, and the length of the adaptive matching window is dynamically adjusted based on the historical data patterns corresponding to the current semantic type identifier. A fixed-length data block is placed into an adaptive matching window, and the matching degree between each data segment in the data block and the feature template corresponding to the preset semantic type identifier is calculated by bit offset. When the matching degree exceeds the dynamic threshold, the data segment boundary is determined, and the validity of the boundary is verified by combining the characteristics of the data bits before and after the data segment boundary. The verified data segments and their start and end positions are marked, and the prediction information about the start position of subsequent semantic units in the parsing context is updated to form the original semantic fragment.

7. The data processing optimization method for an archive management system according to claim 1, characterized in that, S4 specifically includes: Geometric features, topological relationships, and engineering attributes are extracted from the target structured data file and the validation file, respectively, to generate the corresponding first feature set and second feature set; A multi-level difference comparison analysis is performed on the first feature set and the second feature set. The spatial position deviation between geometric elements, the connection consistency of topological relationships, and the numerical consistency of engineering attributes are calculated in sequence to generate a set of difference measurement values. Based on the preset complexity level of the proprietary format file, an adaptive weight is assigned to each item in a set of difference measures, and a comprehensive difference index is calculated by weighted fusion. The comprehensive difference index is mapped to a preset confidence interval to obtain the final processing confidence of the independent parsing program for the current file.

8. The data processing optimization method for an archive management system according to claim 7, characterized in that, The generation of a set of difference metrics specifically includes: Fast matching based on spatial bounding boxes is performed on the geometric features in the first feature set and the second feature set. For each matched feature pair, the average spatial distance of its surface sampling points is calculated as the spatial position deviation. Based on the matched element pairs, topological adjacency matrices are constructed for the target structured data file and the validation file, respectively. The connection consistency is calculated by comparing the consistency ratio of the connection relationships between corresponding nodes in the two matrices. Based on the matched feature pairs, the engineering attribute values ​​of the corresponding features in the first feature set and the second feature set are compared one by one. The relative error of each attribute value is calculated using a predefined tolerance range, and the average of the relative errors of all attributes is calculated as the numerical consistency. By combining spatial location deviation, connectivity consistency, and numerical consistency, a set of difference measures is generated.

9. The data processing optimization method for an archive management system according to claim 1, characterized in that, S5 specifically includes: Analyze and process a set of difference measures corresponding to proprietary format files with confidence scores below a preset threshold, identify the main types of differences, and classify the main types as feature extraction bias or structure generation bias. Based on the classification results, corresponding problem data fragments are extracted from the new paired data, and a targeted fine-tuning training sample set is constructed. The sample set focuses on enhancing the semantic understanding network's ability to correct identified bias types. The semantic understanding network is trained in a targeted manner using a fine-tuned training sample set. Priority is given to adjusting the internal parameters of the feature extraction unit or structure generation unit that are directly related to the type of bias. After training, an updated independent parsing program is generated.

Images