Traffic engineering multi-modal pre-training data enhancement method based on multi-dimensional semantic reconstruction

Abstract

The application discloses a traffic engineering multi-modal pre-training data enhancement method based on multi-dimensional semantic reconstruction, comprising the following steps: obtaining an original traffic engineering specification PDF document, generating a basic semantic unit set; generating a hierarchical semantic path, a table attribute chain, a formula variable mapping and a compliance label; constructing a traffic engineering specification heterogeneous semantic graph, generating a node set, a relation set and an edge set, and writing an edge attribute; generating an initial node vector and an initial relation vector; using a compliance gate CompGCN for weighted node-relation combination propagation to generate a relation enhanced semantic representation; generating a pre-training task sample through masking reconstruction; and packaging an output JSONL format dataset. The application improves the structural integrity, semantic accuracy and compliance expression capability of the traffic engineering specification corpus.

Description

Technical Field

[0001] This invention relates to the field of intelligent parsing technology for traffic engineering specification documents, and in particular to a method for augmenting multimodal pre-trained data of traffic engineering based on multidimensional semantic reconstruction. Background Technology

[0002] Traffic engineering specifications are mostly published in PDF format, containing chapters, clauses, engineering parameter tables, calculation formulas, and mandatory provisions. Existing document parsing technologies primarily employ OCR recognition, layout analysis, table extraction, and formula recognition to convert specifications into text, tables, or LaTeX expressions, which are then used to construct a large language model pre-training corpus. While these methods can extract basic content, the results often remain at the level of text blocks, table blocks, and formula blocks, failing to adequately express the hierarchical constraints, parameter attributions, formula interpretations, and compliance requirements within traffic engineering specifications.

[0003] Current technologies still have shortcomings in the analysis of traffic engineering specifications. After OCR segmentation, clauses are easily fragmented into discrete text slices, making it difficult to preserve the hierarchical relationships between chapters, sections, articles, and clauses, resulting in a lack of applicability of the learned clauses. Complex tables contain multi-level headers and merged cells, and linear extraction results can easily detach numerical values ​​from row headers, table headers, and engineering parameters. Formula recognition largely focuses on LaTeX expression extraction, and formula variables lack stable binding with explanatory text such as "wherein" or "wherein". Mandatory clauses rely on font features and safety keywords to express compliance weights; ordinary corpus construction processes struggle to distinguish between mandatory requirements, recommended requirements, and general clauses, affecting the learning effectiveness of traffic engineering models in terms of specification logic, engineering parameters, and compliance boundaries.

[0004] Therefore, how to provide a method for augmenting multimodal pre-trained data in traffic engineering based on multidimensional semantic reconstruction is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a method for augmenting multimodal pre-training data in traffic engineering based on multidimensional semantic reconstruction. This invention combines hierarchical reconstruction of traffic regulations, backtracking of table attributes, binding of formula interpretations, and CompGCN propagation of compliance gating to generate enhanced pre-training samples, which have the advantages of clear semantic relationships, accurate parameter correspondence, stable formula understanding, and strong ability to express compliance constraints.

[0006] The traffic engineering multimodal pre-training data augmentation method based on multidimensional semantic reconstruction according to embodiments of the present invention includes the following steps: Obtain the original traffic engineering specification PDF document and generate a set of basic semantic units; Based on the set of basic semantic units, hierarchical semantic paths, table attribute chains, formula variable mappings, and compliance tags are generated; A heterogeneous semantic graph of traffic engineering specifications is constructed based on hierarchical semantic paths, table attribute chains, formula variable mappings, and compliance labels. A set of nodes, a set of relationships, and a set of edges are generated. OCR confidence, hierarchical distance, formula interpretation distance, compliance weight, and relationship priority are written to the edge set. Encode the set of nodes and the set of relations, and generate initial node vectors and initial relation vectors; Based on OCR confidence, hierarchical distance, formula interpretation distance, compliance weight, and relationship priority, compliance gating propagation coefficients are generated. The initial node vector, initial relationship vector, edge set, and compliance gating propagation coefficients are input into the compliance gating CompGCN to perform weighted node-relation combination propagation and generate relationship-enhanced semantic representations. Based on relation-enhanced semantic representation, masking reconstruction is performed, masking the last-level path field in the hierarchical semantic path, masking the row constraint relationship and column constraint relationship of the numerical node connection, masking the variable definition relationship and unit binding relationship of the variable node connection, masking the mandatory constraint relationship and compliance weight relationship of the clause node connection, and generating pre-trained task samples. Encapsulate the original text, relation-enhanced semantic representation, and pre-trained task samples to output a JSONL format dataset.

[0007] Optionally, the generation process of the hierarchical semantic path includes: Select semantic units from the title area and the body area from the basic semantic unit set, and generate a text slice sequence according to the page number and vertical coordinate; Using a traffic regulation numbering regularity library, chapter-level, section-level, sub-section-level, and clause-level numbering recognition is performed on text slice sequences to generate numbering level labels; Indentation coordinates are extracted based on the bounding box coordinates of semantic units in the title area and semantic units in the body area. Clustering is then performed on the indentation coordinates to generate indentation level labels. Merge the numbered hierarchical labels and the indented hierarchical labels to generate hierarchical labels; Based on the hierarchical label changes of adjacent text slices in the text slice sequence, backtrack to determine the corresponding superior chapter number, superior section number, superior article number, and superior clause number for each text slice; The hierarchical semantic path is generated by sequentially concatenating the superior chapter number, superior section number, superior article number, superior clause number, and text slice identifier.

[0008] Optionally, the process of generating the table attribute chain includes: Select semantic units for the table area from the basic semantic unit set, and construct a table cell matrix based on the semantic units for the table area; Record the text content, row index, column index, row merging range, column merging range, and cell type of each table cell in the table cell matrix; Use a table cell with the cell type "data cell" as the starting point for backtracking; Starting from the data cell, search upwards row by row for the column header cells with overlapping column indices, and generate the list header path in order from the outer header to the inner header; Starting from the data cell, search the row header cells with overlapping row indices column by column to the left, and generate the row header path in the order from the outer header to the inner header; Concatenate the list header path, row header path, and data cell text content into an "attribute-value" key-value pair; Sort all "attribute-value" key-value pairs within the same table area according to row index and column index to generate a table attribute chain.

[0009] Optionally, the process of generating the formula variable mapping includes: Select semantic units from the formula area and the main text area from the basic semantic unit set; Extract the LaTeX expression, page number, and formula coordinates of the semantic units in the formula area; Using the lower boundary of the formula coordinates as the search starting point, search downwards within the same page for semantic units in the main text area containing "wherein", "of which", and "symbol explanation"; If no semantic unit in the main text area is found, adjust the search range to 400 pixels; Starting from the semantic unit in the retrieved main text area, extract the explanatory text continuously according to the vertical coordinate, with the extraction endpoint being the first position between the blank line position and the next title area semantic unit position; Variable symbol extraction is performed on the LaTeX expression to generate a set of formula variables; Perform physical definition extraction and unit extraction on the explanatory text to generate a set of explanatory items; Based on the order in which the variable symbols in the formula variable set appear in the LaTeX expression, the variable symbols are bound to the physical definitions and units in the episody set to generate the formula variable mapping.

[0010] Optionally, the process of generating the compliance label includes: Read text slices, font visual features, and text content from a set of basic semantic units; Extract text slices with bold font visual features and text content containing "must", "strictly prohibited", or "must not" from text slices to generate mandatory clause labels; Assign a compliance weight of 1 to the mandatory clause label; Extracting text slices in which the font visual feature is not boldface and the text content contains "Yi", "Ke" and "Tui Jian" from the text slices, and generating recommendation clause labels; Assigning a compliance weight of 0.3 to the recommendation clause label; Marking text slices that have not been generated with mandatory clause labels nor recommendation clause labels as general clause labels, and assigning a compliance weight of 0 to the general clause labels; Writing the mandatory clause labels, recommendation clause labels, general clause labels and compliance weights into compliance labels; Binding the compliance labels to text slice identifiers in the hierarchical semantic path.

[0011] Optionally, the construction process of the heterogeneous semantic graph for traffic engineering specifications includes: Generating chapter nodes, section nodes, clause nodes and text slice nodes based on hierarchical semantic paths; Generating table header nodes, engineering parameter nodes and value nodes based on table attribute chains; Generating formula nodes, variable nodes, interpretation nodes and unit nodes based on formula variable mapping; Generating compliance label nodes and compliance weight nodes based on compliance labels; Writing chapter nodes, section nodes, clause nodes, text slice nodes, table header nodes, engineering parameter nodes, value nodes, formula nodes, variable nodes, interpretation nodes, unit nodes, compliance label nodes and compliance weight nodes into the node set; Writing belonging relations, inclusion relations, upper-level constraint relations, row constraint relations, column constraint relations, attribute-value corresponding relations, formula reference relations, variable definition relations, unit binding relations, mandatory constraint relations and compliance weight relations into the relation set; Connecting corresponding nodes according to node source semantic unit identifiers and relation types to generate an edge set.

[0012] Optionally, the edge attribute writing process of the edge set includes: Determining each edge in the edge set as an edge , the edge connects the start node , the relation and the target node ; Writing the smaller value of the OCR confidence degrees of the basic semantic units corresponding to the start node and the target node into the OCR confidence degree of the edge ; ; Writing the start node and the target node Write edges in the path hierarchy of the hierarchical semantic path. Hierarchical distance ; Divide the vertical pixel distance between the semantic unit in the formula area and the explanatory text by the average line height of the main text on the page to generate the edge. Formula explanation of distance ; Side The compliance weights in the corresponding compliance labels of the connected text slices are written into the edges. Compliance weight ; Prioritize the relationships corresponding to mandatory constraint relationships and compliance weight relationships. The value is assigned to 1; The relationship priority is assigned to the variable definition relationship, unit binding relationship, and attribute value mapping relationship. The value is assigned to 0.8; The relation precedence corresponding to the membership relation, containment relation, superordinate constraint relation, row constraint relation, and column constraint relation is determined. The value is assigned to 0.6.

[0013] Optionally, the generation process of the compliance gating propagation coefficient and the relation-enhanced semantic representation includes: Based on edge Relationship Generate formula relation marker values Formula relationship marker values ​​corresponding to formula reference relationships, variable definition relationships, and unit binding relationships. A value of 1 represents the formula relationship marker value for relationships other than formula references, variable definition relationships, and unit binding relationships. =0; Based on OCR confidence Hierarchical distance Formula Explanation Distance Compliance weight Relationship priority Relationship between formula and marker value Calculate the compliance gating propagation coefficient The compliance gate propagation coefficient is calculated using the following formula: ; in, For the Sigmoid function, , , , and These are the gating weight parameters; Compliance gate propagation coefficient Write the corresponding edge attributes to the edge set; The initial node vector, initial relation vector, edge set, and compliance gating propagation coefficient are used. Input compliance gate CompGCN; Compliance gate propagation coefficient As the weighting coefficient of the edge message, weighted node-relation combination propagation is performed; The last-layer node vector of the CompGCN compliance gate is used as a semantic representation of relations.

[0014] Optionally, the process of generating the pre-trained task samples includes: Perform masking reconstruction based on relation-enhanced semantic representation; Masking the last-level path field in the hierarchical semantic path generates hierarchical path prediction samples; Mask the row and column constraints connecting numerical nodes to generate table attribute matching samples; Mask the variable definition relationships and unit binding relationships connected by variable nodes, and generate formula interpretation matching samples; By masking the mandatory constraint relationships and compliance weight relationships between clause nodes, mandatory clause identification samples are generated; Use the actual edges in the edge set as positive sample relations; Edges that do not exist in the edge set after replacing the starting node within the same node type range are considered as the first negative sample relation; Edges that do not exist in the edge set after replacing the target node within the same node type range are considered as the second negative sample relation; Generate relationship completion samples based on positive sample relationships, the first negative sample relationship, and the second negative sample relationship; The hierarchical path prediction samples, table attribute matching samples, formula interpretation matching samples, mandatory clause recognition samples, and relationship completion samples are summarized into pre-training task samples.

[0015] The beneficial effects of this invention are: (1) This invention generates hierarchical semantic paths to establish a correspondence between chapter-level numbers, section-level numbers, article-level numbers and clause-level numbers in traffic engineering specifications and text slices, thereby reducing the text discrepancy problem caused by OCR segmentation, allowing pre-trained samples to retain the hierarchical constraint relationship of the specification articles, and improving the model's ability to learn the scope of application of the articles and the logic of the specifications.

[0016] (2) This invention binds the numerical values ​​in complex tables to row headers, column headers and engineering parameters through table attribute chains and formula variable mapping, and binds formula variables to explanatory text, physical definitions and units, thereby reducing the problems of misaligned engineering parameters and isolated formula variables, and improving the accuracy of table parameters and formula semantics in pre-training data.

[0017] (3) This invention uses a heterogeneous semantic graph of traffic engineering specifications and a compliance gating system CompGCN to integrate and propagate OCR confidence, hierarchical distance, formula interpretation distance, compliance weight and relation priority to generate relation-enhanced semantic representations. Based on the relation-enhanced semantic representations, pre-trained task samples are generated to improve the ability of large language models in the field of traffic engineering to express compliance boundaries, parameter relationships and specification semantics. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the multimodal pre-training data augmentation method for traffic engineering based on multidimensional semantic reconstruction proposed in this invention; Figure 2 This is a schematic diagram of the multidimensional semantic reconstruction process in the traffic engineering multimodal pre-training data augmentation method based on multidimensional semantic reconstruction proposed in this invention. Figure 3 This is a schematic diagram of heterogeneous semantic graph construction and pre-training sample generation in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figures 1-3 A method for augmenting multimodal pre-trained data in traffic engineering based on multidimensional semantic reconstruction includes the following steps: Obtain the original traffic engineering specification PDF document and generate a set of basic semantic units; Based on the set of basic semantic units, hierarchical semantic paths, table attribute chains, formula variable mappings, and compliance tags are generated; A heterogeneous semantic graph of traffic engineering specifications is constructed based on hierarchical semantic paths, table attribute chains, formula variable mappings, and compliance labels. A set of nodes, a set of relationships, and a set of edges are generated. OCR confidence, hierarchical distance, formula interpretation distance, compliance weight, and relationship priority are written to the edge set. Encode the set of nodes and the set of relations, and generate initial node vectors and initial relation vectors; Compliance-gated propagation coefficients are generated based on OCR confidence, hierarchical distance, formula interpretation distance, compliance weight, and relation priority. The initial node vector, initial relation vector, edge set, and compliance-gated propagation coefficients are input into the compliance-gated CompGCN, and weighted node-relation combination propagation is performed to generate an enhanced semantic representation of relations. In a specific embodiment of this invention, weighted node-relation combination propagation refers to including the starting node vector, relation vector, and target node vector connected by each edge in the propagation calculation within the compliance-gated CompGCN, and using the compliance-gated propagation coefficients to propagate the edge... The transmitted node-relationship messages are weighted. Specifically, the CompGCN compliance gate first forms a node-relationship combination message based on the starting node vector and the relationship vector. Then, the compliance gate propagation coefficient is used as the edge message multiplication coefficient, so that edges with high OCR confidence, short hierarchical distance, short formula interpretation distance, high compliance weight, and high relationship priority have greater weight in the propagation. This reduces the propagation impact of low confidence, far hierarchical, far interpretation, and low priority edges. Finally, the weighted messages received by the target node are aggregated to update the target node representation and generate a relation-enhanced semantic representation.

[0021] Based on the relation-enhanced semantic representation, masking reconstruction is performed, which masks the last-level path fields in the hierarchical semantic path, masks the row constraint relationship and column constraint relationship of the numerical node connection, masks the variable definition relationship and unit binding relationship of the variable node connection, and masks the mandatory constraint relationship and compliance weight relationship of the clause node connection, thereby generating pre-training task samples; in a specific embodiment of the present invention, masking reconstruction refers to the process of hiding some key information according to the relationship type in the heterogeneous semantic graph of the traffic engineering specification after generating the relation-enhanced semantic representation, and requiring the pre-training samples to restore the hidden content; Encapsulate the original text, relation-enhanced semantic representation, and pre-trained task samples to output a JSONL format dataset.

[0022] In this embodiment, the generation process of the hierarchical semantic path includes: Select semantic units from the title area and the body area from the basic semantic unit set, and generate a text slice sequence according to the page number and vertical coordinate; Using a traffic regulation numbering regularity library, chapter-level, section-level, sub-section-level, and clause-level numbering recognition is performed on text slice sequences to generate numbering level labels; Indentation coordinates are extracted based on the bounding box coordinates of semantic units in the title area and semantic units in the body area. Clustering is then performed on the indentation coordinates to generate indentation level labels. Merge the numbered hierarchical labels and the indented hierarchical labels to generate hierarchical labels; Based on the hierarchical label changes of adjacent text slices in the text slice sequence, backtrack to determine the corresponding superior chapter number, superior section number, superior article number, and superior clause number for each text slice; The hierarchical semantic path is generated by sequentially concatenating the superior chapter number, superior section number, superior article number, superior clause number, and text slice identifier.

[0023] In this embodiment, the process of generating the table attribute chain includes: Select semantic units for the table area from the basic semantic unit set, and construct a table cell matrix based on the semantic units for the table area; Record the text content, row index, column index, row merging range, column merging range, and cell type of each table cell in the table cell matrix; Use a table cell with the cell type "data cell" as the starting point for backtracking; Starting from the data cell, search upwards row by row for the column header cells with overlapping column indices, and generate the list header path in order from the outer header to the inner header; Starting from the data cell, search the row header cells with overlapping row indices column by column to the left, and generate the row header path in the order from the outer header to the inner header; Concatenate the list header path, row header path, and data cell text content into an "attribute-value" key-value pair; Sort all "attribute-value" key-value pairs within the same table area according to row index and column index to generate a table attribute chain.

[0024] In this embodiment, the process of generating the formula variable mapping includes: Select semantic units from the formula area and the main text area from the basic semantic unit set; Extract the LaTeX expression, page number, and formula coordinates of the semantic units in the formula area; Using the lower boundary of the formula coordinates as the search starting point, search downwards within the same page for semantic units in the main text area containing "wherein", "of which", and "symbol explanation"; If no semantic unit in the main text area is found, adjust the search range to 400 pixels; Starting from the semantic unit in the retrieved main text area, extract the explanatory text continuously according to the vertical coordinate, with the extraction endpoint being the first position between the blank line position and the next title area semantic unit position; Variable symbol extraction is performed on the LaTeX expression to generate a set of formula variables; Perform physical definition extraction and unit extraction on the explanatory text to generate a set of explanatory items; Bind the variable symbols to the physical definitions and units in the interpretation item set according to the order in which the variable symbols in the formula variable set appear in the LaTeX expression, and generate a formula variable mapping.

[0025] In this embodiment, the generation process of the compliance tag includes: reading text slices, font visual features and text content from the basic semantic unit set; extracting text slices whose font visual feature is boldface and whose text content contains "must", "strictly prohibited" and "shall not" from the text slices, and generating mandatory provision tags; assigning a compliance weight corresponding to the mandatory provision tag to 1; extracting text slices whose font visual feature is not boldface and whose text content contains "should", "may" and "recommended" from the text slices, and generating recommended provision tags; assigning a compliance weight corresponding to the recommended provision tag to 0.3; marking text slices for which neither a mandatory provision tag nor a recommended provision tag is generated as general provision tags, and assigning a compliance weight corresponding to the general provision tags to 0; writing the mandatory provision tags, recommended provision tags, general provision tags and compliance weights into the compliance tags; binding the compliance tags to the text slice identifiers in the hierarchical semantic paths.

[0026] In this embodiment, the construction process of the heterogeneous semantic graph for traffic engineering specifications includes: generating chapter nodes, section nodes, provision nodes and text slice nodes based on hierarchical semantic paths; generating table header nodes, engineering parameter nodes and value nodes based on table attribute chains; generating formula nodes, variable nodes, interpretation nodes and unit nodes based on the formula variable mapping; generating compliance tag nodes and compliance weight nodes based on the compliance tags; writing the chapter nodes, section nodes, provision nodes, text slice nodes, table header nodes, engineering parameter nodes, value nodes, formula nodes, variable nodes, interpretation nodes, unit nodes, compliance tag nodes and compliance weight nodes into a node set; writing the belonging relation, inclusion relation, superordinate constraint relation, row constraint relation, column constraint relation, attribute-value correspondence relation, formula reference relation, variable definition relation, unit binding relation, mandatory constraint relation and compliance weight relation into a relation set; connecting corresponding nodes according to the node source semantic unit identifiers and relation types to generate an edge set.

[0027] In a specific embodiment of the present invention, the heterogeneous semantic graph of traffic engineering specifications consists of a set of nodes, a set of relationships, and a set of edges. Within the node set, chapter nodes are generated from chapter-level numbers in the hierarchical semantic path; clause nodes are generated from clause-level numbers, section-level numbers, and text slices; table header nodes are generated from list header paths and row header paths; engineering parameter nodes are generated from parameter names in attribute-value key-value pairs; value nodes are generated from the text content of data cells; formula nodes are generated from LaTeX expressions; variable nodes are generated from formula variables; definition nodes are generated from physical definitions; unit nodes are generated from units; compliance label nodes are generated from mandatory clause labels, recommended clause labels, and general clause labels; and compliance weight nodes are generated from compliance weights.

[0028] In the relation set, the belonging relation is used to connect clause nodes and chapter nodes, the superior constraint relation is used to connect superior clause nodes and subordinate text slice nodes, the attribute value correspondence relation is used to connect engineering parameter nodes and value nodes, the variable definition relation is used to connect variable nodes and definition nodes, the unit binding relation is used to connect variable nodes and unit nodes, the mandatory constraint relation is used to connect clause nodes and compliance label nodes, and the compliance weight relation is used to connect compliance label nodes and compliance weight nodes.

[0029] Edge attributes are derived from the set of basic semantic units, hierarchical semantic paths, formula variable mappings, and compliance tags. OCR confidence is derived from the OCR confidence of the basic semantic unit corresponding to the edge-connected node. Hierarchical distance is derived from the path level difference of the node in the hierarchical semantic path. Formula interpretation distance is derived from the vertical pixel distance between the semantic unit in the formula area and the interpretation text. Compliance weight is derived from compliance tags, and relation priority is derived from relation type. These edge attributes participate in the CompGCN compliance gating propagation, distinguishing the heterogeneous semantic graph of traffic engineering specifications from ordinary knowledge graphs that only store entity names and relation names.

[0030] In this embodiment, the process of writing the edge attributes of the edge set includes: Each edge in the edge set is defined as an edge. ,side Connect the starting node ,relation and target node ; Start node and target node The smaller value in the OCR confidence of the corresponding basic semantic unit is written to the edge. OCR confidence ; Start node and target node Write edges in the path hierarchy of the hierarchical semantic path. Hierarchical distance ; Divide the vertical pixel distance between the semantic unit in the formula area and the explanatory text by the average line height of the main text on the page to generate the edge. Formula explanation of distance ; Side The compliance weights in the corresponding compliance labels of the connected text slices are written into the edges. Compliance weight ; Prioritize the relationships corresponding to mandatory constraint relationships and compliance weight relationships. The value is assigned to 1; The relationship priority is assigned to the variable definition relationship, unit binding relationship, and attribute value mapping relationship. The value is assigned to 0.8; The relation precedence corresponding to the membership relation, containment relation, superordinate constraint relation, row constraint relation, and column constraint relation is determined. The value is assigned to 0.6.

[0031] In this embodiment, the generation process of the compliance gating propagation coefficient and the relation-enhanced semantic representation includes: Based on edge Relationship Generate formula relation marker values Formula relationship marker values ​​corresponding to formula reference relationships, variable definition relationships, and unit binding relationships. A value of 1 represents the formula relationship marker value for relationships other than formula references, variable definition relationships, and unit binding relationships. =0; Based on OCR confidence Hierarchical distance Formula Explanation Distance Compliance weight Relationship priority Relationship between formula and marker value Calculate the compliance gating propagation coefficient The compliance gate propagation coefficient is calculated using the following formula: ; in, For the Sigmoid function, , , , and These are the gating weight parameters; Compliance gate propagation coefficient Write the corresponding edge attributes to the edge set; The initial node vector, initial relation vector, edge set, and compliance gating propagation coefficient are used. Input compliance gate CompGCN; Compliance gate propagation coefficient As the weighting coefficient of the edge message, weighted node-relation combination propagation is performed; The last-layer node vector of the CompGCN compliance gate is used as a semantic representation of relations.

[0032] This invention employs a compliance-gated CompGCN, incorporating compliance-gated propagation coefficients into the node-relationship combination propagation process of a standard CompGCN. Standard CompGCN performs node-relationship combination propagation based on node vectors and relationship vectors, primarily utilizing relationship types to express the connection semantics between nodes, without separately controlling OCR recognition confidence, specification hierarchy distance, formula interpretation distance, and compliance constraint strength. The compliance-gated propagation coefficients are generated from OCR confidence, hierarchy distance, formula interpretation distance, compliance weight, relationship priority, and formula relationship marker values, participating in propagation as edge message multiplication coefficients. Formula relationship marker values ​​ensure that formula interpretation distance only applies to formula reference relationships, variable definition relationships, and unit binding relationships; compliance weights give higher propagation weight to mandatory clause-related relationships; and relationship priority is used to distinguish between mandatory constraint relationships, formula interpretation relationships, table attribute relationships, and hierarchical subordinate relationships.

[0033] Through the above processing, CompGCN, a compliance-gated system, can strengthen high-confidence textual relationships, close-level relationships, close-paraphrased relationships, and mandatory compliance relationships during the propagation phase, while suppressing the impact of low-confidence recognition results and weak associations on the semantic representation of relationships. The compliance-gated propagation coefficient constitutes the core improvement of this invention that distinguishes it from ordinary graph neural network propagation.

[0034] In this embodiment, the process of generating the pre-training task samples includes: Perform masking reconstruction based on relation-enhanced semantic representation; Masking the last-level path field in the hierarchical semantic path generates hierarchical path prediction samples; Mask the row and column constraints connecting numerical nodes to generate table attribute matching samples; Mask the variable definition relationships and unit binding relationships connected by variable nodes, and generate formula interpretation matching samples; By masking the mandatory constraint relationships and compliance weight relationships between clause nodes, mandatory clause identification samples are generated; Use the actual edges in the edge set as positive sample relations; Edges that do not exist in the edge set after replacing the starting node within the same node type range are considered as the first negative sample relation; Edges that do not exist in the edge set after replacing the target node within the same node type range are considered as the second negative sample relation; Generate relationship completion samples based on positive sample relationships, the first negative sample relationship, and the second negative sample relationship; The hierarchical path prediction samples, table attribute matching samples, formula interpretation matching samples, mandatory clause recognition samples, and relationship completion samples are summarized into pre-training task samples.

[0035] Example 1: To verify the feasibility of this invention in practice, it was applied to a scenario involving the construction of a traffic engineering specification corpus. The material to be processed was a traffic engineering specification PDF document, which contained chapters, clauses, engineering parameter tables, calculation formulas, formula explanation paragraphs, and mandatory clauses. Conventional OCR slicing methods can extract text content, but they easily sever the hierarchical relationships between chapters, sections, clauses, and sections. General multimodal corpus processing methods can recognize tables and formulas, but the correspondence between table values ​​and row and column headers is unstable, there is a lack of binding between formula variables and explanatory texts such as "wherein" and "of which," and mandatory clauses and general clauses are easily mixed into the same type of training samples.

[0036] In this embodiment, the original traffic engineering specification PDF document is input into the parsing process to generate a basic semantic unit set. This set includes semantic units for the title area, body text area, table area, formula area, and font visual features. For the title and body text semantic units, a traffic specification numbering regularity library is used to identify chapter-level, section-level, article-level, and clause-level numbers. These are then combined with indentation coordinate clustering results to generate hierarchical labels, attaching each text slice to its corresponding hierarchical semantic path. After processing, the article content is no longer used as isolated text for training but carries the complete parent chapter path, facilitating the model's learning of the article's applicable scope.

[0037] For semantic units within the table area, a table cell matrix is ​​constructed, recording the text content, row index, column index, row merging range, column merging range, and cell type of each table cell. Using the data cell as the starting point, the system searches upwards for column index overlaps in the header cells and to the left for row index overlaps in the header cells. The header paths, row header paths, and data cell text content are concatenated into attribute-value pairs, generating a table attribute chain. After processing, individual values ​​can be associated with engineering parameters, applicable conditions, and table hierarchy paths, reducing parameter mismatches caused by multi-level headers and merged cells.

[0038] For semantic units in the formula area, LaTeX expressions, page numbers, and formula coordinates are extracted. Using the lower boundary of the formula coordinates as the search starting point, the search continues downwards within the same page for semantic units in the main text area containing phrases like "wherein," "of which," and "symbol explanation." After retrieving the explanatory text, formula variables are extracted from the LaTeX expressions, and physical definitions and units are extracted from the explanatory text. These variables are then bound according to their order of appearance, generating a formula variable mapping. After processing, formula variables are no longer just symbolic text but are associated with engineering meanings and units, reducing the problem of isolated formula variables.

[0039] For mandatory clauses, the font visual features and text content of text slices are read. Text slices with bold font visual features and text content containing "must," "strictly prohibited," or "must not" are generated as mandatory clause labels and assigned compliance weights. Text slices with non-bold font visual features and text content containing "appropriate," "permissible," or "recommended" are generated as recommended clause labels and assigned lower compliance weights. The remaining text slices are generated as general clause labels. Compliance labels are bound to text slice identifiers in the hierarchical semantic path, providing a source of compliance constraints for subsequent construction of heterogeneous semantic graphs.

[0040] In the construction phase of the heterogeneous semantic graph for traffic engineering specifications, chapter nodes, section nodes, clause nodes, and text slice nodes are generated based on hierarchical semantic paths; table header nodes, engineering parameter nodes, and numerical nodes are generated based on table attribute chains; formula nodes, variable nodes, definition nodes, and unit nodes are generated based on formula variable mapping; and compliance label nodes and compliance weight nodes are generated based on compliance labels. Nodes are connected through relationships such as membership, inclusion, superior constraints, row constraints, column constraints, attribute value correspondence, formula reference, variable definition, unit binding, mandatory constraints, and compliance weight, forming the heterogeneous semantic graph for traffic engineering specifications.

[0041] During the CompGCN compliance gating process, OCR confidence, hierarchical distance, formula interpretation distance, compliance weight, and relationship priority are written to the edge set. Compliance gating propagation coefficients are generated based on these edge attributes. These propagation coefficients are input into CompGCN as edge message multiplication coefficients, resulting in more stable propagation weights for mandatory constraint relationships, compliance weight relationships, variable definition relationships, unit binding relationships, and attribute value correspondence relationships. After CompGCN outputs the enhanced semantic representation of relationships, it performs masking reconstruction on hierarchical semantic paths, table attribute chains, formula variable mappings, and compliance labels, generating hierarchical path prediction samples, table attribute matching samples, formula interpretation matching samples, mandatory clause recognition samples, and relationship completion samples. Finally, these samples are encapsulated into a JSONL format dataset.

[0042] During the comparative verification, the results of manual review were used as a reference to compare the conventional OCR slicing scheme, the general multimodal corpus scheme, and the scheme of the present invention, as shown in Table 1.

[0043] Table 1: Comparison of the Construction Results of Traffic Engineering Standard Corpus

[0044] The data in the table shows that when comparing the three schemes on the same batch of 620 pages of standard documents with the same input scale, the differences mainly stem from different processing methods. The ordinary OCR slicing scheme focuses on text recognition and basic segmentation, achieving a text effectiveness rate of 82.4%. The general multimodal corpus scheme, after incorporating layout, table, and formula processing, improves this to 87.1%. The scheme of this invention further introduces hierarchical semantic paths, table attribute chains, formula variable mapping, and compliance tags, increasing the text effectiveness rate to 95.3%. This demonstrates that the present invention can improve the structural integrity and training usability of standard text slices.

[0045] Regarding the handling of hierarchical relationships, the path integrity rate of the ordinary OCR slicing scheme is 70.8%, indicating that when simply segmenting according to text blocks, the subordinate relationships between chapters, sections, articles, and clauses are easily lost. The general multimodal corpus scheme improves the path integrity rate to 78.5%, but it still struggles to stably preserve the hierarchical relationships between articles. The path integrity rate of the scheme of this invention reaches 94.6%, indicating that by using a traffic specification number regularity library, indentation coordinate clustering, and hierarchical semantic path generation, the path of chapters and clauses can be recovered relatively well, giving the text slices a clear specification source and scope of application.

[0046] In handling complex tables and formulas, the table matching rate and formula matching rate of the ordinary OCR slicing scheme were 62.9% and 57.6%, respectively, reflecting that linear recognition easily leads to table values ​​being separated from row and table headers, and formula variables being separated from their definitions and units. The general multimodal corpus scheme improved both indicators to 76.8% and 71.9%, respectively, indicating that conventional table extraction and formula LaTeX recognition have a certain improvement effect. The scheme of this invention achieves a table matching rate of 92.7% and a formula matching rate of 90.8%, indicating that the table attribute chain and formula variable mapping can effectively establish the correspondence between "value - attribute" and "variable - definition - unit", reducing parameter mismatch and formula variable isolation problems.

[0047] Regarding compliance expression, the ordinary OCR slicing scheme achieves a mandatory clause recognition rate of 69.4% and a mandatory clause false weakening rate of 14.2%, indicating that simple text recognition is insufficient to reliably distinguish between mandatory requirements and general clauses. The general multimodal corpus scheme achieves a mandatory clause recognition rate of 75.8% and reduces the mandatory clause false weakening rate to 9.5%, but its combination of font visual features and mandatory keywords is still insufficient. The scheme of this invention achieves a mandatory clause recognition rate of 93.2% and reduces the mandatory clause false weakening rate to 2.6%, indicating that compliance labels and compliance weights can effectively strengthen the expression of mandatory clauses and reduce the risk of mandatory requirements being weakened in pre-training samples.

[0048] Regarding relation completion and sample construction, ordinary OCR slicing schemes do not support relation completion, indicating that ordinary slicing methods lack a graph structure foundation suitable for relation learning. The general multimodal corpus scheme achieves a relation completion rate of 68.7%, primarily reflecting its ability to extract basic structured relations. The scheme of this invention achieves a relation completion rate of 87.4% and a sample effectiveness rate of 94.1%, demonstrating that the heterogeneous semantic graph of traffic engineering specifications and the compliance gating CompGCN can enhance the expressive effect between nodes and relations, and can transform the enhanced semantic representation of relations into hierarchical path prediction, table attribute matching, formula interpretation matching, mandatory clause recognition, and relation completion samples.

[0049] Based on the results of manual review, the error rate of the ordinary OCR slicing scheme was 18.7%, the general multimodal corpus scheme decreased to 11.4%, and the scheme of this invention further decreased to 4.9%. This result demonstrates that the structured processing of hierarchy, tables, formulas, and compliance relationships in this invention can reduce semantic misalignment and missing relationships. Combining various indicators, compared with the two comparative schemes, the scheme of this invention has more stable implementation performance in terms of preserving the specification hierarchy, binding engineering parameters, binding formula interpretations, expressing compliance weights, and generating pre-training samples.

[0050] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for augmenting multimodal pre-trained data in traffic engineering based on multidimensional semantic reconstruction, characterized in that, Includes the following steps: Obtain the original traffic engineering specification PDF document and generate a set of basic semantic units; Based on the set of basic semantic units, hierarchical semantic paths, table attribute chains, formula variable mappings, and compliance tags are generated; A heterogeneous semantic graph of traffic engineering specifications is constructed based on hierarchical semantic paths, table attribute chains, formula variable mappings, and compliance labels. A set of nodes, a set of relationships, and a set of edges are generated. OCR confidence, hierarchical distance, formula interpretation distance, compliance weight, and relationship priority are written to the edge set. Encode the set of nodes and the set of relations, and generate initial node vectors and initial relation vectors; Based on OCR confidence, hierarchical distance, formula interpretation distance, compliance weight, and relationship priority, compliance gating propagation coefficients are generated. The initial node vector, initial relationship vector, edge set, and compliance gating propagation coefficients are input into the compliance gating CompGCN to perform weighted node-relation combination propagation and generate relationship-enhanced semantic representations. Based on relation-enhanced semantic representation, masking reconstruction is performed, masking the last-level path field in the hierarchical semantic path, masking the row constraint relationship and column constraint relationship of the numerical node connection, masking the variable definition relationship and unit binding relationship of the variable node connection, masking the mandatory constraint relationship and compliance weight relationship of the clause node connection, and generating pre-trained task samples. Encapsulate the original text, relation-enhanced semantic representation, and pre-trained task samples to output a JSONL format dataset; The process of generating the pre-training task samples includes: Perform masking reconstruction based on relation-enhanced semantic representation; Masking the last-level path field in the hierarchical semantic path generates hierarchical path prediction samples; Mask the row and column constraints connecting numerical nodes to generate table attribute matching samples; Mask the variable definition relationships and unit binding relationships connected by variable nodes, and generate formula interpretation matching samples; By masking the mandatory constraint relationships and compliance weight relationships between clause nodes, mandatory clause identification samples are generated; Use the actual edges in the edge set as positive sample relations; Edges that do not exist in the edge set after replacing the starting node within the same node type range are considered as the first negative sample relation; Edges that do not exist in the edge set after replacing the target node within the same node type range are considered as the second negative sample relation; Generate relationship completion samples based on positive sample relationships, the first negative sample relationship, and the second negative sample relationship; The hierarchical path prediction samples, table attribute matching samples, formula interpretation matching samples, mandatory clause recognition samples, and relationship completion samples are summarized into pre-training task samples.

2. The method for augmenting multimodal pre-trained data in traffic engineering based on multidimensional semantic reconstruction according to claim 1, characterized in that, The generation process of the hierarchical semantic path includes: Select semantic units from the title area and the body area from the basic semantic unit set, and generate a text slice sequence according to the page number and vertical coordinate; Using a traffic regulation numbering regularity library, chapter-level, section-level, sub-section-level, and clause-level numbering recognition is performed on text slice sequences to generate numbering level labels; Indentation coordinates are extracted based on the bounding box coordinates of semantic units in the title area and semantic units in the body area. Clustering is then performed on the indentation coordinates to generate indentation level labels. Merge the numbered hierarchical labels and the indented hierarchical labels to generate hierarchical labels; Backtracking is performed to determine the superior chapter-level number, superior section-level number, superior clause-level number and superior sub-clause-level number corresponding to each text slice according to the hierarchical label change relationship of adjacent text slices in the text slice sequence; The superior chapter-level number, superior section-level number, superior clause-level number, superior sub-clause-level number and the text slice identifier are spliced in sequence to generate a hierarchical semantic path.

3. The method for augmenting multimodal pre-trained data in traffic engineering based on multidimensional semantic reconstruction according to claim 1, characterized in that, The generation process of the table attribute chain includes: Filtering table region semantic units from the basic semantic unit set, and constructing a table cell matrix based on the table region semantic units; Recording the text content, row index, column index, row merging range, column merging range and cell type of each table cell in the table cell matrix; Taking table cells with cell type being data cell as backtracking starting points; Starting from the data cell, searching column header cells with overlapping column indexes row by row upwards, and generating a column header path according to the order from outer-layer table headers to inner-layer table headers; Starting from the data cell, searching row header cells with overlapping row indexes column by column to the left, and generating a row header path according to the order from outer-layer table headers to inner-layer table headers; Splicing the column header path, the row header path and the text content of the data cell into an "attribute-value" key-value pair; Sorting all the "attribute-value" key-value pairs in the same table region semantic unit according to row indexes and column indexes, and generating the table attribute chain.

4. The method for augmenting multimodal pre-trained data in traffic engineering based on multidimensional semantic reconstruction according to claim 1, characterized in that, The generation process of the formula variable mapping includes: Filtering formula region semantic units and text region semantic units from the basic semantic unit set; Extracting the LaTeX expression, page number and formula coordinates of the formula region semantic unit; Taking the lower boundary of the formula coordinates as a search starting point, searching text region semantic units containing "wherein", "symbol description" downward within the same page; When no text region semantic unit is retrieved, the search range is adjusted to 400 pixels; Extracting explanatory text continuously according to longitudinal coordinates starting from the retrieved text region semantic unit, the extraction end point is the earlier position between the blank line position and the position of the next title region semantic unit; Extracting variable symbols from the LaTeX expression to generate a formula variable set; Extracting physical definitions and units from the explanatory text to generate an explanatory item set; Binding the variable symbols with the physical definitions and units in the explanatory item set according to the occurrence order of the variable symbols in the formula variable set in the LaTeX expression, and generating the formula variable mapping.

5. The method for augmenting multimodal pre-trained data in traffic engineering based on multidimensional semantic reconstruction according to claim 1, characterized in that, The generation process of the compliance label includes: Reading text slices, font visual features and text content from the basic semantic unit set; Extracting text slices with bold font visual features and text content containing "must", "strictly prohibited", "shall not" from the text slices, and generating mandatory provision labels; Assigning a compliance weight of 1 to the mandatory provision labels; Extracting text slices with non-bold font visual features and text content containing "should", "may", "recommended" from the text slices, and generating recommended provision labels; Assigning a compliance weight of 0.3 to the recommended provision labels; Marking text slices for which neither mandatory provision labels nor recommended provision labels are generated as general provision labels, and assigning a compliance weight of 0 to the general provision labels; Mandatory clause labels, recommended clause labels, general clause labels, and compliance weights are incorporated into the compliance label; Bind compliance tags to text slice identifiers in the hierarchical semantic path.

6. The method for augmenting multimodal pre-trained data in traffic engineering based on multidimensional semantic reconstruction according to claim 1, characterized in that, The process of constructing the heterogeneous semantic graph of the traffic engineering specifications includes: Generate chapter nodes, section nodes, clause nodes, and text slice nodes based on hierarchical semantic paths; Generate header nodes, engineering parameter nodes, and numerical nodes based on the table attribute chain; Formula nodes, variable nodes, definition nodes, and unit nodes are generated based on formula variable mapping; Compliance label nodes and compliance weight nodes are generated based on compliance labels; Write the chapter node, section node, clause node, text slice node, table header node, engineering parameter node, numerical node, formula node, variable node, definition node, unit node, compliance label node, and compliance weight node into the node set; Write the following relationships into the relationship set: membership relationship, inclusion relationship, superior constraint relationship, row constraint relationship, column constraint relationship, attribute value correspondence relationship, formula reference relationship, variable definition relationship, unit binding relationship, mandatory constraint relationship, and compliance weight relationship; Connect the corresponding nodes according to the semantic unit identifier of the node source and the relationship type to generate an edge set.

7. The method for augmenting multimodal pre-trained data in traffic engineering based on multidimensional semantic reconstruction according to claim 6, characterized in that, The process of writing the edge attributes of the edge set includes: Each edge in the edge set is defined as an edge. ,side Connect the starting node ,relation and target node ; Start node and target node The smaller value in the OCR confidence of the corresponding basic semantic unit is written to the edge. OCR confidence ; Start node and target node Write edges in the path hierarchy of the hierarchical semantic path. Hierarchical distance ; Divide the vertical pixel distance between the semantic unit in the formula area and the explanatory text by the average line height of the main text on the page to generate the edge. Formula explanation of distance ; Side The compliance weights in the corresponding compliance labels of the connected text slices are written into the edges. Compliance weight ; Prioritize the relationships corresponding to mandatory constraint relationships and compliance weight relationships. The value is assigned to 1; The relationship priority is assigned to the variable definition relationship, unit binding relationship, and attribute value mapping relationship. The value is assigned to 0.8; The relation precedence corresponding to the membership relation, containment relation, superordinate constraint relation, row constraint relation, and column constraint relation is determined. The value is assigned to 0.

6.

8. The method for augmenting multimodal pre-trained data in traffic engineering based on multidimensional semantic reconstruction according to claim 7, characterized in that, The generation process of the compliance gating propagation coefficient and the relation-enhanced semantic representation includes: Based on edge Relationship Generate formula relation marker values Formula relationship marker values ​​corresponding to formula reference relationships, variable definition relationships, and unit binding relationships. A value of 1 represents the formula relationship marker value for relationships other than formula references, variable definition relationships, and unit binding relationships. =0; Based on OCR confidence Hierarchical distance Formula Explanation Distance Compliance weight Relationship priority Relationship between formula and marker value Calculate the compliance gating propagation coefficient The compliance gate propagation coefficient is calculated using the following formula: ； in, For the Sigmoid function, , , , and These are the gating weight parameters; Compliance gate propagation coefficient Write the corresponding edge attributes to the edge set; The initial node vector, initial relation vector, edge set, and compliance gating propagation coefficient are used. Input compliance gate CompGCN; Compliance gate propagation coefficient As the weighting coefficient of the edge message, weighted node-relation combination propagation is performed; The last-layer node vector of the CompGCN compliance gate is used as a semantic representation of relations.

Images