Vehicle test field knowledge graph construction method and system based on global semantic driving

Abstract

This invention relates to a method and system for constructing a knowledge graph in the vehicle testing domain based on global semantics. The method includes extracting tables and related text from vehicle testing PDF documents; outputting a table image that retains its complete two-dimensional layout; inputting the table image into a multimodal large model to generate a global semantic description containing the table's business themes and row-column logical relationships; performing topic clustering on the global semantic description, identifying representative entities under each topic, and selecting entities with appropriate granularity and high relevance as seed entities; using the seed entities as guides, combining the table image and the global semantic description, extracting candidate knowledge triples from the vehicle testing PDF document; and performing global knowledge fusion on the candidate knowledge triples to form a globally consistent knowledge graph. This invention can effectively solve the problems of table logic breakage, entity omission, and relationship generalization in existing technologies when processing vehicle testing documents.

Description

Technical Field

[0001] This invention relates to the fields of intelligent vehicle testing and artificial intelligence technology, and in particular to a method and system for constructing a knowledge graph in the field of vehicle testing based on global semantic driving. Background Technology

[0002] In the R&D system of intelligent connected vehicles, technical specifications such as functional definitions, signal interaction rules, and diagnostic protocols are usually delivered in PDF documents as the standard delivery medium. PDF documents in the field of vehicle testing have strong professional and structured characteristics. Their test logic (such as the correspondence between input signals and expected outputs, the constraint relationship between operational behavior and system response, and the association rules between parameter thresholds and working states) are mostly contained in complex tables. The row and column layout and cell relationships of the tables are the carriers for conveying the test scenario logic and parameter constraint relationships. Currently, existing technologies for constructing vehicle testing knowledge graphs from PDF documents mainly fall into two categories: one is dedicated data extraction solutions for the vehicle testing field. Existing solutions mostly focus on the structured extraction of single-type data, such as parsing fixed fields in test reports using PDFMiner, or extracting isolated data such as signal codes and voltage thresholds from tables using customized regular expressions. Some solutions store this extracted data in relational databases, but do not build logical relationships between entities. The other category is general technology construction solutions, namely the "OCR+LLM" model. This involves first converting document content into a linear plain text stream using an OCR tool, and then using a large language model to identify entities and extract relationships from the text stream. This method is widely used for the initial knowledge organization of general documents, but it is not adapted to vehicle testing scenarios. It adopts a local perspective, focusing on knowledge extraction at the single sentence or paragraph level, and lacks the integration and fusion of global knowledge.

[0003] The aforementioned technologies have several shortcomings when processing highly structured tabular content in the vehicle testing field. First, regarding the visual semantics of tables, existing technologies convert two-dimensional tables into one-dimensional linear text, disrupting the table's row and column topology, cell alignment, and merging relationships. This leads to a break in the test logic carried by the table. Furthermore, extraction from a local perspective only focuses on isolated text fragments, failing to capture complex relationships between entities across tables and documents. Second, in terms of entity guidance, existing technologies typically perform indiscriminate scanning extraction of the entire document, lacking focus on core entities. This easily results in the extraction of invalid entities with overly broad or fine granularity. Additionally, the lack of targeted guidance for core entities in the test scenario can easily lead to the omission of entities such as boundary conditions and signal fault tolerance thresholds. Moreover, regarding domain-specific relationships, general knowledge graph construction technologies do not set extraction keywords specific to the vehicle testing domain, only recognizing generalized relationships such as inclusion and association, failing to accurately capture the professional logic of the test scenario. Furthermore, the keywords and relationships in a few vehicle testing-specific data processing solutions rely on manual pre-sorting and preset fixed templates, unable to automatically adapt to the dynamic logical differences of different test types. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a method and system for constructing a vehicle testing domain knowledge graph based on global semantics. This method achieves accurate knowledge extraction by guiding core entities and automatically adapts to domain-specific relationships for different test types, thereby effectively solving the problems of table logic breakage, entity omission, and relationship generalization in the processing of vehicle test documents in the prior art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution.

[0006] Firstly, the present invention provides a method for constructing a vehicle testing domain knowledge graph based on global semantic driving, which adopts the following technical solution:

[0007] A method for constructing a vehicle testing domain knowledge graph based on global semantics includes:

[0008] Preprocess the vehicle test PDF document to extract tables and related text.

[0009] Use visual layout analysis tools to identify table boundaries, cell coordinates, and row and column indices in a document, and output a table image that retains the complete two-dimensional layout;

[0010] The table image is input into a multimodal large model to generate a global semantic description that includes the table's business themes and row and column logical relationships;

[0011] The global semantic description is subjected to topic clustering to identify representative entities under each topic, and entities with appropriate granularity and high relevance are selected as seed entities.

[0012] Guided by the seed entity, and combining the table image and the global semantic description, candidate knowledge triples are extracted from the vehicle test PDF document; and

[0013] The candidate knowledge triples are subjected to global knowledge fusion, including entity merging, conflict resolution, and new triple inference, to form a globally consistent knowledge graph.

[0014] Furthermore, in the method, the step of using a visual layout analysis tool to identify table boundaries, cell coordinates, and row and column indices in the document also includes:

[0015] Identify cell spanning information (rows and columns); and

[0016] Construct a table structured intermediate representation, which includes table identifier, page number, cell identifier, cell bounding box, row index, column index, row span, column span, cell text, and table header path obtained by parsing multi-level table headers.

[0017] Furthermore, in the method, an expansion rule is applied to merged cells, forming a hierarchical path between the parent header and the child header and binding it to each data cell, thereby preserving the logical topological relationship between the row header or column header and the data cells.

[0018] Furthermore, in the method, selecting entities with appropriate granularity and high relevance as seed entities includes:

[0019] Candidate entities are extracted from the global semantic description, the header path, and the domain key columns; and

[0020] A comprehensive score is calculated for each candidate entity, which is based on the cosine similarity between the candidate entity vector and the topic vector, the frequency of occurrence of the candidate entity, structural importance, and noise penalty.

[0021] Furthermore, in the method, the structural importance includes the distribution entropy of the covered rows and columns, whether it appears in the header path, and whether it is located in a key column.

[0022] Furthermore, in the method, the step of extracting candidate knowledge triples guided by the seed entity includes:

[0023] Design specific extraction prompts for the vehicle testing field, clearly defining the types of entities to be identified and the types of exclusive relationships to be extracted;

[0024] The seed entities, the table image, and the global semantic description are integrated to form complete input data containing topic focus, visual semantic information, and a global description; and

[0025] The complete input data is input into the large language model, and the model is guided to complete the extraction of candidate knowledge triples by progressive logic of seed entity guidance, judging professional logical relationships between entities, and generating standard triples of head entity-relationship-tail entity.

[0026] Furthermore, the method also includes:

[0027] The seed entity is used as an anchor point to locate its position in the structured intermediate representation of the table; and

[0028] Candidate entity pairs are generated within a controlled area, which includes local regions of the same row, column, header path, or test entry.

[0029] Furthermore, the method also includes:

[0030] The constraint model outputs triple records according to a predefined pattern. Each triple record includes a head entity, relation, tail entity, head entity type, tail entity type, evidence, and confidence level.

[0031] Perform pattern validation, type validation, and field validity validation on the model output. When the validation fails or the confidence level is lower than the threshold, trigger a local re-evaluation strategy to regenerate only the failed fields or low-confidence fields.

[0032] Furthermore, in the method, the entity merging includes:

[0033] Based on a dictionary and alias table for vehicle testing, rules are standardized and normalized, including Chinese-English translation, abbreviation expansion, synonym mapping, and unit standardization; and

[0034] When the rules cannot be determined, the similarity of entity vectors is calculated and candidate merging pairs are generated. If the similarity is higher than the merging threshold, a unified canonical name is generated and the original alias is written into the alias list.

[0035] Furthermore, in the method, the conflict resolution includes:

[0036] For conflicts involving multiple tail entities corresponding to the same head entity and relation, a total confidence score is calculated, based on extraction confidence, document version score, industry standard score, and evidence completeness score; and

[0037] The relation with the highest total confidence is selected and written into the confirmed set, while the remaining relations are written into the pending verification set and the chain of evidence is preserved.

[0038] Furthermore, in the method, the new triplet inference includes:

[0039] Controlled inference is performed using a rule base from the vehicle testing domain, wherein the rules are stored in template form; and

[0040] The inference results are accompanied by an inference rule identifier, a set of source triples, and a confidence decay coefficient.

[0041] Furthermore, the method also includes:

[0042] For the precondition-triggered action-expected result chain, check whether the extracted results can close within the same test item; and

[0043] Candidate relationships that cannot be closed are marked as pending confirmation, and the chain of evidence is preserved for conflict resolution and manual review in subsequent fusion.

[0044] Secondly, the present invention provides a vehicle testing domain knowledge graph construction system based on global semantic driving, which adopts the following technical solution:

[0045] A knowledge graph construction system for the vehicle testing domain based on global semantics, comprising:

[0046] The preprocessing module is configured to preprocess vehicle test PDF documents, extracting tables and related text.

[0047] The visual layout parsing module is configured to use visual layout parsing tools to identify table boundaries, cell coordinates, and row and column indices in a document, and output a table image that retains the complete two-dimensional layout.

[0048] The global semantic generation module is configured to input the table image into a multimodal large model and generate a global semantic description that includes the table's business themes and row and column logical relationships.

[0049] The seed entity filtering module is configured to perform topic clustering on the global semantic description, identify representative entities under each topic, and filter out entities with appropriate granularity and high relevance as seed entities.

[0050] The triple extraction module is configured to extract candidate knowledge triples from the vehicle test PDF document, guided by the seed entity and combining the table image and the global semantic description; and

[0051] The global knowledge fusion module is configured to perform global knowledge fusion on the candidate knowledge triples, including entity merging, conflict resolution, and new triple inference, to form a globally consistent knowledge graph.

[0052] Thirdly, the present invention provides a readable storage medium, which adopts the following technical solution:

[0053] A readable storage medium storing computer instructions that, when executed by a processor, implement the method as described in any one of the first aspects above.

[0054] In summary, compared with the prior art, the present invention has at least one of the following beneficial technical effects:

[0055] This invention preserves the complete two-dimensional topological structure of tables through visual layout parsing, avoiding the loss of row and column relationships and the breakage of test logic caused by traditional linearization processing. It generates a global semantic description through a multimodal large model and performs topic clustering to filter seed entities, achieving precise focus on core entities in the document and effectively avoiding the problems of uncontrolled granularity and omission of key entities caused by indiscriminate scanning extraction. Through seed entity guidance and domain-specific extraction prompts, it can automatically identify the professional logical relationships of vehicle testing scenarios, adapting to the dynamic logical differences of different test types without the need for manually preset fixed templates. Global knowledge fusion enables entity merging and conflict resolution across tables and documents, ensuring that the final knowledge graph has global consistency and high availability, thereby effectively supporting automated test case generation and intelligent execution of test tasks. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart of an embodiment of the vehicle testing domain knowledge graph construction method based on global semantic driving of the present invention is shown.

[0058] Figure 2 A flowchart of an embodiment of the table-structured intermediate representation construction method of the present invention is shown.

[0059] Figure 3 A flowchart of an embodiment of the seed entity screening method of the present invention is shown.

[0060] Figure 4 A flowchart of an embodiment of the candidate knowledge triple extraction and global knowledge fusion method of the present invention is shown.

[0061] Figure 5 The flowchart illustrates a specific application embodiment of the link closed-loop check and verification process for the candidate triplet extraction stage of the present invention.

[0062] Figure 6 The diagram illustrates the architecture of an embodiment of the vehicle testing domain knowledge graph construction system based on global semantics driven by the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, it should be understood that the specific embodiments described herein are only for illustration and explanation of this application and are not intended to limit this application.

[0064] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments of this application. Furthermore, the descriptions of each embodiment in the following embodiments have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0065] The method steps described in this embodiment of the invention can be executed in the order described in the specific implementation, or the execution order of each step can be adjusted according to actual needs, provided that the technical problem can be solved. These are not listed one by one here.

[0066] The following is in conjunction with the appendix Figure 1 The present invention will be described in further detail below.

[0067] Reference Figure 1 , Figure 1 A flowchart of a method 100 for constructing a vehicle testing domain knowledge graph based on global semantics is shown. Method 100 includes processing steps such as preprocessing vehicle testing PDF documents, visual layout parsing, global semantic generation, seed entity screening, candidate knowledge triple extraction, and global knowledge fusion, thereby forming a globally consistent knowledge graph.

[0068] Method 100 begins with step 102, in which the vehicle test PDF document is preprocessed to extract tables and related text. The vehicle test PDF document includes technical specification documents such as SSTS specifications and test case documents. In step 102, the vehicle test PDF document undergoes format standardization and noise filtering to remove irrelevant content such as headers, footers, page numbers, and watermarks, and tables and related text content are extracted from the document.

[0069] Continue to refer to Figure 1Method 100 then proceeds to step 104, where a visual layout analysis tool is used to identify table boundaries, cell coordinates, and row and column indices in the document, outputting a table image that retains the complete two-dimensional layout. In some implementations, the visual layout analysis tool uses MinerU for layout analysis and table recognition. MinerU identifies table boundary coordinates, cell coordinates, row and column indices, and cross-row and cross-column information, and crops and outputs a table image that retains the complete two-dimensional layout according to the coordinates. Through visual layout analysis, method 100 preserves the two-dimensional topological structure of the table, avoiding the loss of table row and column association information caused by traditional OCR linearization processing.

[0070] Method 100 then proceeds to step 106, in which the table image is input into the multimodal large model to generate a global semantic description containing the table's business themes and row-column logical relationships. In some implementations, the multimodal large model uses Qwen-VL-Plus to generate the global semantic description. Qwen-VL-Plus receives the table image as input and generates a global semantic description oriented towards the vehicle testing scenario. The global semantic description includes the table's business themes, the semantic roles of each column, the row-column logical relationships, and the overall logic of condition-action-response.

[0071] Method 100 proceeds to step 108, where topic clustering is performed on the global semantic description to identify representative entities under each topic, and entities with appropriate granularity and high relevance are selected as seed entities. In step 108, the global semantic description is vectorized and topic clustering is performed to obtain a topic set and topic vectors. Method 100 identifies representative entities under each topic from the global semantic description and selects entities with appropriate granularity and high relevance as seed entities through comprehensive scoring. Seed entities include core entities in the vehicle testing domain such as door handles, digital keys, vehicle unlocking, and horn reminders, which are used to guide the subsequent extraction of candidate knowledge triples.

[0072] Method 100 then proceeds to step 110, in which candidate knowledge triples are extracted from the vehicle test PDF document, guided by seed entities and combined with table images and global semantic descriptions. In some implementations, the large language model uses Qwen-Max to perform candidate triple extraction. In step 110, seed entities, table images, and global semantic descriptions are integrated to form complete input data containing topic focus, visual semantic information, and global descriptions. This complete input data is then input into Qwen-Max, guiding Qwen-Max to complete the candidate knowledge triple extraction through a progressive logic of seed entity guidance, determining professional logical relationships between entities, and generating head entity-relationship-tail entity standard triples.

[0073] In step 112, method 100 performs global knowledge fusion on candidate knowledge triples, including entity merging, conflict resolution, and new triple inference, to form a globally consistent knowledge graph. In step 112, method 100 performs entity merging on candidate knowledge triples, normalizing synonymous or near-synonymous entities and eliminating naming redundancy. Method 100 performs conflict resolution, filtering logically consistent correct relationships for conflicting relationships within the same entity pair, based on test document versions and industry standards. Method 100 performs new triple inference, inferring implicit test logic associations based on existing triples and common sense in the vehicle testing domain, generating new triples to enrich the knowledge graph. In some implementations, the knowledge graph is stored and visualized using the Neo4j graph database. The fused triples are imported into the Neo4j graph database, node labels are set according to entity type, edge types are set according to relationship type, and indexes are created to support evidence traceability.

[0074] Reference Figure 2 , Figure 2 A flowchart is shown for a method 200 for constructing a structured intermediate representation of tables in the construction of a knowledge graph for the vehicle testing domain. Method 200 is used to construct a structured intermediate representation that preserves the logical topological relationships of tables during the visual layout parsing process, thereby supporting subsequent knowledge extraction and evidence tracing and localization.

[0075] like Figure 2 As shown, method 200 begins with step 202, in which the row spanning and column spanning information of cells are identified. Before performing step 202, the document undergoes layout normalization and noise filtering, including the removal of headers, footers, page numbers, watermarks, and irrelevant descriptions, and an image and text block index is generated for each page. In step 202, the visual layout parsing tool identifies the row spanning and column spanning information of each cell, obtaining the row spanning range and column spanning range of the cell in the table.

[0076] Continue to refer to Figure 2 Method 200 then proceeds to step 204, where a structured intermediate representation of the table is constructed. This structured intermediate representation includes table identifiers, page numbers, cell identifiers, cell bounding boxes, row indices, column indices, row spans, column spans, cell text, and header paths obtained from parsing multi-level headers. The table identifier uniquely identifies each table in the document; the page number records the page position of the table; and the cell identifier uniquely identifies each cell in the table. The cell bounding box records the coordinate position of the cell on the page, supporting evidence tracing and location. The row and column indices record the row and column positions of the cell in the table, and the row and column spans record the number of rows and columns the cell spans. The cell text records the text content within the cell, and the header path records the hierarchical path information obtained from parsing multi-level headers.

[0077] like Figure 2 As further shown, method 200 then proceeds to step 206, in which it is determined whether merged cells exist. Merged cells refer to cells that span multiple rows or columns, and are used in tables of vehicle test documents to represent multi-level headers or grouping information.

[0078] Continue to refer to Figure 2 If merged cells are determined to exist in step 206, method 200 proceeds to step 208. In step 208, an expansion rule is applied to form a hierarchical path between the parent and child headers and bind it to each data cell. Applying an expansion rule to merged cells to form a hierarchical path between the parent and child headers and bind it to each data cell preserves the logical topological relationship between row or column headers and data cells. In some implementations, the header path stores the hierarchical relationship in a list format; for example, the header path may contain hierarchical information such as test preconditions, input signals, and thresholds. After step 208 is completed, method 200 proceeds to step 212.

[0079] If it is determined in step 206 that no merged cells exist, method 200 proceeds to step 210, where an intermediate representation containing table identifiers, page numbers, and cell information is output. In step 210, the table-structured intermediate representation directly records information such as the table identifier, page number, cell identifier, cell bounding box, row index, column index, and cell text for each cell. After step 210 is completed, method 200 proceeds to step 212.

[0080] Continue to refer to Figure 2 In step 212, method 200 outputs an intermediate representation that preserves the logical topological relationship between row headers or column headers and data cells. In step 212, the table-structured intermediate representation output by method 200 preserves the two-dimensional topological structure of the table, including the relationships between row header nodes, column header nodes, and data cell nodes. For cross-row or cross-column references, the association boundary is determined by the shortest path in the table topology graph. The shortest path is the path from row header node to data cell node to column header node, thus avoiding noise and mismatches caused by indiscriminate enumeration of the entire text. The table-structured intermediate representation provides a controlled range for subsequent candidate entity pair generation, supporting the generation of candidate entity pairs within a local area of ​​the same row, column, header path, or test item.

[0081] Reference Figure 3 , Figure 3The flowchart illustrates a seed entity selection method 300. Method 300 is used to select entities with appropriate granularity and high relevance from the global semantic description as seed entities, providing anchors for subsequent candidate knowledge triple extraction. The global semantic description includes at least the table's purpose or functional theme, the semantic roles of each column, key constraints, and the overall logic of condition-action-response, where key constraints include information such as thresholds, units, and patterns.

[0082] like Figure 3 As shown, method 300 begins with step 302, in which candidate entities are extracted from the global semantic description, the header path, and the domain key columns. The domain key columns include columns for preconditions, input signals / thresholds, trigger actions, expected results, and diagnostic DTCs. In step 302, method 300 combines the hierarchical information in the header path and the text content in the domain key columns to identify and extract candidate entities related to the vehicle testing scenario. Candidate entities are assigned initial entity types, including operating conditions / modes, input signals, thresholds / units, trigger actions, expected results, and diagnostic events.

[0083] Continue to refer to Figure 3 Method 300 then proceeds to step 304, where the cosine similarity between the candidate entity vector and the topic vector is calculated. In step 304, the candidate entities are vectorized, and the cosine similarity between the candidate entity vector and the topic vector is calculated. The cosine similarity is used to measure the semantic relevance between the candidate entity and the topic; a higher cosine similarity indicates a stronger relevance between the candidate entity and the topic.

[0084] like Figure 3 As further shown, method 300 then executes step 306, in which the occurrence frequency and structural importance of candidate entities are calculated. Occurrence frequency represents the number of times a candidate entity appears in a table or section. Structural importance includes the distribution entropy of the covered rows and columns, whether it appears in the header path, and whether it is located in a key column. The distribution entropy of the covered rows and columns measures the uniformity of the distribution of candidate entities in the table's rows and columns; a higher distribution entropy indicates a wider range of rows and columns covered by the candidate entity. Whether it appears in the header path determines whether the candidate entity is a structural element of the table; candidate entities appearing in the header path have higher structural importance. Whether it is located in a key column determines whether the candidate entity is located in key columns related to preconditions, input signals / thresholds, trigger actions, expected results, and diagnostic DTCs; candidate entities located in key columns have a strong correlation with vehicle testing logic.

[0085] Continue to refer to Figure 3Method 300 proceeds to step 308, where a noise penalty is calculated and a comprehensive score is generated. The noise penalty is used to reduce the scores of noisy entities such as stop words, excessively long phrases, purely numerical / unit words, and domain-irrelevant generic words. The comprehensive score is based on the cosine similarity between the candidate entity vector and the topic vector, the frequency of occurrence of the candidate entity, structural importance, and the noise penalty. The comprehensive score formula is as follows: ,in This represents the cosine similarity between the candidate entity vector and the topic vector. Indicates the frequency of occurrence of candidate entities. Indicates structural importance. Indicates noise penalty, , , , These are the weighting coefficients.

[0086] like Figure 3 As shown, in step 310, method 300 selects entities with appropriate granularity and high relevance as seed entities. In step 310, method 300 selects entities with higher comprehensive scores from candidate entities using a threshold or Top-K method as seed entities. Method 300 performs deduplication on synonymous candidate entities and records the topic identifier, comprehensive score, and source evidence for each seed entity. The source evidence includes page number, table identifier, and cell identifier for subsequent traceability and verification. The selected seed entities include entities with appropriate granularity and high relevance to the vehicle testing scenario, such as door handles, digital keys, vehicle unlocking, and horn reminders.

[0087] Furthermore, refer to Figure 4 , Figure 4 The flowchart illustrates a candidate triple extraction and global knowledge fusion method 400. Method 400 is used to extract candidate knowledge triples guided by seed entities and to perform global knowledge fusion on the candidate knowledge triples, including entity merging, conflict resolution, and new triple inference, thereby forming a globally consistent knowledge graph.

[0088] like Figure 4 As shown, method 400 begins at step 402, where a vehicle testing domain-specific extraction prompt is designed to clearly define the entity types to be identified and the specific relationship types to be extracted. The entity type enumeration includes operating conditions / modes, input signals, thresholds / units, triggering actions, expected results, and diagnostic events. The relationship type enumeration includes specific relationships such as operating condition-constraint signals, actions-preconditions, triggers, inclusion, and association. This vehicle testing domain-specific extraction prompt provides clear extraction guidance for the large language model, enabling it to identify specialized entities and logical relationships in the vehicle testing scenario.

[0089] Continue to refer to Figure 4Method 400 then proceeds to step 404, where the seed entity, table image, and global semantic description are integrated to form complete input data containing topic focus, visual semantic information, and global description. In step 404, the seed entity provides topic focus information, the table image provides visual semantic information, and the global semantic description provides information on the table's business topics and row-column logical relationships. The complete input data also includes a structured intermediate representation of the table and associated text, thus providing multi-source input for the large language model. In some implementations, the seed entity is used as an anchor point to locate its occurrence position in the structured intermediate representation of the table, and candidate entity pairs are generated within a controlled range, which includes local areas of the same row, the same column, the same header path, or the same test item.

[0090] like Figure 4 As further shown, method 400 then executes step 406, in which the complete input data is input into the large language model, guiding the large language model to complete the extraction of candidate knowledge triples through a progressive logic of seed entity guidance, determining the professional logical relationship between entities, and generating standard triples of head entity-relationship-tail entity. In step 406, the large language model outputs triple records according to a predefined pattern. The triple record contains head entity, relation, tail entity, head entity type, tail entity type, evidence, and confidence level. Evidence includes page number, table identifier, cell identifier list, and original text fragment, and the confidence level is a value between 0 and 1. In some implementations, the output of the large language model is subjected to pattern verification, type verification, and field validity verification. When the verification fails or the confidence level is lower than the threshold, a local re-interrogation strategy is triggered to regenerate only the failed fields or low-confidence fields, thereby improving the extraction stability.

[0091] Continue to refer to Figure 4 Method 400 proceeds to step 408, where entity merging is performed. Entity merging is based on a vehicle testing domain dictionary and alias table, and is standardized using rules, including Chinese-English translation, abbreviation expansion, synonym mapping, and unit normalization. Chinese-English translation maps English entity names to Chinese entity names, for example, mapping Walk-Away Lock to Automatic Locking upon leaving the vehicle. Abbreviation expansion expands abbreviated entity names into their full forms. Synonym mapping maps entity names with the same meaning but different expressions. Unit normalization unifies different unit representations into a standard form. When rules cannot be determined, entity vector similarity is calculated, and candidate merging pairs are generated. If the similarity is higher than the merging threshold, a unified standardized name is generated, and the original alias is added to the alias list. Boundary examples can be reconfirmed using a large model. After confirmation, a unified standardized name is generated to ensure the uniqueness and retrieval of entities in the knowledge graph.

[0092] like Figure 4As shown, method 400 then executes step 410, in which conflict resolution is performed. Conflict resolution addresses conflicts between the same head entity and multiple tail entities corresponding to the same relation, calculating the total confidence score. The formula for calculating the total confidence score is as follows: ,in This indicates the extraction confidence level. This indicates the document version score. Indicates the industry standard score. The score indicates the completeness of the evidence. , , , These are weighting coefficients. Document version scores are ranked according to the newest version, with newer versions receiving higher scores. Industry standard scores are based on industry standard priority, which is ordered in the preset order of national standards, European standards, and enterprise standards. Evidence completeness scores are based on whether it simultaneously contains header paths, data cells, and related text; evidence containing all three has a higher completeness score. Method 400 selects the relationship with the highest total confidence and writes it into the confirmed set; the remaining relationships are written into the pending verification set, and the evidence chain is preserved to support auditing and continuous iteration.

[0093] Continue to refer to Figure 4 In step 412, method 400 employs a vehicle testing domain rule base to perform controlled new triple inference. Rules are stored in template form; for example, if there exists <Condition A - Includes - Precondition B> and <Precondition B - Triggers - Action C> and <Action C - Generates - Expected Result D>, then <Condition A - Expected Result - D> is derived. The inference result includes an inference rule identifier (inference_rule_id), a source triple set, and a confidence decay coefficient (confidence_decay). The confidence decay coefficient represents the degree of confidence decay of the inference result relative to the source triples. In some implementations, it is configurable whether manual confirmation is required before writing to the main graph, thereby achieving a balance between automated inference and manual review. Through controlled rule inference, method 400 infers the implicit test logic associations between existing triples and common sense in the vehicle testing domain to generate new triples to enrich the knowledge graph, while avoiding errors introduced by unconstrained inference.

[0094] In summary, referring to Figure 5 , Figure 5 The flowchart illustrates a specific application embodiment of the present invention, including a link loop check and verification process during the candidate triplet extraction stage. The link loop check is used to verify the completeness and correctness of the extraction results, ensuring that the logical link between the preconditions, triggering actions, and expected results forms a closed loop within the same test item.

[0095] The closed-loop check process begins by examining whether the extracted results of the precondition-triggered action-expected result link can be closed within the same test item. A test item refers to the table area corresponding to the same test case number or the same test item row. In vehicle test documentation, preconditions, triggered actions, and expected results are usually logically related within the same test item. Preconditions describe the state or conditions that must be met before test execution, triggered actions describe the operational behavior during test execution, and expected results describe the system response that should occur after test execution. The closed-loop check verifies whether preconditions, triggered actions, and expected results exist simultaneously within the same test item, and whether a complete logical link is formed between them.

[0096] Continue to refer to Figure 5 The closed-loop check process then enters the judgment step, determining whether the candidate relationships can be closed. Closure means that within the same test item, all relationships in the precondition-triggered action-expected result chain exist and are interconnected, forming a complete test logic chain.

[0097] like Figure 5 As further illustrated, if a candidate relationship can be closed, the link closure check process adds the candidate relationship to the confirmed set. The candidate relationships in the confirmed set have complete logical link support and can be directly used for subsequent knowledge graph construction.

[0098] If a candidate relationship cannot be closed, the link closure check process will mark the candidate relationship as pending confirmation. The pending confirmation status indicates that the logical completeness of the candidate relationship is questionable and requires further verification or manual review. Situations where closure is impossible include missing preconditions, missing triggering actions, or missing expected results within the same test item, resulting in an incomplete precondition-triggering action-expected result link.

[0099] After marking candidate relationships as pending confirmation, the link closure check process preserves the evidence chain for conflict resolution and manual review in subsequent fusion stages. The evidence chain includes the source page number, table identifier, cell identifier, and original text fragment of the candidate relationship, supporting subsequent review and verification. By preserving the evidence chain, candidate relationships in the pending confirmation stage can undergo conflict resolution or be manually reviewed and confirmed, thus preventing erroneous relationships from being directly written into the knowledge graph. The link closure check and verification process distinguishes between candidate relationships that can be closed and those that cannot, ensuring the accuracy of the knowledge graph while preserving potentially valuable candidate relationships for further processing.

[0100] This invention also discloses a vehicle testing domain knowledge graph construction system based on global semantics.

[0101] Reference Figure 6 , Figure 6This diagram illustrates the architecture of a vehicle testing domain knowledge graph construction system 500 based on global semantics. The knowledge graph construction system 500 includes a preprocessing module 502, a visual layout parsing module 504, a global semantics generation module 506, a seed entity filtering module 508, a triple extraction module 510, and a global knowledge fusion module 512. The knowledge graph construction system 500 achieves a complete processing flow from document preprocessing to knowledge graph generation through the sequential connection between these modules.

[0102] The preprocessing module 502 is configured to preprocess the vehicle test PDF document, extracting tables and related text. The preprocessing module 502 receives the vehicle test PDF document as input, performs format normalization and noise filtering, and removes irrelevant content such as headers, footers, page numbers, and watermarks. The preprocessing module 502 extracts tables from the document, along with adjacent headings, paragraph descriptions, footnotes, and unit descriptions, as related text. The output of the preprocessing module 502 is connected to the visual layout parsing module 504.

[0103] The visual layout parsing module 504 is configured to use a visual layout parsing tool to identify table boundaries, cell coordinates, and row and column indices in a document, outputting a table image that retains the complete two-dimensional layout. The visual layout parsing module 504 receives the output from the preprocessing module 502 and uses the visual layout parsing tool to perform layout analysis and table recognition. The visual layout parsing module 504 identifies table boundary coordinates, cell coordinates, row and column indices, and cross-row and cross-column information, and crops and outputs a table image that retains the complete two-dimensional layout according to the coordinates. The visual layout parsing module 504 constructs a structured intermediate representation of the table, preserving the table's two-dimensional topological structure. The output of the visual layout parsing module 504 is connected to the global semantic generation module 506.

[0104] The global semantic generation module 506 is configured to input a table image into a multimodal large model and generate a global semantic description containing the table's business themes and row-column logical relationships. The global semantic generation module 506 receives the table image output by the visual layout parsing module 504 and inputs it into the multimodal large model for visual understanding. The global semantic description generated by the global semantic generation module 506 includes the table's purpose or functional themes, the semantic roles of each column, key constraints, and the overall condition-action-response logic. The output of the global semantic generation module 506 is connected to the seed entity filtering module 508.

[0105] The seed entity selection module 508 is configured to perform topic clustering on the global semantic description, identify representative entities under each topic, and select entities with appropriate granularity and high relevance as seed entities. The seed entity selection module 508 receives the global semantic description output by the global semantic generation module 506, vectorizes the global semantic description, and performs topic clustering. The seed entity selection module 508 extracts candidate entities from the global semantic description, header paths, and domain key columns, and selects entities with appropriate granularity and high relevance as seed entities through comprehensive scoring. The output of the seed entity selection module 508 is connected to the triple extraction module 510.

[0106] The triple extraction module 510 is configured to extract candidate knowledge triples from a vehicle test PDF document, guided by seed entities and combined with table images and global semantic descriptions. The triple extraction module 510 receives seed entities output by the seed entity filtering module 508 and combines them with table images output by the visual layout parsing module 504 and global semantic descriptions output by the global semantic generation module 506. The triple extraction module 510 integrates the seed entities, table images, and global semantic descriptions to form complete input data, and then inputs this complete input data into a large language model to complete the candidate knowledge triple extraction. The output of the triple extraction module 510 is connected to the global knowledge fusion module 512.

[0107] The global knowledge fusion module 512 is configured to perform global knowledge fusion on candidate knowledge triples, including entity merging, conflict resolution, and new triple inference, forming a globally consistent knowledge graph. The global knowledge fusion module 512 receives candidate knowledge triples output by the triple extraction module 510 and performs entity normalization to merge synonymous or near-synonymous entities. The global knowledge fusion module 512 performs conflict resolution, calculating the total confidence score for conflicts between the same head entity and multiple tail entities corresponding to the same relation, and selecting the relation with the highest total confidence score. The global knowledge fusion module 512 uses a rule base from the vehicle testing domain to perform controlled new triple inference, based on the implicit test logic associations inferred from existing triples.

[0108] In the knowledge graph output by the Knowledge Graph Construction System 500, the node labels are set according to entity type, including Condition, Signal, Action, ExpectedResult, and Function node labels. Condition node labels identify entities with preconditions, Signal node labels identify entities with input signals, Action node labels identify entities with triggering actions, ExpectedResult node labels identify entities with expected results, and Function node labels identify entities with functions or conditions. Graph edge types are set according to relationship type, including TRIGGER, REQUIRE, RESULT_IN, and CONSTRAIN edge types. TRIGGER edge type represents triggering relationships, REQUIRE edge type represents precondition dependencies, RESULT_IN edge type represents result-generating relationships, and CONSTRAIN edge type represents constraint relationships.

[0109] The knowledge graph construction system 500 indexes page_id, table_id, and cell_id to support evidence tracing. The page_id index is used to locate the document page number from which the triple originates, the table_id index is used to locate the table number from which the triple originates, and the cell_id index is used to locate the cell position from which the triple originates. By creating these indexes, any triple relationship can be traced back to the original PDF page number, table number, and cell position, facilitating rapid review and revision by test engineers.

[0110] The Knowledge Graph Construction System 500 performs graph quality assessment, calculating entity coverage, relation coverage, conflict rate, and Precision / Recall based on manually labeled small samples. Entity coverage measures the degree to which entities in the knowledge graph cover entities in the document; relation coverage measures the degree to which relations in the knowledge graph cover relations in the document; and conflict rate measures the proportion of conflicting candidate triples. Precision measures the proportion of correct triples in the extraction results, and Recall measures the proportion of correct triples extracted from the document. The Knowledge Graph Construction System 500 uses low-confidence or high-conflict entries as hard examples for backflow, updating the domain dictionary, prompt template, validation rules, or threshold parameters, forming a closed-loop construction process. This closed-loop construction process enables the Knowledge Graph Construction System 500 to continuously optimize extraction performance.

[0111] The Knowledge Graph Construction System 500 supports retrieving associated preconditions, input signals / thresholds, trigger actions, and expected results based on function / condition nodes, and automatically assembling them into a set of executable test steps. The system provides a graph query interface, accepting function or condition nodes as query input, and traversing the graph to retrieve associated Condition, Signal, Action, and ExpectedResult nodes. The system then assembles the retrieved preconditions, input signals / thresholds, trigger actions, and expected results into a set of executable test steps according to test logic, and uses graph evidence as test case annotations, providing a structured knowledge foundation for automated test case generation and intelligent test task execution.

[0112] Furthermore, a vehicle testing domain knowledge graph construction system based on global semantics was implemented in a high-performance computing environment. The hardware environment utilizes a high-performance server equipped with an NVIDIA RTX 3090 GPU, which provides graphics processing and parallel computing capabilities, supporting inference operations for multimodal large models and large language models. The software environment is developed based on Python, which provides rich machine learning libraries and natural language processing tools, supporting the implementation of processing flows such as visual layout parsing, topic clustering, vectorized computation, and large model invocation. The software environment integrates MinerU for visual layout analysis, Qwen-VL-Plus as a multimodal visual understanding model, and Qwen-Max as a large language model for candidate triple extraction. The final knowledge graph is stored in the Neo4j graph database, which provides visualization support and a graph query interface.

[0113] This invention also discloses a readable storage medium.

[0114] A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any of the above embodiments. The computer-readable storage medium may include any entity or device capable of carrying a computer program, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc. The computer program includes computer program code. The computer program code may be in the form of source code, object code, an executable file, or some intermediate form, etc. The computer-readable storage medium may include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0115] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0116] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a system including a processing module or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0117] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a vehicle test field knowledge graph based on global semantic driving, characterized in that, include: Preprocess the vehicle test PDF document to extract tables and related text. A visual layout parsing tool is used to identify table boundaries, cell coordinates, and row and column indices in a document, outputting a table image that retains a complete two-dimensional layout. This identification process further includes recognizing cell spanning information (row and column crossings) and constructing a structured intermediate representation of the table. This intermediate representation includes table identifiers, page numbers, cell identifiers, cell bounding boxes, row indices, column indices, row spans, column spans, cell text, and header paths obtained from parsing multi-level headers. For merged cells, an expansion rule is applied, forming hierarchical paths between parent and child headers and binding them to each data cell, thereby preserving the logical topological relationship between row or column headers and data cells. The table image is input into a multimodal large model to generate a global semantic description containing the table's business themes and row and column logical relationships. The global semantic description also includes the semantic roles of each column, key constraints, and the overall logic of condition-action-response. The global semantic description is subjected to topic clustering to identify representative entities under each topic. Entities with appropriate granularity and high relevance are selected as seed entities. This includes: extracting candidate entities from the global semantic description, the header path, and the domain key columns; and calculating a comprehensive score for each candidate entity. The comprehensive score is based on the cosine similarity between the candidate entity vector and the topic vector, the frequency of occurrence of the candidate entity, structural importance, and noise penalty. The structural importance includes the distribution entropy of the covered rows and columns, whether it appears in the header path, and whether it is located in a key column. Guided by the seed entity, and combining the table image and the global semantic description, candidate knowledge triples are extracted from the vehicle test PDF document. The extraction of candidate knowledge triples guided by the seed entity includes: designing vehicle test domain-specific extraction prompts, clearly defining the entity types to be identified and the specific relation types to be extracted; integrating the seed entity, the table image, and the global semantic description to form complete input data containing topic focus, visual semantic information, and a global description; locating the seed entity's position in the structured intermediate representation of the table as an anchor point, and generating candidate entity pairs within a controlled range, where the controlled range includes local areas of the same row, column, header path, or test item; and inputting the complete input data into a large... The language model guides the model to extract candidate knowledge triples through a progressive logic: seed entity guidance, judgment of professional logical relationships between entities, and generation of standard triples for head entity-relationship-tail entity. The constraint model outputs triple records according to a predefined pattern, each triple record containing a head entity, relation, tail entity, head entity type, tail entity type, evidence, and confidence level. Pattern validation, type validation, and field validity validation are performed on the model output. When validation fails or the confidence level falls below a threshold, a local re-interrogation strategy is triggered, regenerating only failed or low-confidence fields. For the precondition-triggered action-expected result chain, it checks whether the extracted results can close within the same test item, marking unclosable candidate relations as pending confirmation and preserving the evidence chain for subsequent conflict resolution and manual review. The candidate knowledge triples are subjected to global knowledge fusion, including entity merging, conflict resolution, and new triple inference, to form a globally consistent knowledge graph.

2. The method of claim 1, wherein, The structural importance includes the distribution entropy of the covered rows and columns, whether it appears in the header path, and whether it is located in a key column.

3. The method of claim 1, wherein, The entity consolidation includes: Based on a dictionary and alias table for vehicle testing, rules are standardized and normalized, including Chinese-English translation, abbreviation expansion, synonym mapping, and unit standardization; and When the rules cannot be determined, the similarity of entity vectors is calculated and candidate merging pairs are generated. If the similarity is higher than the merging threshold, a unified canonical name is generated and the original alias is written into the alias list.

4. The method of claim 1, wherein, The conflict resolution includes: For conflicts involving multiple tail entities corresponding to the same head entity and relation, a total confidence score is calculated, based on extraction confidence, document version score, industry standard score, and evidence completeness score; and The relation with the highest total confidence is selected and written into the confirmed set, while the remaining relations are written into the pending verification set and the chain of evidence is preserved.

5. The method according to claim 1, characterized in that, The new triplet inference includes: performing controlled inference using a rule base in the vehicle testing domain, wherein the rules are stored in template form; and attaching an inference rule identifier, a source triplet set, and a confidence decay coefficient to the inference result.

6. A vehicle testing domain knowledge graph construction system based on global semantic driving, applied to the method described in any one of claims 1-5, characterized in that, include: The preprocessing module is configured to preprocess vehicle test PDF documents, extracting tables and related text. The visual layout parsing module is configured to use visual layout parsing tools to identify table boundaries, cell coordinates, and row and column indices in a document, and output a table image that retains the complete two-dimensional layout. The global semantic generation module is configured to input the table image into a multimodal large model and generate a global semantic description that includes the table's business theme and row and column logical relationships; The seed entity filtering module is configured to perform topic clustering on the global semantic description, identify representative entities under each topic, and filter out entities with appropriate granularity and high relevance as seed entities. The triple extraction module is configured to extract candidate knowledge triples from the vehicle test PDF document, guided by the seed entity and combined with the table image and the global semantic description. as well as The global knowledge fusion module is configured to perform global knowledge fusion on the candidate knowledge triples, including entity merging, conflict resolution, and new triple inference, to form a globally consistent knowledge graph.

7. A readable storage medium, characterized by, The readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-5.

Images