A bridge-specific Agentic RAG intelligent question-answering system and method for multi-source detection data fusion.

Abstract

This invention discloses a bridge-specific Agentic RAG intelligent question-answering system and method for multi-source inspection data fusion. The method includes extracting data from bridge inspection reports and parsing the document content type; segmenting the parsed data into title-level blocks for text and / or table types; generating summary text for data parsed into image types; encoding the segmented text, summary text, and images into dense vectors; encoding the parsed text data into sparse vectors; synchronously establishing a multimodal database using the dense vectors, sparse vectors, and keyword indexes; extracting defect information from the text and table data in the bridge inspection reports to obtain structured fields; mapping the structured fields, their source information, and location information into a structure with hierarchical and relational relationships of bridge—component—location—time. This invention achieves consistency between component descriptions and engineering semantics, improving the credibility of the question-answering process.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, and in particular to a bridge-specific Agentic RAG intelligent question-answering system and method for multi-source detection data fusion. Background Technology

[0002] As a crucial transportation infrastructure, the operational status of bridges directly impacts traffic safety and maintenance decisions. With the continuous increase in the number of bridges, their service life, and traffic loads, bridge maintenance management has gradually evolved from a method of "manual review of paper / electronic documents + experience-based judgment" to a data-driven and information-based management model centered on periodic inspection reports. In practice, periodic inspection reports are typically stored in PDF, Word, or other document formats and are accumulated over time in maintenance management platforms or archive systems, forming a large-scale historical database spanning multiple years, components, and types of defects.

[0003] Against this backdrop, common information needs of engineers in their daily work include: retrieving historical records by bridge name, component, location, time, and / or type of damage; comparing damage evolution; quickly locating report evidence (text, tables, photos); and generating decision-oriented question-and-answer conclusions. Therefore, the technical approach of "structured data storage + retrieval + question-and-answer" for scheduled inspection reports is gradually becoming an important direction for bridge maintenance informatization.

[0004] Existing RAG (Retrieval-Augmented Generation) question-answering methods primarily rely on a general approach of "segmentation + hybrid retrieval + rearrangement + LLM generation," lacking a "bridge-component-location-time" object model and component list constraints specific to the bridge domain. Therefore, during question-answer generation, issues such as inconsistent component names, confusion regarding locations or cross-numbering, and referencing descriptions of components not belonging to the bridge itself can easily arise, impacting the credibility and feasibility of the answers.

[0005] A new technology is urgently needed to solve the above-mentioned problem of inconsistencies in engineering semantics. Summary of the Invention

[0006] The purpose of this invention is to provide artificial intelligence and natural language processing to solve the technical problems described above. The preferred technical solutions among the various technical solutions provided by this invention and their numerous technical effects are detailed below.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] This invention provides a bridge-specific Agentic RAG intelligent question-answering method for multi-source detection data fusion, comprising the following steps:

[0009] Data extraction and document content type parsing are performed on bridge inspection reports. Data parsed into text and / or table types is segmented into title-level blocks to obtain block text. Data parsed into image types is used to generate summary text based on the context.

[0010] The segmented text, the summary text, and the images are encoded into dense vectors, and the data parsed into text type is encoded into sparse vectors. The dense vectors, the sparse vectors, and the keyword indexes corresponding to the dense vectors and the sparse vectors are synchronously used to establish a multimodal database corresponding to the fusion retrieval index.

[0011] To bind source information and location information to the structured fields obtained from the text and tabular data of the bridge inspection report for defect information;

[0012] The structured fields, their source information, and location information are mapped into a structure with a hierarchical and relational relationship of bridge-component-part-time. The actual range of components and parts of the bridge is determined based on the bridge component list.

[0013] In one or more embodiments, if the length of the segmented text is greater than a set value, the segmented text is semantically segmented.

[0014] In one or more embodiments, it further includes:

[0015] The query is mapped to bridge-related professional terminology, and non-standard expressions, including colloquialisms and abbreviations, are rewritten into standardized terms consistent with the inspection report and object model. Ambiguous expressions are disambiguated. The object model is a structural generation model, and the ambiguous expressions include component names, defect names, and times.

[0016] In one or more embodiments, it further includes:

[0017] Determine whether the disambiguated query accurately expresses the user's question. If not, rewrite the query using the standard rewritten prompt word template to obtain a standardized query expression for retrieval and constraint generation.

[0018] In one or more embodiments, subject elements are retrieved from the disambiguated query or standardized query representation, filtering conditions are generated based on the subject elements, the structure generated from the bridge object model, and the list of bridge components, and the filtering conditions are converted into retrieval constraints for structured field records.

[0019] In one or more embodiments, the key elements include bridge name, type of damage, components, location, and / or time range.

[0020] In one or more embodiments, under the constraints of the retrieval constraints, a hybrid retrieval combining semantic retrieval and keyword retrieval is performed to retrieve corresponding evidence from the multimodal database and merge all evidence to obtain a candidate evidence set; wherein, the evidence includes text fragments, table records and image descriptions.

[0021] In one or more embodiments, the candidate evidence set is rearranged, the top K pieces of evidence after rearrangement are selected, and a large model is used to determine the information sufficiency of the top K pieces of evidence;

[0022] When the information is sufficient, the first K pieces of evidence and their structured field records are input into the generation module to generate a response; when the information is insufficient, the answerable part based on the existing evidence is output, and the content that cannot be determined from the evidence in the current test report is indicated.

[0023] According to another aspect of the present invention, a bridge-specific Agentic RAG intelligent question-answering system for multi-source detection data fusion is also provided, comprising:

[0024] The data extraction and file type parsing module is used to extract data and parse file content types from bridge inspection reports;

[0025] The text segmentation module is used to perform header-level segmentation of data parsed into text or table types to obtain segmented text.

[0026] The text summary module is used to generate summary text from data parsed as image type, combined with the context content.

[0027] The encoding module is used to encode the segmented text, the summary text, and the image into dense vectors, and to encode the data parsed as text type into sparse vectors.

[0028] The vectorization embedding module is used to synchronously establish a multimodal database corresponding to the fusion retrieval index by combining the dense vector, the sparse vector, and the keyword index corresponding to the obtained dense vector and the keyword index corresponding to the sparse vector.

[0029] The defect information extraction module is used to extract defect information from the text data and table data in the bridge inspection report to obtain structured fields, and bind source information and location information to each structured field;

[0030] The object model solidification module is used to map structured fields, their source information, and location information into a structure with hierarchical and relational relationships of bridge-component-part-time, and to determine the actual range of components and parts of the bridge based on the bridge component list.

[0031] In one or more embodiments, it further includes:

[0032] The rewriting and disambiguation module is used to map bridge-related professional terms to queries, rewriting non-standard expressions, including colloquialisms and abbreviations, into standardized terms consistent with the inspection report and object model, and disambiguating ambiguous expressions; wherein, the object model is a structure generation model, and the ambiguous expressions include component names, defect names, and time;

[0033] The judgment module is used to determine whether the query after disambiguation accurately expresses the user's question. If the judgment is no, the query is rewritten using the standard rewritten prompt word template to obtain a standardized query expression for retrieval and constraint generation.

[0034] The retrieval constraint generation module is used to retrieve the main elements from the disambiguated query or standardized query expression, generate filtering conditions based on the main elements, the structure generated by the bridge object model, and the list of bridge components, and convert the filtering conditions into retrieval constraints for structured field records.

[0035] The candidate evidence generation module is used to perform a hybrid retrieval combining semantic retrieval and keyword retrieval under the constraints of the retrieval constraints, retrieve the corresponding evidence from the multimodal database, and merge all the evidence to obtain a candidate evidence set.

[0036] The output module is used to rearrange the candidate evidence set, select the top K pieces of evidence after rearrangement, and use a large model to determine the information sufficiency of the top K pieces of evidence. When the information is sufficient, the top K pieces of evidence and their structured field records are input into the generation module to generate a response. When the information is insufficient, the response is output based on the existing evidence, and the content that cannot be determined from the evidence in the current detection report is indicated.

[0037] Implementing one of the above-described technical solutions of the present invention has the following advantages or beneficial effects:

[0038] This invention extracts text, table, and image information from bridge inspection reports and unifies them into searchable evidence units, achieving multi-source and multi-modal data extraction and unified storage for bridge scheduled inspection reports. Simultaneously, by introducing a bridge object model and a Bill of Materials (BOM), core elements such as "bridge-component-location-disease type-time-indicator-grade" are structurally recorded and bound to their source location, establishing a "bridge-component-location-time" object model. The component list limits the retrieval and generation scope, ensuring consistency with the engineering project. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0040] Figure 1 This is a flowchart of a bridge-specific Agentic RAG intelligent question-answering method for multi-source detection data fusion according to an embodiment of the present invention;

[0041] Figure 2 This is a structural block diagram of a bridge-specific Agentic RAG intelligent question-answering system for multi-source detection data fusion, according to an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the present invention clearer, various exemplary embodiments described below will be referenced to the accompanying drawings, which form part of the exemplary embodiments, illustrating various exemplary embodiments that may be used to implement the present invention. Unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. It should be understood that they are merely examples of processes, methods, and apparatuses consistent with some aspects of the present invention disclosed as detailed in the appended claims, and other embodiments may be used, or structural and functional modifications may be made to the embodiments listed herein without departing from the scope and spirit of the present invention.

[0043] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," etc., indicate the orientation or positional relationship based on the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the referred element must have a specific orientation, or be constructed and operated in a specific orientation. The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. The term "multiple" means two or more. The terms "connected" and "linked" should be interpreted broadly, for example, they can be fixed connections, detachable connections, integral connections, mechanical connections, electrical connections, communication connections, direct connections, indirect connections through an intermediate medium, and can be the internal connection of two elements or the interaction relationship between two elements. The term "and / or" includes any and all combinations of one or more of the related listed items. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0044] To illustrate the technical solution described in this invention, specific embodiments are described below, showing only the parts related to the embodiments of this invention.

[0045] Example 1: As Figure 1 As shown, this invention provides a bridge-specific Agentic RAG intelligent question-answering method for multi-source detection data fusion, comprising the following steps:

[0046] S100. Extract data and parse the file content type of the bridge inspection report. Perform header-level segmentation on the parsed data as text and / or table type to obtain segmented text. For the parsed data as image type, generate summary text based on the context.

[0047] In a specific implementation, the bridge inspection report is first accessed. The Dots.OCR model is used to extract and parse multi-source data from the bridge inspection report content, generating a list of Markdown files and parsing the file content types. For text or table content, Markdown heading-level segmentation is used. If the length of the segmented content exceeds a set value (e.g., 1000), a semantic segmentation strategy is used to further segment the content to prevent semantic fragmentation. The segmented content is then stored in a vectorized queue. For image data, a large model is used to summarize the image and its contextual text content, generating summary text. Then, the Base64 links of the images are extracted, decoded, and saved as PNG format. Finally, the multimodal embedding model GME-Qwen2-VL-7B is used to encode the segmented text, image summary (summary text), and images into dense vectors, which are stored in the Milvus vectorized database to form a searchable and traceable report knowledge base.

[0048] S200. Encode the segmented text, summary text, and images into dense vectors, and encode the parsed text data into sparse vectors. Synchronously establish a multimodal database corresponding to the fusion retrieval index by combining the dense vectors, sparse vectors, and keyword indexes corresponding to the obtained dense vectors and sparse vectors.

[0049] In a specific embodiment, text-type data is encoded into sparse vectors using the BM25 algorithm.

[0050] In a specific embodiment, the sparse index corresponding to the sparse vector is obtained by the DAAT algorithm, and the dense index corresponding to the dense vector is obtained by the HNSW algorithm. The sparse index and the dense index are then combined to obtain the fused retrieval index.

[0051] S300: Bind source information and location information to the structured fields obtained from the text and tabular data of the bridge inspection report for defect information.

[0052] In a specific embodiment, the content of the test report (text, table) is used to extract disease information. Information that is strongly related to the questions and answers, such as disease type, disease description, location or part, quantity or index, grade assessment, and test time, is extracted as structured fields. Each field record is bound with source information (such as test report file, page number, chapter, table position, image number, etc.) and confidence information to form a fielded evidence record that can be used for filtering, verification and citation.

[0053] The above extraction specifically includes: for disease detection-related chapters, after chunking (the chunking method in RAG), the disease information in each chunk is extracted by a multimodal model, such as gme-Qwen2-VL-7B-Instruct.

[0054] S400: Map the structured fields, their source information, and location information into a structure with a hierarchical and relational relationship of "bridge-component-location-time", and determine the actual range of components and locations of the bridge based on the bridge component list.

[0055] In a specific embodiment, this step establishes a bridge object model based on bridge engineering semantics, forming at least a hierarchy and association of "bridge-component-part-time". It also solidifies the scope of actual components and parts of the bridge in conjunction with the Bill of Materials (BOM), ensuring that subsequent retrieval and generation are always limited to the real component system of the target bridge, thereby providing a basis for consistency constraints and disambiguation.

[0056] To make it easier to understand, the bridge object model described above defines the rules and organization methods, while the structure is the specific data representation generated according to these rules.

[0057] In this embodiment, one type of structure is:

[0058] {

[0059] Bridge Name: "Bridge No. 3"

[0060] Component type: "Box girder"

[0061] Component part: "web"

[0062] Disease type: "Cracks"

[0063] Date and time: 20251120

[0064] }

[0065] It should be noted that the actual components and parts of the bridge are determined based on the list of bridge components (of course, not limited to being determined manually), and this is used as a constraint for subsequent retrieval and mapping.

[0066] In one or more embodiments, it further includes:

[0067] The query is mapped using bridge-related technical terms. Non-standard expressions, including colloquialisms and abbreviations, are rewritten into standardized terms consistent with the inspection report and object model. Ambiguous expressions are disambiguated. The object model is a structural generation model, and ambiguous expressions include component names, defect names, and timestamps. Specifically, there are ambiguities in names, defect names, and timestamps.

[0068] Furthermore, it is determined whether the disambiguated query accurately expresses the user's question. If not, the query is rewritten using a standard rewritten prompt word template to obtain a standardized query expression used for retrieval and constraint generation.

[0069] It should be noted that specialized terminology in the bridge field can be obtained from a bridge-specific terminology database. A bridge-specific terminology database can be established using a combination of a manually compiled glossary and a thesaurus, or by extracting terminology from standards, specifications, and / or industry standards.

[0070] When a user submits a query, this embodiment can use a large model to map the query to bridge-related professional terminology, rewriting colloquial expressions, abbreviations, and other non-standard expressions into standardized terminology consistent with the report and object models. Furthermore, a general large model is used to disambiguate potentially ambiguous component names, defect names, time expressions, etc. The large model determines whether the disambiguated query accurately expresses the user's question. If not, a standard rewritten prompt word template is input into the large model to rewrite the query, resulting in a standardized query expression that can be used for retrieval and constraint generation.

[0071] In specific implementations, for ambiguous queries such as common spelling errors (e.g., bridge -> bridge connection) or inaccurate disease types (e.g., steel bar corrosion -> steel bar rust), the large model can rewrite the query to eliminate ambiguity without fine-tuning, based on its own capabilities, or by comparing and mapping with a terminology database.

[0072] Based on the above embodiments, a system prompt based on a large model is as follows:

[0073] [Original User Query]

[0074] {{user_query}}

[0075] [Glossary of Bridge-Related Terms / Object Modeling Terms]

[0076] {{domain_terms}}

[0077] [Collection of Standard Names for Bridge Components]

[0078] {{component_terms}}

[0079] [Collection of Standard Names for Bridge Defects]

[0080] {{disease_terms}}

[0081] [Standardizations for Expressing Time]

[0082] {{time_rules}}

[0083] You are a query standardization assistant in the field of bridge maintenance and disease diagnosis. Based on the above content, you convert user-submitted natural language queries into standardized query expressions suitable for bridge inspection report retrieval, object model matching, and maintenance suggestion generation.

[0084] Your tasks include:

[0085] Identify colloquial expressions, abbreviations, alternative names, and non-standard terms in user queries, and map non-standard expressions to standard terms in the bridge industry;

[0086] Identify and resolve ambiguities in queries regarding bridge component names, defect names, time expressions, spatial location expressions, and scope of inspection objects;

[0087] Determine whether the disambiguated query accurately represents the user's intent;

[0088] If the expression can be accurately represented, then output a standardized query.

[0089] If the expression cannot be accurately expressed, explain the reason and provide basic information for the next stage of query rewriting.

[0090] Please strictly adhere to the following principles:

[0091] Without changing the core intent of the user's original question;

[0092] Prioritize the use of standardized terminology in bridge inspection, bridge maintenance, and bridge structural health monitoring scenarios;

[0093] For expressions with multiple possible interpretations, the most reasonable explanation should be given by combining bridge expertise.

[0094] If a unique determination cannot be made, the points of ambiguity should be clearly marked, and facts should not be fabricated subjectively.

[0095] The output should be concise, professional, and structured, suitable for subsequent retrieval and constraint generation.

[0096] In one or more embodiments, subject elements are retrieved from the disambiguated query or standardized query statement. Filtering conditions are generated based on the subject elements and the list of structures and bridge components generated from the bridge object model. These filtering conditions are then converted into retrieval constraints for structured field records. The subject elements include bridge name, damage type, components, location, and / or time range.

[0097] This embodiment extracts the main search elements from standardized queries, including at least the bridge name, defect type, component / location, and time range. It generates filtering conditions based on the bridge object model and BOM component list, and matches structured field records to form search constraints. This allows the search process to prioritize screening candidate evidence within the target bridge, valid components, locations, and specified time range, reducing irrelevant recalls across bridges, components, or time periods.

[0098] In one or more embodiments, under the constraints of retrieval constraints, a hybrid retrieval combining semantic retrieval and keyword retrieval is performed to recall corresponding evidence from a multimodal database, and all evidence is merged to obtain a candidate evidence set. The evidence includes text fragments, table records, and image descriptions.

[0099] It should be noted that the candidate set can simultaneously contain evidence of precise term matching and evidence of semantically related but different expressions, providing sufficient basis for subsequent ranking and generation.

[0100] In one or more embodiments, the candidate evidence set is rearranged, the top K pieces of evidence after rearrangement are selected, and a large model is used to determine the information sufficiency of the top K pieces of evidence. When the information is sufficient, the top K pieces of evidence and their structured field records are input into the generation module to generate a response. When the information is insufficient, the answerable part based on the existing evidence is output, and the content that cannot be determined from the evidence in the current detection report is prompted.

[0101] It should be noted that the candidate evidence set is rearranged, taking into account factors such as the matching degree of query elements, consistency of structured fields, consistency of BOM, and reliability of evidence sources to complete the sorting and return the Top-K evidence. At the same time, the large model judges the information sufficiency of the Top-K evidence to determine whether the current evidence is sufficient to support the conclusion range and accuracy requirements required to answer the question, thereby avoiding giving unreliable conclusions directly when the evidence is insufficient.

[0102] When performing information sufficiency assessment, the RAGAs framework (an open-source framework specifically designed for evaluating RAG systems) is added to evaluate the RAG retrieval results. The metric is Context Precision / Recall, with a threshold of 0.6. A value higher than 0.6 is considered accurate.

[0103] The Top-K evidence and its structured field records are input into the generation module. Based on the structured fields and BOM component system, constrained hints and generation rules are generated to restrict the large model to organize answers only within the scope of retrieved evidence and maintain the consistency of "bridge-component-location-time", thereby suppressing illusions and inconsistent descriptions. The final output is a question-and-answer result that is traceable to the source of evidence.

[0104] In summary, this embodiment extracts text, table, and image information from bridge inspection reports and unifies them into searchable evidence units, achieving multi-source / multi-modal data extraction and unified storage for bridge scheduled inspection reports. Simultaneously, by introducing a bridge object model and a Bill of Materials (BOM), core elements such as "bridge-component-location-disease type-time-indicator-grade" are structurally recorded and bound to their source locations, establishing a "bridge-component-location-time" object model. The component list limits the retrieval and generation scope, ensuring project consistency.

[0105] Example 2: Figure 2 As shown, this embodiment also provides a bridge-specific Agentic RAG intelligent question-answering system for multi-source detection data fusion, used to implement the bridge-specific Agentic RAG intelligent question-answering method for multi-source detection data fusion described in Embodiment 1. This system includes:

[0106] The data extraction and file type parsing module is used to extract data and parse the file content type of bridge inspection reports.

[0107] The text segmentation module is used to perform header-level segmentation of data parsed as text or table types to obtain segmented text.

[0108] The text summary module is used to generate summary text from data parsed as image type, combined with the context content.

[0109] The encoding module is used to encode block text, summary text, and images into dense vectors, and to encode data parsed as text into sparse vectors.

[0110] The vectorized embedding module is used to synchronously build a multimodal database corresponding to the fusion retrieval index by combining dense vectors, sparse vectors, and the keyword indexes corresponding to the obtained dense vectors and sparse vectors.

[0111] The defect information extraction module is used to extract defect information from the text and tabular data in the bridge inspection report into structured fields, and bind source information and location information to each structured field.

[0112] The object model solidification module is used to map structured fields, their source information, and location information into structures with hierarchical and relational relationships of "bridge-component-part-time". Based on the bridge component list, the actual range of components and parts of the bridge is determined.

[0113] Furthermore, it also includes:

[0114] The rewriting and disambiguation module is used to map bridge-related professional terms to queries, rewriting non-standard expressions, including colloquialisms and abbreviations, into standardized terms consistent with the inspection report and object model, and disambiguating potentially ambiguous expressions such as component names, defect names, and time.

[0115] The judgment module is used to determine whether the query after disambiguation accurately expresses the user's question. If the judgment is not correct, the query is rewritten using the standard rewritten prompt word template to obtain a standardized query expression for retrieval and constraint generation.

[0116] The retrieval constraint generation module is used to retrieve the main elements from the disambiguated query or standardized query expression, generate filtering conditions based on the main elements and the list of structures and bridge components generated from the bridge object model, and convert the filtering conditions into retrieval constraints for structured field records.

[0117] The candidate evidence generation module is used to perform a hybrid retrieval combining semantic retrieval and keyword retrieval under the constraints of retrieval constraints, recall the corresponding evidence from the multimodal database, and merge all the evidence to obtain a candidate evidence set.

[0118] The output module is used to rearrange the candidate evidence set, select the top K pieces of evidence after rearrangement, and use a large model to determine the information sufficiency of the top K pieces of evidence. When the information is sufficient, the top K pieces of evidence and their structured field records are input into the generation module to generate a response. When the information is insufficient, the output is the answerable part based on the existing evidence, and the content that cannot be determined from the evidence in the current detection report is indicated.

[0119] This system extracts text, tables, and images from bridge inspection reports and unifies them into searchable evidence units, achieving multi-source / multi-modal data extraction and unified storage for bridge inspection reports. By introducing a bridge object model and a Bill of Materials (BOM), core elements such as "bridge-component-location-disease type-time-indicator-grade" are structurally recorded and bound to their source locations, establishing a "bridge-component-location-time" object model. The component list limits the retrieval and generation scope, ensuring project consistency. Furthermore, the system performs unified parsing and key element extraction on the text, tables, and images of the inspection reports, establishing cross-modal associations to achieve unified recall and organized presentation, forming a traceable evidence chain of "text-data-image." Terminology standardization and disambiguation are performed on user questions to accurately extract elements such as bridge name, component, location, disease, and time, improving recall stability and accuracy and reducing missed and false positives. Furthermore, an evidence-constrained RAG (Related Information Group) question-and-answer system is constructed, binding source identifiers such as chapters, page numbers, tables, and images during generation, reducing the risk of unsubstantiated generation and improving verifiability and credibility.

[0120] It should be noted that the technical features of the methods executed by each module in this embodiment are consistent with those in Embodiment 1, as described in Embodiment 1.

[0121] Example 3: This example also provides a processing device, including:

[0122] The configuration module is used to configure and edit the protocol file corresponding to the bridge-specific Agentic RAG intelligent question-answering method for multi-source detection data fusion, as described above. The protocol file is in binary format. Specifically, the steps for configuring and editing the protocol file are as described in Embodiment 1.

[0123] The parsing module is used to parse the protocol file into JSON format data. Specifically, the steps for parsing the protocol file into JSON format data are as described in Example 1.

[0124] Example 4: As will be understood by those skilled in the art, all or part of the features / steps of the above method embodiments can be implemented by methods, data processing systems, or computer programs. These features can be implemented without hardware, entirely in software, or in a combination of hardware and software. The aforementioned computer program can be stored in one or more computer-readable storage media. When the computer program is executed (e.g., by a processor), it performs the steps of the above-described embodiment of a bridge-specific Agentic RAG intelligent question-answering method for multi-source detection data fusion.

[0125] The aforementioned storage media capable of storing program code include: static disks, solid-state drives, random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), optical storage devices, magnetic storage devices, flash memory, magnetic disks or optical disks, and / or combinations of the above devices, that is, they can be implemented by any type of volatile or non-volatile storage devices or combinations thereof.

[0126] Example 5: The present invention also provides an electronic device, comprising:

[0127] One or more processors;

[0128] A memory for storing one or more computer programs, and one or more processors for executing the one or more computer programs stored in the memory to cause the one or more processors to perform the features / steps of a bridge-specific Agentic RAG intelligent question answering method for multi-source detection data fusion as described in Embodiment 1.

[0129] It should be understood that the above embodiments are only special cases and do not indicate that the present invention is implemented in such a way.

[0130] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0131] The above description is merely a preferred embodiment of the present invention. Those skilled in the art will understand that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the present invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A bridge-specific Agentic RAG intelligent question-answering method for multi-source detection data fusion, characterized in that, Includes the following steps: Data extraction and document content type parsing are performed on bridge inspection reports. Data parsed into text and / or table types is segmented into title-level blocks to obtain block text. Data parsed into image types is used to generate summary text based on the context. The segmented text, the summary text, and the images are encoded into dense vectors, and the data parsed into text type is encoded into sparse vectors. The dense vectors, the sparse vectors, and the keyword indexes corresponding to the dense vectors and the sparse vectors are synchronously used to establish a multimodal database corresponding to the fusion retrieval index. To bind source information and location information to the structured fields obtained from the text and tabular data of the bridge inspection report for defect information; The structured fields, their source information, and location information are mapped into a structure with a hierarchical and relational relationship of bridge-component-part-time. The actual range of components and parts of the bridge is determined based on the bridge component list. Also includes: The query is mapped to bridge-related professional terminology, and non-standard expressions, including colloquialisms and abbreviations, are rewritten into standardized terms consistent with the inspection report and object model. Ambiguous expressions are disambiguated. The object model is a structural generation model, and the ambiguous expressions include component names, defect names, and times. Determine whether the disambiguation-processed query accurately expresses the user's question. If not, rewrite the query using the standard rewritten prompt word template to obtain a standardized query expression for retrieval and constraint generation. The main elements are retrieved from the disambiguated query or standardized query expression. Filtering conditions are generated based on the main elements, the structure generated from the bridge object model, and the list of bridge components. The filtering conditions are then converted into retrieval constraints for structured field records. The main elements include the bridge name, type of damage, components, location, and / or time range; Under the constraints of the retrieval constraints, a hybrid retrieval combining semantic retrieval and keyword retrieval is performed to retrieve corresponding evidence from the multimodal database and merge all evidence to obtain a candidate evidence set; wherein, the evidence includes text fragments, table records and image descriptions.

2. The bridge-specific Agentic RAG intelligent question-answering method for multi-source detection data fusion according to claim 1, characterized in that, If the length of the segmented text is greater than a set value, semantic segmentation is performed on the segmented text.

3. The bridge specialized Agentic RAG intelligent question and answer method for multi-source detection data fusion according to claim 2, characterized in that, The candidate evidence set is rearranged, and the top K pieces of evidence after rearrangement are selected. A large model is then used to determine the information sufficiency of the top K pieces of evidence. When sufficient information is available, the first K pieces of evidence and their structured field records are input into the generation module to generate a response; When information is insufficient, output the answerable portion based on existing evidence and indicate what cannot be determined from the evidence in the current test report.

4. A bridge-specific Agentic RAG intelligent question-answering system for multi-source detection data fusion, characterized in that, To implement the bridge-specific Agentic RAG intelligent question answering method for multi-source detection data fusion as described in any one of claims 1-3, the method includes: The data extraction and file type parsing module is used to extract data and parse file content types from bridge inspection reports; The text segmentation module is used to perform header-level segmentation of data parsed into text or table types to obtain segmented text. The text summary module is used to generate summary text from data parsed as image type, combined with the context content. The encoding module is used to encode the segmented text, the summary text, and the image into dense vectors, and to encode the data parsed as text type into sparse vectors. The vectorization embedding module is used to synchronously establish a multimodal database corresponding to the fusion retrieval index by combining the dense vector, the sparse vector, and the keyword index corresponding to the obtained dense vector and the keyword index corresponding to the sparse vector. The defect information extraction module is used to extract defect information from the text data and table data in the bridge inspection report to obtain structured fields, and bind source information and location information to each structured field; The object model solidification module is used to map structured fields, their source information, and location information into a structure with hierarchical and relational relationships of bridge-component-part-time, and to determine the actual range of components and parts of the bridge based on the bridge component list.

5. The bridge specialized Agentic RAG intelligent question-answering system for multi-source detection data fusion according to claim 4, characterized in that, Also includes: The rewriting and disambiguation module is used to map bridge-related professional terms to queries, rewriting non-standard expressions, including colloquialisms and abbreviations, into standardized terms consistent with the inspection report and object model, and disambiguating ambiguous expressions; wherein, the object model is a structure generation model, and the ambiguous expressions include component names, defect names, and time; The judgment module is used to determine whether the query after disambiguation accurately expresses the user's question. If the judgment is no, the query is rewritten using the standard rewritten prompt word template to obtain a standardized query expression for retrieval and constraint generation. The retrieval constraint generation module is used to retrieve the main elements from the disambiguated query or standardized query expression, generate filtering conditions based on the main elements, the structure generated by the bridge object model, and the list of bridge components, and convert the filtering conditions into retrieval constraints for structured field records. The candidate evidence generation module is used to perform a hybrid retrieval combining semantic retrieval and keyword retrieval under the constraints of the retrieval constraints, retrieve the corresponding evidence from the multimodal database, and merge all the evidence to obtain a candidate evidence set. The output module is used to rearrange the candidate evidence set, select the top K pieces of evidence after rearrangement, and use a large model to determine the information sufficiency of the top K pieces of evidence. When the information is sufficient, the top K pieces of evidence and their structured field records are input into the generation module to generate a response. When the information is insufficient, the response is output based on the existing evidence, and the content that cannot be determined from the evidence in the current detection report is indicated.

Images