Engineering specification intelligent question-answering system building method oriented to road surface health management

Abstract

The invention discloses a construction method of an engineering specification intelligent question-answering system oriented to pavement health management. Currently, knowledge processing in the field of road engineering faces the problems of tedious information retrieval, limited application of a large language model and the like, and an efficient question-answering system is urgently needed for intelligent road development. The system is built through the steps of data preprocessing, rewriting, retrieval optimization, model testing and the like. A BERT text filter screening problem is introduced, a road industry specification context is divided, a three-layer retrieval mechanism of a vector, a full text and a knowledge graph is constructed, and a basic large model and an RAG technology are applied. According to the method, efficient retrieval and quick response are achieved, the answer accuracy is improved, the data quality and retrieval efficiency are guaranteed, industry knowledge resources are integrated, the requirement of an intelligent highway for accurately obtaining industry specifications is met, and the problem of knowledge processing and intelligent interaction in the field of road engineering is effectively solved.

Description

Technical Field

[0001] The present invention relates to the technical fields of natural language processing and artificial intelligence applications, and in particular to a method for building an intelligent Q&A system for engineering specifications for pavement health management. Background Art

[0002] (1) Development of Artificial Intelligence and Large Language Models

[0003] In recent years, machine learning has developed rapidly, driving artificial intelligence to a new height. In the field of natural language processing, large language models (LLMs) have achieved remarkable results. Based on the Transformer network framework, with the help of technologies such as multi-head attention mechanisms and residual networks, it effectively overcomes the problems of long-distance sentence dependencies and deep network layers, endows the model with parallel processing capabilities, greatly improves computational efficiency and scalability, meets the needs of large-scale parallel computing, and can better understand and follow human natural language instructions. Pre-trained language models such as BERT and GPT have opened up a pre-training mode of self-supervised learning for unlabeled text data, enabling the model to learn a vast amount of language content. Based on these models, only a small amount of specific task data is needed for fine-tuning to obtain a customized training model.

[0004] (2) Requirements and Current Situation in the Field of Road Engineering

[0005] As a key component of urban and national infrastructure, road engineering is related to people's travel and cargo transportation, integrating multi-disciplinary knowledge such as civil engineering, traffic engineering, environmental science, and urban planning, and many factors need to be considered comprehensively. During the construction process, specification and standard documents are the technical guidelines that must be followed, covering the basic requirements and quality standards for all aspects such as design, construction, supervision, and maintenance, ensuring the quality and safety of road engineering. By implementing and consulting relevant specifications and standards, requirements for engineering materials, construction methods, quality control, etc. are clarified to ensure the reliability and durability of the road.

[0006] (3) Existing Technical Problems

[0007] Main Problems

[0008] Problems in information retrieval and processing: During the road engineering planning, design, and construction stages, decision-makers need to refer to a vast amount of specifications and standards. The information integration work is cumbersome, and a large amount of manpower and time are required to retrieve, gather, and summarize the data.

[0009] Bottlenecks in the Application of Large Language Models: Although large language models perform well in natural language processing, they have many limitations in the field of road engineering. Their Chinese processing ability is relatively weak, the deployment is difficult, they cannot obtain the latest specification content in real time, and misleading "hallucinated facts" are likely to occur. In the face of professional terms and complex logical structures, the answers are inaccurate; trained on general data, it is difficult to meet personalized needs and has poor synergy with the professional field. Although fine-tuning the large model can improve performance in specific scenarios, it may cause "catastrophic forgetting", and the computing cost is high, which poses an obstacle to organizations with limited resources.

[0010] Minor Issues: With the development of technologies such as 5G, the Internet of Things, and big data, the demand for smart roads has increased sharply. Accurately and conveniently obtaining road engineering industry specifications has become the key to the development of the field. Existing systems are difficult to meet this demand, and there is an urgent need to build an efficient intelligent question-answering system for road engineering specifications to integrate industry knowledge resources. Summary of the Invention

[0011] The purpose of the present invention is to provide a method for building an intelligent question-answering system for engineering specifications for pavement health management. The intelligent question-answering system for road engineering specifications based on a large model can quickly locate relevant texts through semantic similarity calculation, reducing the time and effort of manual collation and browsing, reducing the error tolerance rate, and achieving rapid response to queries. The special knowledge base constructed by the present invention integrates the knowledge resources in the field of road engineering to meet the demand for accurately and conveniently obtaining road engineering industry specifications in the development of smart roads.

[0012] The present invention is achieved through the following technical solutions:

[0013] A method for building an intelligent question-answering system for engineering specifications for pavement health management, comprising the following steps:

[0014] Step S1, Data Preprocessing:

[0015] Data Collection: Collect relevant road engineering specifications, standards, case studies, relevant policies, and on-site construction consultation logs;

[0016] Data Cleaning: Clean the data to remove invalid, incorrect, and incomplete information;

[0017] Data Masking: Process sensitive data and comply with data protection regulations;

[0018] Low-Frequency Filtering: Remove low-frequency words or phrases;

[0019] Structured Transformation: Convert unstructured data into a structured format;

[0020] AI Extraction to Form Question-Answer Pairs: Use an artificial intelligence model to extract questions and answers from structured data to form preliminary question-answer pairs;

[0021] Step S2, Data Supplementary Writing:

[0022] Manually supplement the explicit context of the Q&A pairs: Professionals supplement the missing context information of the Q&A pairs; Manually supplement the implicit context of the Q&A pairs: Supplement the implicit context to form the Markdown format; Slice and enter the vector + full-text knowledge base: Slice and integrate the data into the vector and full-text knowledge base, perform retrieval and rearrangement optimization to form a vector + full-text double-layer retrieval structure;

[0023] Step S3, Retrieval Optimization:

[0024] Design the knowledge graph structure: Construct the knowledge graph framework;

[0025] Collect the actual problems at the construction site: Continuously collect the actual problems at the construction site;

[0026] Improve and import the knowledge graph database: Improve the collected knowledge, import it into the knowledge graph database, and combine the actual engineering problems with the specifications and standards;

[0027] Step S4, Model Testing:

[0028] Form a three-layer retrieval of vector + full-text + knowledge graph: Combine the vector, full-text, and knowledge graph to construct a three-layer retrieval mechanism;

[0029] Single-round testing: Conduct preliminary model testing to evaluate the system performance;

[0030] Multi-round testing: Conduct multi-round testing to examine the continuous dialogue performance of the system;

[0031] Iterative optimization: Iteratively optimize the model according to the test results.

[0032] Further as an improvement of the technical solution of the present invention, introduce a BERT-based text filter to screen and filter questions; The development process of the text filter is as follows: Use the BERT model to process the training data, and the output generated by BERT is sent to the fully connected layer to obtain the text classification token [CLS], and train based on the existing labels in the dataset. Only the parameters of the fully connected layer are adjusted during training.

[0033] Further as an improvement of the technical solution of the present invention, when BERT is used for text classification, use the specific word vector H output by BERT combined with the softmax function to construct a classifier, and the softmax function is:

[0034]

[0035] In the formula: x is a vector, x iare the elements therein; the classifier predicts the probabilities of different class labels L; in this method, a fully connected layer is introduced to calculate the probability of each label. The input vector dimension of the fully connected layer is 768, including two hidden layers. The dimension of the first hidden layer is 384, the second dimension is 768, and the output layer dimension depends on the number of classes. Since the task is a binary classification problem, the output layer dimension is 2. The label with the highest probability is selected as the classification result of [CLS].

[0036] Furthermore, as an improvement of the technical solution of the present invention, when dealing with road industry specifications, the document content is divided into explicit context and implicit context; the explicit context is included in the head and tail of the document, which is the context shared by content slices, mainly the industry standard name; the implicit context is the key professional terms and chapters in the specification, corresponding content categories and manually interpreted tabular data; question-and-answer pairs are extracted through manual recognition combined with AI technology to correct and rewrite the slice content to be put into the knowledge base.

[0037] Furthermore, as an improvement of the technical solution of the present invention, a three-layer retrieval mechanism including vector retrieval, full-text retrieval, and knowledge graph retrieval is constructed; the vector retrieval technology converts text data into points in a vector space, uses a greedy algorithm to perform similarity matching on keyword features, calculates the one-way greedy matching score of two sequences, and the final GM score is the average of the greedy matching scores in two directions; the full-text retrieval deeply analyzes the text content and supports complex keyword combination queries; the knowledge graph retrieval uses the knowledge graph to perform semantic analysis on the text content, and uses a bottom-up construction method, starting from specific data and examples, gradually inducing and abstracting high-level concepts and relationships. On the basis of the first two layers of retrieval, by constructing a relationship network, the system's understanding and reasoning ability of domain knowledge are enhanced.

[0038] Furthermore, as an improvement of the technical solution of the present invention, a basic large model is used as the underlying support of the intelligent question-and-answer system. By deeply learning engineering consulting cases and road engineering normative documents, the subtle differences in the text are captured; information is retrieved from the data source to assist the large language model in generating answers. RAG combines search technology and the prompt function of the large language model, and integrates the query and the retrieved context information into the prompt sent to the large language model.

[0039] Furthermore, as an improvement of the technical solution of the present invention, in the data collection stage, web crawler technology is used to automatically collect road engineering-related data publicly available on the Internet, and the reliability of the collected data sources is evaluated and classified for storage.

[0040] Furthermore, as an improvement of the technical solution of the present invention, in the initialization stage, the ship's mass, moment of inertia, and its own detailed parameters of the hydrodynamic coefficients under different loading conditions are input.

[0041] As a further improvement of the technical solution of the present invention, the test data set is divided into multiple subsets, and one of the subsets is taken as the test set in turn, and the remaining subsets are taken as the training set.

[0042] As a further improvement of the technical solution of the present invention, for different types of road engineering projects such as highways, urban roads, and rural roads, a dedicated sub-knowledge base is established, and during the retrieval and Q&A process, the corresponding sub-knowledge base is automatically matched according to the project type.

[0043] It should be noted that:

[0044] (I) Main steps of the construction method

[0045] Data preprocessing: Widely collect data such as road engineering-related specifications, standards, case studies, relevant policies, and on-site construction consultation logs. Use specialized algorithms and tools to clean the data, remove invalid, incorrect, and incomplete information such as format errors, garbled characters, input errors, and incomplete fields; use mature technologies to encrypt, replace, or obfuscate and desensitize sensitive data such as personal privacy and business secrets; according to the set frequency threshold, remove the low-frequency part by counting the number of occurrences of words or phrases; use natural language processing text structuring technology to convert unstructured data such as documents and reports into structured formats such as tables and JSON; use artificial intelligence models such as sequence-to-sequence models based on deep learning to extract questions and answers from the structured data to form preliminary Q&A pairs.

[0046] Data supplementation: Road engineering professionals rely on their professional knowledge and experience to supplement the missing explicit context of the Q&A pairs, such as background information such as construction scenarios and engineering stages for road construction technology problems; further supplement the implicit context, that is, the professional knowledge and industry practices behind the questions and answers, and organize them into Markdown format, which is suitable for use by different systems and tools because of its simplicity, readability, and ease of conversion; use data slicing technology to split the data, convert it into vector form using vector embedding algorithms, and at the same time establish a full-text index, optimize it using retrieval re-rank algorithms, and integrate it into the vector and full-text knowledge bases to form a vector + full-text double-layer retrieval structure.

[0047] Retrieval optimization: Based on the road engineering knowledge system, use knowledge graph construction technology to define entity types, relationship types, and attributes, design the knowledge graph framework, and organize relevant concepts, terms, and specifications into a structured semantic network; through on-site investigations, exchanges with construction personnel, collection of construction records, etc., continuously collect actual problems on the construction site to provide real-scene data for the improvement of the knowledge graph and system optimization; after reviewing, supplementing, and optimizing the collected and sorted knowledge, import it into the knowledge graph database, store and manage it using a graph database management system, and combine the actual engineering problems with the specifications and standards to enrich the content of the knowledge graph.

[0048] Model Testing: Integrate vector retrieval, full-text retrieval, and knowledge graph retrieval to construct a three-layer retrieval mechanism. Vector retrieval is based on the vector space model and uses the greedy algorithm to calculate the similarity of word vectors to quickly locate text segments with similar semantics; full-text retrieval builds an inverted index to support complex keyword combination queries and expand the retrieval scope; knowledge graph retrieval uses semantic relation networks for inference queries. Use the test dataset for single-round Q&A testing to evaluate indicators such as answer accuracy, relevance, and integrity; conduct multi-round continuous dialogue testing to examine the system's ability to handle continuous questions, understand context, and maintain dialogue coherence; according to the test results, use model optimization algorithms to adjust model parameters, optimize the network structure, and increase training data to improve system performance and user experience.

[0049] (2) BERT-based Text Filter

[0050] To screen and filter questions, limit the processing scope of the large model, and reduce misleading information, a BERT-based text filter is introduced. Use the powerful natural language understanding ability of the BERT model to process the training data, extract features and understand semantics, send the output to the fully connected layer, and train based on the existing labels in the dataset, only adjusting the parameters of the fully connected layer to quickly adapt to the text classification task. When BERT is used for text classification, a classifier is constructed using the specific word vector H output by it combined with the softmax function, and a fully connected layer is introduced to calculate the label probability. Since the task is a binary classification problem of determining whether a question belongs to the field of road engineering, the output layer dimension is set to 2, and the label with the highest probability is selected as the classification result.

[0051] (3) Road Industry Specification Processing

[0052] When processing road industry specifications, the document content is divided into explicit and implicit contexts. Explicit contexts are mainly at the head and tail of the document, which are the contexts shared by content slices, mostly industry standard names such as "JTG / T 3512-2020" and "JTG5110-2023", etc., which have unique retrievability and are convenient for identification and retrieval. Implicit contexts are the key professional terms and chapters within the specifications, corresponding content categories, and manually interpreted tabular data, which require professional knowledge and in-depth analysis and understanding. Extract Q&A pairs through manual identification combined with AI technology, accurately grasp the context meaning with the experience and knowledge of professionals, and use AI technology to improve the extraction efficiency and accuracy. Correct and rewrite the slice content to be put into the knowledge base to make it meet the storage and retrieval requirements. Compile the document in Markdown format to clearly display the titles, sub-titles, chapters to which each specification item belongs, and the reply answers, which is convenient for understanding and vectorization processing, and is more easily converted into a vector representation that can be understood by the computer during vectorization.

[0053] (4) Three-layer Retrieval Mechanism

[0054] Vector retrieval converts text data into points in a vector space, calculates vector distances or similarities to achieve semantic-level matching, and uses a greedy algorithm to calculate the one-way greedy matching score between the keyword retrieval sequence and the question sequence of the database Q&A pairs. Finally, the average of the scores in two directions is taken as the GM score to quickly locate semantically similar documents or paragraphs. Full-text retrieval deeply analyzes the text content through technologies such as building an inverted index, supports complex keyword combination queries, expands the retrieval scope based on vector retrieval, and ensures that text fragments containing the answer are not missed. Knowledge graph retrieval uses the knowledge graph for text semantic analysis, adopts a bottom-up construction method, abstracts high-level concepts and relationships from specific data and examples, screens highly credible knowledge to build a semantic relationship network, and on the basis of the first two layers of retrieval, enhances the system's understanding and reasoning ability of domain knowledge by constructing a relationship network, providing more accurate, logical, and structured answers.

[0055] In summary, the present invention has the following beneficial effects:

[0056] Efficient retrieval and fast response: The intelligent Q&A system for road engineering specifications based on a large model can quickly locate relevant texts through semantic similarity calculation, reducing the time and effort of manual collation and browsing, reducing the error tolerance rate, and achieving fast response to queries.

[0057] Improve answer accuracy: Combining vector retrieval and knowledge graphs helps the model better understand professional terms and logical structures in the field of road engineering, reduces the output of misleading hallucination results of traditional large models, and improves answer accuracy.

[0058] Data quality and retrieval efficiency guarantee: Steps such as cleaning, desensitization, and low-frequency filtering in the data preprocessing stage improve data quality and ensure accurate relevance of model input. The RAG technology optimizes the knowledge retrieval process, reduces the amount of retrieved data, reduces the search space, significantly improves the retrieval efficiency, and enhances the overall response speed of the system.

[0059] Meet the needs of industry development: The constructed special knowledge base integrating the intelligent Q&A system of road engineering and large language models integrates knowledge resources in the field of road engineering, meeting the needs of the development of smart highways for accurately and conveniently obtaining road engineering industry specifications. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes, and advantages of the present invention will become more apparent:

[0061] Figure 1 It is a flowchart of a method for building an intelligent Q&A system for engineering specifications for pavement health management according to an embodiment of the present invention;

[0062] Figure 2 It is a schematic diagram of the result of corpus processing according to an embodiment of the present invention;

[0063] Figure 3 It is a schematic diagram of the processing of Q&A corpus made according to Table 4.0.2;

[0064] Figure 4 It is a flowchart of the implementation of a method for building an intelligent Q&A system for engineering specifications for pavement health management according to an embodiment of the present invention;

[0065] Figure 5 It is a flowchart of building a knowledge graph from bottom to top according to an embodiment of the present invention;

[0066] Figure 6 It is a framework diagram of a method for building an intelligent Q&A system for engineering specifications for pavement health management according to an embodiment of the present invention. Detailed implementation manners

[0067] To make the objectives, features, and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application are described clearly and completely. Obviously, the embodiments described below are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0068] Refer to Figure 6 , a method for building an intelligent Q&A system for engineering specifications for pavement health management, includes the following steps:

[0069] Step S1, data preprocessing:

[0070] Data collection: Collect relevant specifications, standards, case studies, relevant policies, and on-site construction consultation logs of road engineering;

[0071] Data cleaning: Clean the data to remove invalid, incorrect, and incomplete information;

[0072] Data desensitization: Process sensitive data and comply with data protection regulations;

[0073] Low-frequency filtering: Remove low-frequency words or phrases;

[0074] Structured conversion: Convert unstructured data into a structured format;

[0075] AI extraction to form Q&A pairs: Use an artificial intelligence model to extract questions and answers from structured data to form preliminary Q&A pairs;

[0076] Step S2, data supplementation:

[0077] Manually supplement the explicit context of Q&A pairs: Professionals supplement the missing context information of Q&A pairs;

[0078] Implicit context for manually supplemented Q&A pairs: Supplement the implicit context to form Markdown format;

[0079] Slicing into vectors + full-text knowledge base: Slice and integrate the data into vectors and the full-text knowledge base, perform retrieval rearrangement optimization to form a vector + full-text dual-layer retrieval structure;

[0080] Step S3, Retrieval optimization:

[0081] Design the knowledge graph structure: Construct the knowledge graph framework;

[0082] Collect actual problems at the construction site: Continuously collect actual problems at the construction site;

[0083] Improve and import the knowledge graph database: Improve the collected knowledge, import it into the knowledge graph database, and combine engineering actual problems with specification standards;

[0084] Step S4, Model testing:

[0085] Form a three-layer retrieval of vector + full-text + knowledge graph: Combine vectors, full-text, and knowledge graph to construct a three-layer retrieval mechanism;

[0086] Single-round testing: Conduct preliminary model testing to evaluate the system performance;

[0087] Multi-round testing: Conduct multi-round testing to examine the system's continuous dialogue performance;

[0088] Iterative optimization: Iteratively optimize the model according to the test results.

[0089] Specifically, the establishment of this system can be divided into four main stages: data preprocessing, data supplementation, retrieval optimization, and model testing. The detailed steps are as Figure 1 shown.

[0090] In the construction of the intelligent Q&A system for road engineering specifications based on large models, in the first stage, data preprocessing is carried out, and the specific steps are described as follows: (1) Data collection: In this process, data related to road engineering is collected, including specifications, standards, case studies, relevant policies, and on-site construction consultation logs, etc. (2) Data cleaning: Clean the data to remove invalid, incorrect, or incomplete information. (3) Data desensitization: Process sensitive data to protect privacy and comply with data protection regulations. (4) Low-frequency filtering: Remove low-frequency words or phrases to improve the relevance and quality of the data. (5) Structured conversion: Convert unstructured data into a structured format for convenient overall processing. (6) AI extraction to form Q&A pairs: Use existing artificial intelligence models to extract questions and answers from structured data to form preliminary Q&A pairs.

[0091] The second stage mainly focuses on data supplementation. In this stage, the Q&A pairs are expanded to cover as many possible question forms corresponding to the results as possible. This stage is designed in three aspects: (1) Manually supplement the explicit context of the Q&A pairs: Professional personnel supplement the missing context information in the Q&A pairs to ensure the integrity of the Q&A. (2) Manually supplement the implicit context of the Q&A pairs: Further supplement the implicit context and format it into Markdown for subsequent processing. (3) Slice and integrate into vector + full-text knowledge base: Slice the data and integrate it into the vector and full-text knowledge bases, and at the same time perform retrieval rearrangement optimization to form a two-layer retrieval structure of vector + full-text.

[0092] In this stage

[0093] The third stage mainly focuses on retrieval optimization. In this stage, the knowledge graph structure is designed to build a knowledge graph framework for better organizing and presenting knowledge. Continuously collect the actual problems on the construction site. Starting from the actual problems can better meet the application requirements. Finally, improve and import the collected and sorted knowledge into the knowledge graph database, combining the problems actually encountered in the engineering process with the specification standards to better feedback on the problems.

[0094] The fourth stage mainly focuses on model testing. The following aspects will be carried out in this stage: (1) Form a three-layer retrieval of vector + full-text + knowledge graph: Combine vectors, full-text, and knowledge graphs to build a three-layer retrieval mechanism to improve the accuracy and efficiency of retrieval. (2) Single-round testing: Conduct preliminary model testing to evaluate the performance of the system. (3) Multi-round testing: Conduct more in-depth multi-round testing to examine the performance of the system in continuous conversations. (4) Iterative optimization: Perform iterative optimization of the model based on the test results to continuously improve the performance of the system.

[0095] 1.2 Information filtering

[0096] This system introduces a BERT-based text filter to screen and filter questions, thereby limiting the scope of questions processed by the large model and reducing the risk of generating misleading information. Suppose the set of all questions that can be input into the large model is Q, the set of questions that the large model can answer in the field of road engineering is R, and the set of questions that can generate professional answers is D. Obviously, Q > R > D. To make the system professional and targeted, after using the filter, Q should approach R as much as possible to ensure that the questions asked are within the scope of R. In the process of developing this text filter, we first use the BERT model to process the training data, and then the output generated by BERT will be fed into a fully connected layer to obtain a text classification token [CLS]. This token is trained based on the existing labels in the dataset, and during the training process, we only adjust the parameters of the fully connected layer. Usually, in the scenario where BERT is used for text classification, we will use a specific word vector H output by BERT and combine it with the softmax function to construct a simple classifier, where the softmax function is as follows:

[0097]

[0098] where: x is a vector, and x i is one of its elements. This classifier can predict the probabilities of different class labels L. In our method, we made some adjustments to this process by introducing a fully connected layer to calculate the probability of each label. In the training stage, the input vector dimension received by the fully connected layer is 768, it contains two hidden layers, the dimension of the first hidden layer is 384, the second is 768, and the dimension of the output layer depends on the number of classes. Since our specific task is a binary classification problem, the dimension of the output layer is 2. Finally, we will select the label with the highest probability as the classification result of [CLS]. In the field of agricultural science, the [CLS] token is used to determine whether the question is related to agriculture. In this way, by filtering out irrelevant questions, we can reduce the risk of generating misleading information and combine the retrieval results to decide whether to provide a professional answer.

[0099] 1.3 Road Engineering Industry Specification Segmentation

[0100] Road engineering is an important part of infrastructure construction. It not only relates to the development of the national economy but also directly affects people's daily lives. Road engineering industry specifications are the key to ensuring the quality, safety, and durability of road construction. Normative documents generally include the name of the document, hierarchical division, and each main content element.

[0101] Refer to Figure 2, when dealing with road industry specifications, the document content is divided into explicit context and implicit context. The explicit context is mainly contained in the head and tail of the document, which is the context shared by all content slices of the entire article and is relatively easy to be discovered and understood by the model. It is mainly the names of industry standards, such as: "JTG / T3512-2020", "JTG 5110-2023". Such standard name codes are uniquely retrievable and can be quickly identified and consulted.

[0102] The implicit context mainly includes the key professional terms, chapters described in the specification, the corresponding content categories and the tabular data with corresponding manual interpretations. By manually identifying the explicit context and implicit context and combining AI technology, question-answer pairs are extracted to correct and rewrite the slice content to be put into the knowledge base. This process can improve the accuracy of the content and enhance its retrievability.

[0103] To better organize and present the content of regulatory documents and facilitate subsequent vectorized storage and information processing, the Markdown format is used for document compilation. The Markdown format file can clearly display the titles, sub-titles, chapters to which they belong, and reply answers of each item of the specification, making the structure of the regulatory document clearer and facilitating understanding and vectorized processing.

[0104] For the specified data in the table in the specification, for example, in "JTG B01-2014 Technical Standard for Highway Engineering", in Section 4.0.2, the lane width shall conform to the provisions of Table 4.0.2:

[0105] Table 4.0.2 Lane Width

[0106]

[0107] Table 4.0.2 provides the lane width standards under different design speeds. To ensure the accuracy and easy understanding of the information, we need to list the horizontal and vertical header data in the table in detail and accurately split the corresponding content. Figure 3 It is the question-answer pair corpus made according to Table 4.0.2.

[0108] 1.4 Algorithm Retrieval Process

[0109] To further improve the performance of the intelligent question-answer system, we have constructed a three-layer retrieval mechanism including vector retrieval, full-text retrieval and knowledge graph retrieval. This mechanism can combine the full-text content, vectorized keywords and semantic relationships in the knowledge graph to provide more accurate and efficient retrieval services. In this way, the intelligent question-answer system can quickly retrieve relevant information from a large number of regulatory documents and provide timely and accurate answers for users. The main implementation process of the system is as Figure 4 .

[0110] In this system, the vector retrieval technology realizes the similarity calculation at the semantic level by converting text data into points in a vector space. This method enables the system to quickly locate documents or paragraphs semantically similar to the user's query in a high-dimensional space, mapping semantically similar texts to adjacent positions in the vector space. When the user enters a query, the system converts it into a vector and searches for the most similar vector in the vector database, thereby quickly locating the text fragments that may contain the answer. This mechanism can effectively solve the problem of inaccurate keyword matching and improve the retrieval accuracy. The greedy algorithm is used to perform similarity matching on keyword features. First, calculate the unidirectional greedy matching scores of two sequences. Taking the vector representation corresponding to the keyword retrieval sequence x=(x1,...,x n ) as The question sequence of the database Q&A pair The corresponding vector representation is The greedy matching score from the keyword retrieval sequence to the question sequence For example:

[0111]

[0112] where cos_sim is the similarity calculation of two word vectors, x i and are the i-th and j-th words in the retrieval sequence and the question sequence respectively, and are the vectors corresponding to the words in the sequences respectively. The unidirectional greedy matching score of the question sequence to the retrieval sequence x is calculated in a similar way as the above formula. The final GM score is the average of the greedy matching scores G in both directions:

[0113]

[0114] Full-text retrieval, through in-depth analysis of the text content, ensures that the system can comprehensively scan and analyze regulatory documents without missing any possible relevant information. It supports users to perform complex keyword combination queries. Based on vector retrieval, full-text retrieval can further expand the retrieval scope, ensuring that the system does not miss text fragments that may contain the answer and improving the comprehensiveness of the answer.

[0115] Finally, semantic analysis is performed on the text content using a knowledge graph. Adopting a bottom-up construction method, starting from specific data and examples, high-level concepts and relationships are gradually induced and abstracted. It focuses on screening out highly credible knowledge from open linked data, extracting information from unstructured text, and then constructing a knowledge graph to extract entities, relationships, and attributes, and build a semantic relationship network. Based on the first two levels of retrieval, knowledge graph retrieval enhances the system's understanding and reasoning ability of domain knowledge by constructing a relationship network, providing more accurate, logical, and structured answers.

[0116] This system uses a basic large model as the underlying support for the intelligent Q&A system. The application of the basic large model provides the system with powerful natural language understanding capabilities. Through in-depth learning of engineering consulting cases and road engineering regulatory documents, the basic large model can capture subtle differences in the text, providing high-quality input for subsequent retrieval and Q&A. Information is retrieved from data sources to assist the large language model in generating answers. RAG combines search technology and the prompt function of the large language model, that is, questions are posed to the model, and the information found by the search algorithm is used as background context. These queries and retrieved context information will be integrated into the prompts sent to the large language model. Such an approach can greatly improve the system accuracy for industry models and increase the substitutability of industry models.

[0117] Through this series of technology integrations and process optimizations, the application of the intelligent Q&A system in the field of road engineering will be more extensive and in-depth. It can not only quickly respond to users' queries but also provide in-depth analysis and suggestions, providing strong knowledge support for the planning, design, and construction of road engineering.

[0118] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0119] Efficient retrieval and quick response: The intelligent Q&A system for road engineering specifications based on a large model can quickly locate relevant texts through semantic similarity calculation, reducing the time and effort of manual collation and browsing, reducing the error tolerance rate, and achieving quick response to queries.

[0120] Improve answer accuracy: Combining vector retrieval and knowledge graph helps the model better understand professional terms and logical structures in the field of road engineering, reducing the output of misleading hallucination results by traditional large models and improving answer accuracy.

[0121] Guarantee data quality and retrieval efficiency: Steps such as cleaning, desensitization, and low-frequency filtering in the data preprocessing stage improve data quality and ensure accurate and relevant model input. RAG technology optimizes the knowledge retrieval process, reduces the amount of retrieved data, reduces the search space, significantly improves the retrieval efficiency, and enhances the overall response speed of the system.

[0122] Meet the needs of industry development: The constructed special knowledge base integrating the road engineering and the intelligent Q&A system of large language models integrates the knowledge resources in the field of road engineering to meet the needs of the development of smart highways for accurately and conveniently obtaining road engineering industry specifications.

[0123] The technical solutions provided by the embodiments of the present invention have been introduced in detail above. Specific examples are used in this article to elaborate on the principles and implementation manners of the embodiments of the present invention. The descriptions of the above embodiments are only applicable to helping understand the principles of the embodiments of the present invention; at the same time, for those of ordinary skill in the art, based on the embodiments of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for building an intelligent Q&A system for engineering specifications for pavement health management, characterized in that, It includes the following steps: Step S1, data preprocessing: Data collection: Collect relevant specifications, standards, case studies, relevant policies, and on-site construction consultation logs for road engineering; Data cleaning: Clean the data to remove invalid, incorrect, and incomplete information; Data desensitization: Process sensitive data and comply with data protection regulations; Low-frequency filtering: Remove low-frequency words or phrases; Structured conversion: Convert unstructured data into a structured format; AI extraction to form question-and-answer pairs: Use an artificial intelligence model to extract questions and answers from structured data to form preliminary question-and-answer pairs; Step S2, data supplementation: Manually supplement the explicit context of question-and-answer pairs: Professionals supplement the missing context information of question-and-answer pairs; Manually supplement the implicit context of question-and-answer pairs: Supplement the implicit context to form a Markdown format; Slice and enter the vector + full-text knowledge base: Slice the data and integrate it into the vector and full-text knowledge bases, perform retrieval rearrangement optimization, and form a vector + full-text double-layer retrieval structure; Step S3, retrieval optimization: Design the knowledge graph structure: Build the knowledge graph framework; Collect actual problems at the construction site: Continuously collect actual problems at the construction site; Improve and import into the knowledge graph database: Improve the collected and sorted knowledge, import it into the knowledge graph database, and combine the actual engineering problems with the specifications and standards; Step S4, model testing: Form a three-layer retrieval of vector + full-text + knowledge graph: Combine vectors, full-text, and knowledge graphs to build a three-layer retrieval mechanism; Single-round testing: Conduct preliminary model testing to evaluate the system performance; Multi-round testing: Conduct multi-round testing to examine the continuous dialogue performance of the system; Iterative optimization: Iteratively optimize the model according to the test results.

2. The method for building an intelligent Q&A system for engineering specifications for road surface health management according to claim 1, wherein Introduce a BERT-based text filter to screen and filter questions; the development process of the text filter is as follows: Use the BERT model to process the training data, and the output generated by BERT is sent to the fully connected layer to obtain the text classification token [CLS], and train based on the existing labels in the dataset. Only the parameters of the fully connected layer are adjusted during training.

3. The method for building an intelligent Q&A system for engineering specifications for pavement health management according to claim 1, characterized in that: When BERT is used for text classification, a classifier is constructed using the specific word vector H output by BERT in combination with the softmax function. The softmax function is: where: x is a vector, and x i is an element therein; the classifier predicts the probabilities of different class labels L; in this method, a fully connected layer is introduced to calculate the probability of each label. The input vector dimension of the fully connected layer is 768, including two hidden layers. The dimension of the first hidden layer is 384, the second dimension is 768, and the dimension of the output layer depends on the number of classes. Since the task is a binary classification problem, the dimension of the output layer is 2, and the label with the highest probability is selected as the classification result of [CLS].

4. A method for building an intelligent Q&A system for engineering specifications for pavement health management according to claim 1, characterized in that: When processing road industry specifications, the document content is divided into explicit context and implicit context; the explicit context is included in the head and tail of the document, which is the context shared by content slices, mainly the industry standard names; the implicit context is the key professional terms and chapters within the specification, corresponding content categories, and manually interpreted tabular data; Question-and-answer pairs are extracted through manual identification combined with AI technology, and the sliced content to be entered into the knowledge base is corrected and rewritten.

5. A method for building an intelligent Q&A system for engineering specifications for pavement health management according to claim 1, characterized in that: Build a three - layer retrieval mechanism that includes vector retrieval, full - text retrieval, and knowledge graph retrieval; the vector retrieval technology converts text data into points in a vector space, uses the greedy algorithm to perform similarity matching on keyword features, calculates the unidirectional greedy matching scores of two sequences, and the final GM score is the average of the greedy matching scores in both directions; the full - text retrieval deeply analyzes the text content and supports complex keyword combination queries; the knowledge graph retrieval uses the knowledge graph to perform semantic analysis on the text content, adopts a bottom - up construction method, starts from specific data and examples, and gradually induces and abstracts high - level concepts and relationships. On the basis of the first two layers of retrieval, by constructing a relationship network, it enhances the system's understanding and reasoning ability of domain knowledge.

6. A method for building an intelligent Q&A system for engineering specifications for road surface health management according to claim 1, characterized in that: Adopt a basic large - model as the underlying support of the intelligent Q&A system, and capture the subtle differences in the text through deep learning of engineering consulting cases and road engineering regulatory documents. Assist the large - language model in generating answers by retrieving information from data sources. RAG combines search technology and the prompt function of the large - language model, and integrates the query and the retrieved context information into the prompt sent to the large - language model.

7. A method for building an intelligent Q&A system for engineering specifications for pavement health management according to claim 1, characterized in that: In the data collection stage, use web crawler technology to automatically collect road - engineering - related data publicly available on the Internet, and conduct reliability assessment and classified storage of the collected data sources.

8. A method for building an intelligent Q&A system for engineering specifications for pavement health management according to claim 1, characterized in that: In the initialization stage, input detailed parameters such as ship mass, moment of inertia, and hydrodynamic coefficients under different loading conditions.

9. The method for building an intelligent Q&A system for engineering specifications for pavement health management according to claim 1, wherein: Divide the test data set into multiple subsets, and take turns using one subset as the test set and the remaining subsets as the training set.

10. A method for building an intelligent Q&A system for engineering specifications for pavement health management according to claim 1, characterized in that: For different types of road engineering projects such as highways, urban roads, and rural roads, establish specialized sub - knowledge bases, and automatically match the corresponding sub - knowledge bases according to the project type during the retrieval and Q&A process.

Images