Multi-text-multi-table question and answer thinking chain and thinking program combined reasoning method

Abstract

The invention belongs to the technical field of intelligent model processing, and particularly relates to a multi-text-multi-table question and answer thinking chain and thinking program combined reasoning method which comprises the following steps: S1, constructing a multi-text-multi-table data set; s2, encoding texts and tables in the multi-text-multi-table data set; s3, decomposing the complex problem to be processed into a plurality of sub-problems which are sequentially combined by adopting a thinking chain CoT method; s4, solving each sub-question by using a large language model F to obtain a preliminary answer of each sub-question; s5, inputting the sub-questions and the corresponding contexts into the GPT-4 model for re-solving, and recording a re-solving result as a preset reference answer; comparing whether the preset reference answers of the sub-questions are consistent with the preliminary answers or not; checking and tuning of the large language model F are completed; and S6, generating a final answer of the complex question by checking the optimized large language model F. According to the method, a bridge detection result and analysis key information can be accurately and efficiently acquired, and powerful technical support is provided for the field of bridge management and maintenance.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent model processing, and in particular relates to a thinking chain and thinking program joint reasoning method for multi-text-multi-table question and answer. Background Art

[0002] In the field of bridge management and maintenance, the inspection and health monitoring data of bridge structures are huge in scale, and these data are widely stored in the industry information system in the form of electronic text, pictures and table information. However, key documents such as regular bridge inspections and regular inspection reports mainly exist in the form of a mixed context of long text and multiple tables. This data format is not only complex and diverse, but also contains a large amount of unstructured text information and structured table data, making it difficult for operation and maintenance personnel to accurately and efficiently obtain the inspection results and analyze key information of the bridge.

[0003] In recent years, with the rapid development of natural language processing technology, related algorithms and models have been gradually applied to multiple fields and have shown great potential. Among them, large language models (LLMs) have been widely used in vertical industries such as recommendation systems and intelligent customer service due to their powerful semantic understanding and text generation capabilities. However, when faced with the long text-multiple table mixed context in the field of bridge maintenance, existing large language models and related technical methods still face many challenges.

[0004] First, insufficient knowledge fusion is one of the main problems of current technical methods. Due to the significant differences between unstructured text and structured tables in data format, expression and information density, it is difficult for existing technical methods to effectively fuse these two types of data, thus limiting the model's comprehensive understanding and analysis of bridge inspection results.

[0005] Secondly, the difficulty of complex reasoning is another major challenge faced by current technical methods. In bridge inspection reports, it is often necessary to make joint inductive judgments on multiple cell information in a table, or to perform multi-information reasoning across texts and tables. However, existing large language models often lack accurate reasoning rules when dealing with such complex reasoning tasks, resulting in inaccurate or difficult to obtain reasoning results.

[0006] In addition, the lack of numerical computing capabilities is also a key problem that current technical methods need to solve. The regular bridge inspection report contains a large number of calculation tasks such as technical condition scores, defect areas, and defect ratios. However, the existing large language models often perform poorly when processing these complex numerical calculations and are difficult to meet the requirements of practical applications.

[0007] The existence of these problems not only limits the application effect of existing technical methods in the field of bridge management and maintenance, but also increases the difficulty for operation and maintenance personnel to obtain bridge inspection results and analyze key information. The root of the problem lies in the limitations of existing technical methods in processing long text-multiple table mixed contexts, as well as the shortcomings of large language models in knowledge fusion, complex reasoning and numerical calculation.

[0008] Therefore, how to use innovative technical methods, such as the thinking chain of multi-text-multi-table question and answer and the joint reasoning method of thinking program, to effectively solve the challenges faced by existing technical methods in dealing with the mixed context of long text and multiple tables in bridge inspection reports, and achieve accurate and efficient acquisition of key information of bridge inspection results and analysis has become a problem that needs to be solved urgently. Summary of the invention

[0009] In view of the above-mentioned deficiencies in the prior art, the present invention provides a multi-text-multi-table question-and-answer thinking chain and thinking program joint reasoning method, which can accurately and efficiently obtain bridge inspection results and analyze key information, providing strong technical support for the field of bridge management and maintenance.

[0010] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0011] The thinking chain and thinking program combined reasoning method for multi-text-multi-table question answering includes the following steps:

[0012] S1. Extract text and tables from bridge periodic inspection reports to construct a multi-text-multi-table dataset for the bridge field;

[0013] S2, using a preset pre-trained language model, respectively encode the text and table in the multi-text-multi-table dataset to obtain corresponding text encoding and table encoding;

[0014] S3. Use the CoT method to decompose the complex problem to be processed into multiple sub-problems in a sequential combination, and obtain the context information related to each sub-problem; then classify each sub-problem into retrieval sub-problems and calculation sub-problems;

[0015] S4, combining the text encoding and table encoding obtained in S2, using the large language model F to solve each sub-problem and obtain preliminary answers to each sub-problem; among them, for the computational sub-problems, the thinking program PoT technology is used to guide the large language model F to solve;

[0016] S5. Obtain the preset reference answer for each sub-question; compare the preliminary answer to each sub-question with the preset reference answer to see if they are consistent; for inconsistent sub-questions, return to S4 to generate a new preliminary answer and compare it with the corresponding preset reference answer again until the preset reference answer is consistent with the preliminary answer, thus completing the verification and tuning of the large language model F;

[0017] S6. Taking each sub-question, the answer to each sub-question and the corresponding prompt as context, the final answer to the complex question to be processed is comprehensively generated by verifying and tuning the large language model F.

[0018] Glossary:

[0019] Chain-of-Thought (CoT) is a method that can significantly improve the logical reasoning ability of a large language model by guiding it to answer questions step by step.

[0020] Thinking program PoT technology, that is, program-based thinking chain (Program of thoughts, PoT); this technology further utilizes context learning to guide the large language model to generate program code to solve complex numerical reasoning tasks, thereby improving the accuracy of the answer.

[0021] Prompts are used to clearly indicate the tasks that the large language model needs to perform and provide relevant contextual information to guide the large language model to generate specific, accurate, and coherent outputs or responses.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. By extracting text and tables from regular bridge inspection reports, a multi-text-multi-table dataset for the bridge field is constructed. This helps the model better understand the professional terms and context in the bridge field and improve the accuracy of question answering. Existing technologies often rely on general datasets and lack specialized datasets for the bridge field. This method achieves effective integration and utilization of knowledge in the bridge field by constructing a specialized dataset.

[0024] 2. Use the preset pre-trained language model to encode the text and table, and obtain the corresponding text encoding and table encoding. This helps the model understand and use the information in the text and table at the same time when dealing with complex problems. Existing technologies often process text and tables separately and lack an effective fusion mechanism. This method achieves seamless connection between text and table information through encoding fusion, and improves the information processing ability of the model.

[0025] 3. Use the thinking chain CoT method to decompose complex problems into multiple sub-problems and classify them into retrieval-type and computational sub-problems. This helps the model to gradually solve complex problems and reduce the difficulty of solving them. At the same time, combined with the large language model F and the thinking program PoT technology, accurate solutions to sub-problems are achieved. Existing technologies often lack effective decomposition and solution strategies when dealing with complex problems. This method uses the thinking chain CoT method and classification strategy to achieve effective decomposition and accurate solution of complex problems, thereby improving the solution efficiency of the model.

[0026] 4. Obtain the preset reference answers for each sub-question and compare them with the preliminary answers to achieve verification and tuning of the large language model F. Finally, the final answers to complex questions are generated by combining the sub-questions, answers, and prompts as context. This helps to ensure the accuracy and completeness of the answers. Existing technologies often lack a verification and tuning mechanism for the model, making it difficult to ensure the accuracy of the answers. This method achieves strict control of the answer quality through the verification and tuning of the GPT-4 model and the final answer generation strategy, thereby improving the reliability and practicality of the model.

[0027] In summary, this method effectively solves the challenges faced by existing technical methods in processing the long text-multiple table mixed context in bridge inspection reports by building a special data set for the bridge field, realizing the coding fusion of text and tables, effectively decomposing and solving complex problems, and verifying and tuning the model and generating the final answer. It achieves accurate and efficient acquisition of bridge inspection results and key information for analysis, providing strong technical support for the field of bridge management and maintenance.

[0028] Preferably, in S4, when solving each sub-problem, if the reasoning of a sub-problem requires external knowledge, the large language model F generates a query according to the sub-problem and obtains knowledge related to the sub-problem from the knowledge base.

[0029] With this setup, 1. When the reasoning of a sub-question needs to rely on external knowledge, the large language model F can actively generate query statements and retrieve relevant information from the knowledge base. This capability significantly enhances the model's reasoning ability when dealing with complex, expertise-intensive problems. By obtaining accurate and authoritative knowledge from the knowledge base, the large language model F can more accurately answer sub-questions that require expertise support. This helps to improve the accuracy and credibility of the answers of the overall question-answering system.

[0030] 2. The introduction of the knowledge base enables the large language model F to handle a wider range of problems, especially those involving expertise in the bridge field. This expands the model's knowledge coverage, making it more applicable to specific fields such as bridge maintenance.

[0031] 3. By combining text, table encoding, and knowledge obtained from the knowledge base, the large language model F can more comprehensively understand and utilize various types of information. This helps promote the integration and efficient use of knowledge and improves the overall performance of the question-answering system.

[0032] Preferably, S1 includes:

[0033] Obtaining regular bridge inspection reports and extracting important information from the reports, including bridge overview, bridge disease information inspection result table, bridge technical condition assessment table, and assessment specifications;

[0034] Integrate each important information into different paragraphs P or tables T according to its format and content;

[0035] The integrated associated paragraphs P and / or tables T are integrated into a hybrid context Context to complete the construction of a multi-text-multi-table dataset for the bridge field; wherein, in one context Context, the sum of the number of tables T and paragraphs P is not less than 2.

[0036] With this setup, 1. By obtaining regular bridge inspection reports and extracting important information (such as bridge overview, disease information inspection result table, technical condition assessment table, assessment specifications, etc.), the technology achieves in-depth analysis and structured information extraction of bridge inspection reports. This helps with subsequent data processing and analysis. Each important information is integrated into different paragraphs P or tables T, and classified according to their format and content, making the data clearer and more organized. At the same time, this integration method also achieves standardized data processing, which facilitates subsequent data analysis and model training.

[0037] 2. The integrated related paragraphs P and / or tables T are integrated into a hybrid context Context. This design fully considers the diversity and relevance of information in the bridge inspection report. By constructing a hybrid context, the complexity and diversity of actual bridge inspection reports can be better simulated, and the generalization ability of the model in practical applications can be improved. In a context Context, the sum of the number of tables T and paragraphs P is not less than 2, which ensures the richness and diversity of the data set. This design helps the model better capture the relevance and regularity of information when processing complex and changeable data.

[0038] Preferably, in S1, each important information is integrated into different paragraphs P or tables T, and the precise geographical location and bridge position pile number information in the important information are desensitized to ensure data privacy.

[0039] In this setting, desensitization is to hide or replace sensitive information in the data to avoid leaking personal privacy or commercial secrets. In the bridge inspection report, the precise geographic location and bridge position pile number information may be sensitive data. Through desensitization, this data can be effectively protected from being obtained by unauthorized personnel, thereby ensuring data privacy.

[0040] Preferably, in S2, the pre-trained language model is RoBERTa.

[0041] With this setup, choosing RoBERTa as the pre-trained language model in step S2 can significantly improve the model’s ability to understand text and table information in bridge inspection reports, encoding efficiency and accuracy, as well as the model’s generalization ability and training efficiency. These effects together provide strong support for subsequent question solving and answer generation.

[0042] Preferably, in S2, the process of encoding the text in the multi-text-multi-table dataset includes:

[0043] For a paragraph P in a Context, first, the paragraph is divided into sentences to obtain multiple sentences S, P = (S1, S2, ..., S t ), t is the number of sentences;

[0044] Then, use the pre-trained language model to encode each sentence to get the representation of each sentence i is the sentence number;

[0045] Then, the representation of each sentence in the paragraph Aggregate using the average pooling method to get the representation of the paragraph j is the number of the paragraph;

[0046] Finally, the representations of all paragraphs of the Context are aggregated through the concatenation method to obtain the overall text representation H of the Context T .

[0047] Such a setting, 1. The encoding process combines the powerful encoding ability of the pre-trained language model and the multi-level information aggregation method to improve the efficiency and accuracy of text encoding. By refining the steps of text processing, sentence-level encoding, paragraph-level aggregation, and context-level integration, the model can more accurately capture and understand the key information in the text, providing strong support for subsequent question solving and answer generation.

[0048] 2. The encoding process is suitable for processing complex and changeable text data, such as the multi-text-multi-table mixed context in bridge inspection reports. Through multi-level information aggregation and integration, the model can cope with the diversity and complexity of text data and improve processing effect and accuracy.

[0049] Preferably, Jieba is used to sentence each paragraph.

[0050] Such a setting, using Jieba to segment each paragraph, has significant effects in Chinese text processing, including improving the accuracy and efficiency of sentence segmentation, supporting subsequent natural language processing tasks, providing flexibility and scalability, and being suitable for Chinese environments. These effects together provide strong support for Chinese text processing.

[0051] Preferably, in S2, the process of encoding the table in the multi-text-multi-table dataset includes:

[0052] For a table T in a Context, use the pre-trained language model to encode the data of each cell in each column of the table;

[0053] For each row, use weighted average pooling to aggregate the vector representations of the data of each cell in the row into the representation of the row;

[0054] Concatenate the representations of all rows to get the vector representation of table T

[0055] Then, all the table representations of the Context are aggregated using the splicing method to obtain the overall table representation of the Context.

[0056] With this setting, 1. By encoding each cell in each column of data in the table, fine information at the cell level can be captured. This encoding method can retain the original characteristics of the table data and provide a basis for subsequent information aggregation and table understanding. The weighted average pooling method is used to aggregate the data vector representations of each cell in each row into the representation of the row, realizing information integration from the cell level to the row level. This aggregation method not only retains the association information between cells in the row, but also simplifies the subsequent processing process.

[0057] 2. By concatenating the representations of all rows, we get the vector representation of table T, and achieve comprehensive information integration from row level to table level. This integration method helps the model capture the global information and structural features in the table, and improves the accuracy and depth of table understanding. The representations of all tables in the Context are aggregated using the concatenation method to get the overall table representation of the Context, and achieves comprehensive integration of table information from a single table to the context level. This integration method helps the model better capture and understand the associated information and contextual relationships between tables when dealing with multi-text-multi-table mixed contexts.

[0058] 3. The encoding process improves the efficiency and accuracy of table encoding through steps such as cell-level fine encoding, row-level information aggregation, table-level overall representation, and context-level table information integration, and supports the processing and understanding of complex table data. These effects together provide strong support for subsequent problem solving and answer generation.

[0059] Preferably, in S5, the preset reference answer to the sub-problem is obtained by resolving the sub-problem by the GPT-4 model, or is obtained in advance by manually annotating the preset reference answer to the sub-problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0061] Figure 1 A flowchart of the method is shown in FIG.

[0062] Figure 2 A schematic diagram of the process of solving a complex problem in an embodiment. DETAILED DESCRIPTION

[0063] The following is a further detailed description through specific implementation methods:

[0064] Example:

[0065] like Figure 1 , Figure 2 As shown, this embodiment discloses a method for combining thinking chain and thinking program in multi-text-multi-table question and answer. It should be noted that Figure 2 The LLM in the figure is the large language model F of this method, and the Check module is the functional module that uses the GPT-4 model in S5 of this method to check and tune the large language model F, corresponding to the function of S5 of this method.

[0066] The method comprises the following steps:

[0067] S1. Extract text and tables from bridge periodic inspection reports to construct a multi-text-multi-table dataset for the bridge field.

[0068] In specific implementation, S1 includes:

[0069] Obtaining regular bridge inspection reports and extracting important information from the reports, including bridge overview, bridge disease information inspection result table, bridge technical condition assessment table, and assessment specifications;

[0070] Integrate each important information into different paragraphs P or tables T according to its format and content;

[0071] The integrated associated paragraphs P and / or tables T are integrated into a hybrid context Context to complete the construction of a multi-text-multi-table dataset for the bridge field; wherein, in one context Context, the sum of the number of tables T and paragraphs P is not less than 2.

[0072] In this way, by obtaining the regular bridge inspection report and extracting important information (such as bridge overview, disease information inspection result table, technical condition assessment table, assessment specifications, etc.), the technology realizes the deep analysis and structured information extraction of the bridge inspection report. This is helpful for subsequent data processing and analysis. The important information is integrated into different paragraphs P or tables T respectively, and classified according to their format and content, making the data clearer and more organized. At the same time, this integration method also realizes the standardized processing of data, which facilitates subsequent data analysis and model training. The integrated associated paragraphs P and / or tables T are integrated into a mixed context Context. This design fully considers the diversity and relevance of information in the bridge inspection report. By constructing a mixed context, the complexity and diversity of the actual bridge inspection report can be better simulated, and the generalization ability of the model in practical applications can be improved. In a context Context, the sum of the number of tables T and paragraphs P is not less than 2, which ensures the richness and diversity of the data set. This design helps the model better capture the relevance and regularity between information when processing complex and changeable data.

[0073] Among them, each important information is integrated into different paragraphs P or tables T, and the precise geographical location and bridge position pile number information in the important information are desensitized to ensure data privacy. In this way, the data can be effectively protected from being obtained by unauthorized personnel, thereby ensuring data privacy.

[0074] S2. Use the preset pre-trained language model to encode the text and table in the multi-text-multi-table dataset respectively to obtain the corresponding text encoding and table encoding.

[0075] Among them, the pre-trained language model is RoBERTa. Selecting RoBERTa as the pre-trained language model in step S2 can significantly improve the model's ability to understand the text and table information in the bridge inspection report, the encoding efficiency and accuracy, as well as the model's generalization ability and training efficiency. These effects together provide strong support for subsequent problem solving and answer generation. In specific implementation, RoBERTa can be replaced with other table pre-trained models such as TAPAS, T5, BERT, BART, etc.

[0076] The process of encoding text in a multi-text-multi-table dataset includes:

[0077] For a paragraph P in a Context, first, Jieba is used to segment the paragraph into sentences, and multiple sentences S are obtained, P = (S1, S2, ..., S t ), t is the number of sentences; in specific implementation, jieba can be replaced by other sentence segmentation methods such as SpaCy, NLTK, etc., as long as they can achieve the corresponding functions, which will not be repeated here.

[0078] Then, use the pre-trained language model to encode each sentence to get the representation of each sentence i is the sentence number;

[0079] Then, the representation of each sentence in the paragraph Aggregate using the average pooling method to get the representation of the paragraph j is the number of the paragraph;

[0080] Finally, the representations of all paragraphs of the Context are aggregated through the concatenation method to obtain the overall text representation H of the Context T .

[0081] In this way, the encoding process combines the powerful encoding capabilities of the pre-trained language model with a multi-level information aggregation method to improve the efficiency and accuracy of text encoding. By refining steps such as text processing, sentence-level encoding, paragraph-level aggregation, and context-level integration, the model can more accurately capture and understand the key information in the text, providing strong support for subsequent problem solving and answer generation. In addition, the encoding process is suitable for processing complex and changeable text data, such as the multi-text-multi-table mixed context in the bridge inspection report. Through multi-level information aggregation and integration, the model can cope with the diversity and complexity of text data and improve processing effects and accuracy.

[0082] The process of encoding tables in a multi-text-multi-table dataset includes:

[0083] For a table T in a Context, use the pre-trained language model to encode the data of each cell in each column of the table;

[0084] For each row, use weighted average pooling to aggregate the vector representations of the data of each cell in the row into the representation of the row;

[0085] Concatenate the representations of all rows to get the vector representation of table T

[0086] Then, all the table representations of the Context are aggregated using the splicing method to obtain the overall table representation of the Context.

[0087] In this way, by encoding each cell in each column of data in the table, the fine information at the cell level can be captured. This encoding method can retain the original characteristics of the table data and provide a basis for subsequent information aggregation and table understanding. The weighted average pooling method is used to aggregate the vector representation of each cell data in each row into the representation of the row, realizing information integration from the cell level to the row level. This aggregation method not only retains the association information between cells in the row, but also simplifies the subsequent processing process. In addition, by splicing the representations of all rows, the vector representation of the table T is obtained, realizing comprehensive information integration from the row level to the table level. This integration method helps the model capture the global information and structural features in the table and improve the accuracy and depth of table understanding. The representations of all tables in the Context are aggregated using the splicing method to obtain the overall table representation of the Context, realizing comprehensive integration of table information from a single table to the context level. This integration method helps the model better capture and understand the association information and contextual relationships between tables when processing multi-text-multi-table mixed contexts.

[0088] In addition, the encoding process improves the efficiency and accuracy of table encoding through steps such as cell-level fine encoding, row-level information aggregation, table-level overall representation, and context-level table information integration, and supports the processing and understanding of complex table data. These effects together provide strong support for subsequent problem solving and answer generation.

[0089] S3. Use the CoT method to decompose the complex problem to be processed into multiple sub-problems in sequence, and obtain the context information related to each sub-problem.

[0090] Each sub-problem is then classified into retrieval sub-problems and computational sub-problems.

[0091] Chain-of-Thought (CoT) is a method that can significantly improve the logical reasoning ability of a large language model by guiding it to answer questions step by step.

[0092] S4. Combine the text encoding and table encoding obtained in S2, and use the large language model F to solve each sub-problem and obtain preliminary answers to each sub-problem. Among them, for computational sub-problems, the thinking program PoT technology is used to guide the large language model F to solve them.

[0093] Thinking program PoT technology, that is, program-based thinking chain (Program of thoughts, PoT); this technology further utilizes context learning to guide the large language model to generate program code to solve complex numerical reasoning tasks, thereby improving the accuracy of the answer.

[0094] In specific implementation, when solving each sub-problem, if the reasoning of a sub-problem requires external knowledge, the large language model F generates a query based on the sub-problem and obtains the knowledge related to the sub-problem from the knowledge base. In this way, when the reasoning of a sub-problem needs to rely on external knowledge, the large language model F can actively generate query statements and retrieve relevant information from the knowledge base. This capability significantly enhances the model's reasoning ability when dealing with complex, professional knowledge-intensive problems. By obtaining accurate and authoritative knowledge from the knowledge base, the large language model F can more accurately answer sub-problems that require professional knowledge support. This helps to improve the accuracy and credibility of the answers of the overall question-answering system. In addition, the introduction of the knowledge base enables the large language model F to handle a wider range of types of questions, especially those involving professional knowledge in the bridge field. This expands the knowledge coverage of the model, making it more suitable for specific fields such as bridge maintenance. In addition, by combining text, table encoding, and knowledge obtained from the knowledge base, the large language model F can more comprehensively understand and utilize various types of information. This helps to promote the integration and efficient use of knowledge and improve the overall performance of the question-answering system.

[0095] S5. Obtain the preset reference answer for each sub-question; compare the preliminary answer to each sub-question with the preset reference answer to see if they are consistent; for inconsistent sub-questions, return to S4 to generate a new preliminary answer and compare it with the corresponding preset reference answer again until the preset reference answer is consistent with the preliminary answer, thus completing the verification and tuning of the large language model F;

[0096] The preset reference answers to the sub-questions are obtained by resolving the sub-questions with the GPT-4 model. Compared with other models, the GPT-4 model has excellent performance in question-answering tasks and can ensure the effectiveness of the preset reference answers. In specific implementation, the preset reference answers can also be obtained by other means, such as by manually annotating the preset reference answers to the sub-questions in advance, or by using other well-performing models for prediction and solution, etc.

[0097] S6. Taking each sub-question, the answer to each sub-question and the corresponding prompt as context, the final answer A of the complex question to be processed is comprehensively generated by verifying and optimizing the large language model F.

[0098] Prompts are used to clearly indicate the tasks that the large language model needs to perform and provide relevant contextual information to guide the large language model to generate specific, accurate, and coherent outputs or responses.

[0099] The present invention constructs a multi-text-multi-table dataset for the bridge field by extracting text and tables from the periodic bridge inspection report. This helps the model to better understand the professional terms and context in the bridge field and improve the accuracy of questions and answers. The existing technology often relies on general data sets and lacks special data sets for the bridge field. The present method realizes the effective integration and utilization of knowledge in the bridge field by constructing a special data set. In addition, the text and table are encoded using a preset pre-trained language model to obtain the corresponding text encoding and table encoding. This helps the model to understand and utilize the information in the text and table at the same time when dealing with complex problems. The existing technology often processes text and tables separately and lacks an effective fusion mechanism. The present method realizes the seamless connection of text and table information through encoding fusion, and improves the information processing capability of the model.

[0100] In addition, this method uses the thinking chain CoT method to decompose complex problems into multiple sub-problems, and classify them into retrieval-type and computational sub-problems. This helps the model to gradually solve complex problems and reduce the difficulty of solving them. At the same time, combined with the large language model F and the thinking program PoT technology, the accurate solution of sub-problems is achieved. The existing technology often lacks effective decomposition and solution strategies when dealing with complex problems. This method uses the thinking chain CoT method and classification strategy to achieve effective decomposition and accurate solution of complex problems, and improves the solution efficiency of the model. In addition, this method re-solves each sub-problem and the corresponding context through the GPT-4 model, and compares it with the preliminary answer, thereby realizing the verification and tuning of the large language model F. Finally, the final answer to the complex question is generated by combining each sub-problem, answer and prompt Prompt as context. This helps to ensure the accuracy and completeness of the answer. The existing technology often lacks a verification and tuning mechanism for the model, which makes it difficult to ensure the accuracy of the answer. This method achieves strict control of the answer quality through the verification and tuning of the GPT-4 model and the final answer generation strategy, and improves the reliability and practicality of the model.

[0101] This method effectively solves the challenges faced by existing technical methods in processing the long text-multiple table mixed context in bridge inspection reports by building a special data set for the bridge field, realizing the coding fusion of text and tables, effectively decomposing and solving complex problems, and verifying and tuning the model and generating the final answer. It achieves accurate and efficient acquisition of bridge inspection results and key analysis information, providing strong technical support for the field of bridge management and maintenance.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. The thinking chain and thinking program combined reasoning method for multi-text-multi-table question and answer is characterized by: The following steps are involved: S1. Extract text and tables from bridge periodic inspection reports to construct a multi-text-multi-table dataset for the bridge field; S2, using a preset pre-trained language model, respectively encode the text and table in the multi-text-multi-table dataset to obtain corresponding text encoding and table encoding; S3. Use the CoT method to decompose the complex problem to be processed into multiple sub-problems in a sequential combination, and obtain the context information related to each sub-problem; then classify each sub-problem into retrieval sub-problems and calculation sub-problems; S4, combining the text encoding and table encoding obtained in S2, using the large language model F to solve each sub-problem and obtain preliminary answers to each sub-problem; among them, for the computational sub-problems, the thinking program PoT technology is used to guide the large language model F to solve; S5. Obtain the preset reference answer for each sub-question; compare the preliminary answer to each sub-question with the preset reference answer to see if they are consistent; for inconsistent sub-questions, return to S4 to generate a new preliminary answer and compare it with the corresponding preset reference answer again until the preset reference answer is consistent with the preliminary answer, thus completing the verification and tuning of the large language model F; S6. Taking each sub-question, the answer to each sub-question and the corresponding prompt as context, the final answer to the complex question to be processed is comprehensively generated by verifying and tuning the large language model F.

2. The method for combining thinking chain and thinking procedure in multi-text-multi-table question-answering as claimed in claim 1 is characterized by: In S4, when solving each sub-problem, if the reasoning of a sub-problem requires external knowledge, the large language model F generates a query based on the sub-problem and obtains knowledge related to the sub-problem from the knowledge base.

3. The method for combining thinking chain and thinking procedure in multi-text-multi-table question-answering as claimed in claim 1, characterized in that: S1 includes: Obtaining regular bridge inspection reports and extracting important information from the reports, including bridge overview, bridge disease information inspection result table, bridge technical condition assessment table, and assessment specifications; Integrate each important information into different paragraphs P or tables T according to its format and content; The integrated associated paragraphs P and / or tables T are integrated into a hybrid context Context to complete the construction of a multi-text-multi-table dataset for the bridge field; wherein, in one context Context, the sum of the number of tables T and paragraphs P is not less than 2.

4. The method for combining thinking chain and thinking procedure in multi-text-multi-table question-answering as claimed in claim 3 is characterized by: In S1, each important information is integrated into different paragraphs P or tables T, and the precise geographical location and bridge position pile number information in the important information are desensitized.

5. The method for combining thinking chain and thinking procedure in multi-text-multi-table question-answering as claimed in claim 3 is characterized by: In S2, the pre-trained language model is RoBERTa.

6. The method for combining thinking chain and thinking procedure in multi-text-multi-table question-answering as claimed in claim 3, characterized in that: In S2, the process of encoding text in a multi-text-multi-table dataset includes: For a paragraph P in a Context, first, the paragraph is divided into sentences to obtain multiple sentences S, P = (S1, S2, ..., S t ), t is the number of sentences; Then, use the pre-trained language model to encode each sentence to get the representation of each sentence i is the sentence number; Then, the representation of each sentence in the paragraph Aggregate using the average pooling method to get the representation of the paragraph j is the number of the paragraph; Finally, the representations of all paragraphs of the Context are aggregated through the splicing method to obtain the overall text representation H of the Context T .

7. The method for combining thinking chain and thinking procedure in multi-text-multi-table question-answering as claimed in claim 6, characterized in that: Use Jieba to divide each paragraph into sentences.

8. The method for combining thinking chain and thinking procedure in multi-text-multi-table question-answering as claimed in claim 3 is characterized by: In S2, the process of encoding tables in the multi-text-multi-table dataset includes: For a table T in a Context, use the pre-trained language model to encode the data of each cell in each column of the table; For each row, use weighted average pooling to aggregate the vector representations of the data of each cell in the row into the representation of the row; Concatenate the representations of all rows to get the vector representation of table T Then, all the table representations of the Context are aggregated using the splicing method to obtain the overall table representation of the Context.

9. The method for combining thinking chain and thinking procedure in multi-text-multi-table question-answering as claimed in claim 1, characterized in that: In S5, the preset reference answers to the sub-questions are obtained by resolving the sub-questions by the GPT-4 model, or are obtained in advance by manually annotating the preset reference answers to the sub-questions.

Images