BERT model-based engineering investigation report automatic generation system and method

Abstract

The invention provides an engineering investigation report automatic generation system and method based on a BERT model. The system comprises a data preprocessing module, a BERT model optimization customization module, an information extraction and structuring module and a report generation engine. According to the method, the limitation that time and labor are consumed and errors are prone to occurring in the traditional report compiling process can be overcome, the engineering investigation data can be automatically analyzed and processed by integrating an advanced BERT model for optimization and customization, then a high-quality report meeting the industry standard and the customer requirement is generated, and intelligent processing of the engineering investigation data and automatic report compiling are achieved.

Description

Technical Field

[0001] The present invention relates to the field of engineering survey, and in particular to a system and method for automatically generating an engineering survey report based on a BERT model. Background Art

[0002] As the foundation of construction projects, the accuracy and timeliness of engineering survey reports are crucial. Traditionally, engineering survey reports rely on manual writing by experienced engineers, a process that is not only time-consuming and costly, but also easily affected by personal experience and subjective judgment, resulting in uneven report quality. With the rapid development of artificial intelligence technology, especially breakthroughs in natural language processing technology, such as the emergence of the BERT model, it is possible to automatically generate professional texts. However, how to effectively apply these advanced technologies to the highly specialized field of engineering survey reports remains a technical challenge to be solved. Summary of the invention

[0003] The present invention provides a system and method for automatically generating engineering survey reports based on a BERT model, aiming to overcome the limitations of the traditional report writing process, which is time-consuming, labor-intensive and error-prone. By integrating an advanced BERT model for optimization and customization, it is capable of automatically analyzing and processing engineering survey data, thereby generating high-quality reports that meet industry standards and customer needs, and realizing intelligent processing of engineering survey data and automatic report preparation.

[0004] An automatic generation system for engineering survey reports based on the BERT model, including

[0005] Data preprocessing module, used to collect multi-dimensional data from engineering survey sites, perform data cleaning, standardization and text coding;

[0006] The BERT model optimization and customization module is used to pre-process professional literature to obtain professional corpus, and use the professional corpus to perform transfer learning and domain adaptability adjustment based on the BERT model to enhance the BERT model's understanding of domain-specific vocabulary and expressions; through multiple rounds of iterative training and verification, the performance of the BERT model in named entity recognition and relationship extraction tasks is continuously improved, and an optimized and customized BERT model is obtained to serve the processing of engineering survey text data;

[0007] An information extraction and structuring module, used to extract key data from the data preprocessed by the data preprocessing module using the optimized customized BERT model, and structure the key data into a logically structured information unit;

[0008] The report generation engine is used to use the optimized and customized BERT model to perform contextual association analysis on the input structured information units to generate a draft report. Through the built-in professional rule engine, according to the report writing standards in the field of engineering survey, the report draft is formatted, the language is polished, necessary industry terminology analysis is added, charts are generated, and targeted improvement suggestions are put forward. The final output is an engineering survey report covering a summary, current situation overview, problem analysis and countermeasures.

[0009] Furthermore, the multi-dimensional data includes geological, hydrological and environmental information.

[0010] Furthermore, data cleaning is performed in the data preprocessing module, specifically including: using regular expressions and natural language processing technology to remove invalid information, wherein the invalid information includes punctuation marks, numbers, and irrelevant characters.

[0011] Furthermore, the data preprocessing module performs standardization processing, specifically including: normalizing the numerical data to ensure the consistency and comparability of the data.

[0012] Furthermore, text encoding is performed in the data preprocessing module, specifically including: using the Tokenizer tool provided by the BERT model to convert the text data into the input format required by the BERT model, including word segmentation and adding special tags.

[0013] Furthermore, the BERT model optimization customization module pre-processes professional documents, including: using NLTK or spaCy tools to segment and tag the professional documents to obtain professional corpus.

[0014] Furthermore, the BERT model optimization customization module is based on the BERT model and uses the professional corpus for transfer learning, including: using the professional corpus to fine-tune the BERT model, including adjusting the model structure and parameters, and the loss function used in the fine-tuning process is the cross-entropy loss function; performing domain adaptability adjustments, including adding domain-specific embedding layers and optimizing the learning rate.

[0015] Furthermore, the structured information unit includes geological characteristics, groundwater level, and soil type.

[0016] Furthermore, the key data extracted by the information extraction and structuring module include geological characteristics, groundwater level and soil type.

[0017] A method for automatically generating an engineering survey report based on a BERT model comprises the following steps:

[0018] Collect multi-dimensional data from engineering survey sites, perform data cleaning, standardization and text coding;

[0019] Preprocess professional literature to obtain professional corpus, use the professional corpus to perform transfer learning and domain adaptability adjustment based on the BERT model, and enhance the BERT model's understanding of domain-specific vocabulary and expressions; through multiple rounds of iterative training and verification, continuously improve the performance of the BERT model in named entity recognition and relationship extraction tasks, and obtain an optimized and customized BERT model to serve engineering survey text data processing;

[0020] Utilizing the optimized customized BERT model, extracting key data from the preprocessed data, and structuring the key data into logically structured information units;

[0021] The optimized and customized BERT model is used to perform contextual analysis on the input structured information units to generate a draft report. Through the built-in professional rule engine, according to the report writing standards in the field of engineering survey, the draft report is formatted, the language is polished, necessary industry terminology is analyzed, charts are generated, and targeted improvement suggestions are made. The final output is an engineering survey report that includes a summary, current situation overview, problem analysis, and countermeasures.

[0022] The present invention has the following beneficial effects:

[0023] 1. Deep customization: Through the deeply customized BERT model, the professional context in the field of engineering survey is accurately captured, which significantly improves the professionalism and accuracy of the report content.

[0024] 2. High degree of automation: The entire process from data processing to report generation is automated, which greatly shortens the report production cycle and reduces the manpower burden.

[0025] 3. Dynamic optimization: The introduced feedback loop mechanism enables the system to continuously learn and self-optimize based on user feedback and actual application effects.

[0026] 4. Personalized service: While ensuring standardization, the system supports a certain degree of report personalization to meet the specific needs of different projects and customers. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the method for automatically generating an engineering survey report based on the BERT model of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] An embodiment of the present invention provides an automatic generation system for engineering survey reports based on a BERT model, comprising: a data preprocessing module, a BERT model optimization and customization module, an information extraction and structuring module, and a report generation engine.

[0030] Data preprocessing module, used to collect multi-dimensional data from engineering survey sites, perform data cleaning, standardization and text coding;

[0031] The multi-dimensional data includes geological, hydrological, environmental information, etc. The collected data is cleaned, standardized and text-coded. The specific implementation steps are as follows:

[0032] Data collection: Use sensors, drones, geological drilling equipment and other tools to collect geological, hydrological and environmental information at the engineering survey site. This data can include soil samples, groundwater level measurements, topographic maps, etc.

[0033] Data cleaning: Use regular expressions and natural language processing techniques to remove invalid information, such as punctuation, numbers, irrelevant characters, etc. For example, you can use Python's re library to perform regular matching and replacement operations.

[0034] Standardization: Normalize numerical data to ensure data consistency and comparability. The normalization formula is as follows:

[0035]

[0036] Among them, x is the original value, x' is the standardized value, and x min and x max are the minimum and maximum values ​​of the attribute, respectively.

[0037] Text encoding: Convert text data into the input format required by the BERT model, including word segmentation, adding special tags, etc. You can use the Tokenizer tool that comes with the BERT model for word segmentation and encoding.

[0038] The BERT model optimization and customization module is used to pre-process professional literature to obtain professional corpus. Based on the BERT model, the professional corpus is used for transfer learning and domain adaptability adjustment to enhance the BERT model's understanding of domain-specific vocabulary and expressions. Through multiple rounds of iterative training and verification, the performance of the BERT model in named entity recognition and relationship extraction tasks is continuously improved to obtain an optimized and customized BERT model to serve engineering survey text data processing.

[0039] The specific implementation steps are as follows:

[0040] Preprocessing: Use tools such as NLTK or spaCy to segment and tag professional documents to obtain professional corpus. For example, you can use spaCy's nlp object for segmentation and part-of-speech tagging.

[0041] Transfer learning: Use professional corpus to fine-tune the BERT model and optimize model parameters. The loss function used in the fine-tuning process is the cross entropy loss function: Among them, y represents the true label, represents the predicted probability distribution.

[0042] Domain adaptability adjustment: Add domain-specific embedding layers, optimize learning rates, etc. to improve the domain adaptability of the model. For example, you can use the nn.Embedding layer in the PyTorch framework to add a domain-specific embedding layer and use the Adam optimizer for training.

[0043] The information extraction and structuring module is used to use the optimized customized BERT model to extract key data from the data preprocessed by the data preprocessing module, and structure the key data into a logically strong structured information unit to facilitate subsequent report generation. The structured information unit includes geological characteristics, groundwater level, and soil type.

[0044] The specific implementation steps are as follows:

[0045] Key data identification: Use named entity recognition (NER) technology, combined with domain-specific vocabulary, to identify key data. During the identification process, you can set a threshold to decide whether to extract an entity:

[0046] if score(e)>threshold,then extract(e) where score(e) is the confidence score of entity e and threshold is a pre-set threshold.

[0047] Information structuring: The identified key data are organized into structured information units by category, each of which contains labels such as geological characteristics, groundwater level, soil type, etc. For example, Python dictionaries or DataFrame structures can be used to store structured information.

[0048] The report generation engine is used to use the optimized and customized BERT model to perform contextual association analysis on the input structured information units to generate a draft report. Through the built-in professional rule engine, according to the report writing standards in the field of engineering survey, the report draft is formatted, the language is polished, necessary industry terminology analysis is added, charts are generated, and targeted improvement suggestions are put forward. The final output is an engineering survey report covering a summary, an overview of the current situation, an analysis of problems, and countermeasures and suggestions, ensuring the quality and professionalism of the report.

[0049] The specific implementation steps are as follows:

[0050] Contextual relevance analysis: Use the optimized and customized BERT model to perform contextual relevance analysis on structured information units to generate coherent text paragraphs. For example, you can use the generate method of the model to generate text.

[0051] Report formatting: Format the generated text according to the standard format of engineering survey reports, including adding titles, sections, charts, etc. Report layout can be done in Markdown or LaTeX format.

[0052] Language polishing: Use the probability evaluation mechanism of the language model to ensure that the generated language is fluent and professional. The probability evaluation formula is as follows:

[0053]

[0054] Among them, p(w1,w2,...,w n ) represents the probability of the entire sentence, p(w i |w1,w2,...,w i-1 ) represents the probability of the ii-th word appearing given the previous context.

[0055] See also Figure 1 The embodiment of the present invention also provides a method for automatically generating an engineering survey report based on a BERT model, comprising the following steps:

[0056] Collect multi-dimensional data from engineering survey sites, perform data cleaning, standardization and text coding;

[0057] Preprocess professional literature to obtain professional corpus, use the professional corpus to perform transfer learning and domain adaptability adjustment based on the BERT model, and enhance the BERT model's understanding of domain-specific vocabulary and expressions; through multiple rounds of iterative training and verification, continuously improve the performance of the BERT model in named entity recognition and relationship extraction tasks, and obtain an optimized and customized BERT model to serve engineering survey text data processing;

[0058] Utilizing the optimized customized BERT model, extracting key data from the preprocessed data, and structuring the key data into logically structured information units;

[0059] The optimized and customized BERT model is used to perform contextual analysis on the input structured information units to generate a draft report. Through the built-in professional rule engine, according to the report writing standards in the field of engineering survey, the draft report is formatted, the language is polished, necessary industry terminology is analyzed, charts are generated, and targeted improvement suggestions are made. The final output is an engineering survey report that includes a summary, current situation overview, problem analysis, and countermeasures.

[0060] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An automatic generation system for engineering survey reports based on the BERT model, characterized in that: include: Data preprocessing module, used to collect multi-dimensional data from engineering survey sites, perform data cleaning, standardization and text coding; The BERT model optimization and customization module is used to pre-process professional literature to obtain professional corpus, and use the professional corpus to perform transfer learning and domain adaptability adjustment based on the BERT model to enhance the BERT model's understanding of domain-specific vocabulary and expressions; through multiple rounds of iterative training and verification, the performance of the BERT model in named entity recognition and relationship extraction tasks is continuously improved, and an optimized and customized BERT model is obtained to serve the processing of engineering survey text data; An information extraction and structuring module, used to extract key data from the data preprocessed by the data preprocessing module using the optimized customized BERT model, and structure the key data into a logically structured information unit; The report generation engine is used to use the optimized and customized BERT model to perform contextual association analysis on the input structured information units to generate a draft report. Through the built-in professional rule engine, according to the report writing standards in the field of engineering survey, the report draft is formatted, the language is polished, necessary industry terminology analysis is added, charts are generated, and targeted improvement suggestions are put forward. The final output is an engineering survey report covering a summary, current situation overview, problem analysis and countermeasures.

2. The automatic generation system of engineering survey report based on BERT model as claimed in claim 1, characterized in that: The multi-dimensional data includes geological, hydrological and environmental information.

3. The automatic generation system of engineering investigation report based on BERT model as claimed in claim 1, characterized in that: The data preprocessing module performs data cleaning, specifically including: using regular expressions and natural language processing technology to remove invalid information, wherein the invalid information includes punctuation marks, numbers, and irrelevant characters.

4. The automatic generation system of engineering investigation report based on BERT model as claimed in claim 1, characterized in that: The data preprocessing module performs standardization processing, specifically including: normalizing the numerical data to ensure the consistency and comparability of the data.

5. The automatic generation system of engineering investigation report based on BERT model as claimed in claim 1, characterized in that: The data preprocessing module performs text encoding, specifically including: using the Tokenizer tool provided by the BERT model to convert text data into the input format required by the BERT model, including word segmentation and adding special tags.

6. The automatic generation system of engineering investigation report based on BERT model as claimed in claim 1, characterized in that: The BERT model optimization customization module pre-processes professional documents, including: using NLTK or spaCy tools to segment and tag the professional documents to obtain professional corpus.

7. The automatic generation system of engineering investigation report based on BERT model as claimed in claim 1, characterized in that: The BERT model optimization customization module is based on the BERT model and uses the professional corpus for transfer learning, including: using the professional corpus to fine-tune the BERT model, including adjusting the model structure and parameters, and the loss function used in the fine-tuning process is the cross entropy loss function; performing domain adaptability adjustment, including adding domain-specific embedding layers and optimizing the learning rate.

8. The automatic generation system of engineering investigation report based on BERT model as claimed in claim 1, characterized in that: The structured information units include geological characteristics, groundwater levels, and soil types.

9. The automatic generation system of engineering investigation report based on BERT model as claimed in claim 1, characterized in that: The key data extracted by the information extraction and structuring module include geological characteristics, groundwater level and soil type.

10. A method for automatically generating engineering survey reports based on a BERT model, characterized in that: The steps include: Collect multi-dimensional data from engineering survey sites, perform data cleaning, standardization and text coding; Preprocess professional literature to obtain professional corpus, use the professional corpus to perform transfer learning and domain adaptability adjustment based on the BERT model, and enhance the BERT model's understanding of domain-specific vocabulary and expressions; through multiple rounds of iterative training and verification, continuously improve the performance of the BERT model in named entity recognition and relationship extraction tasks, and obtain an optimized and customized BERT model to serve engineering survey text data processing; Utilizing the optimized customized BERT model, extracting key data from the preprocessed data, and structuring the key data into logically structured information units; The optimized and customized BERT model is used to perform contextual analysis on the input structured information units to generate a draft report. Through the built-in professional rule engine, according to the report writing standards in the field of engineering survey, the draft report is formatted, the language is polished, necessary industry terminology is analyzed, charts are generated, and targeted improvement suggestions are made. The final output is an engineering survey report that includes a summary, current situation overview, problem analysis, and countermeasures.

Images