Bridge experiment data report generation mode and system based on natural language processing

Abstract

The invention discloses a bridge experiment data report generation method and system based on natural language processing, and the method comprises the steps: S1, collecting original data from a bridge health detection experiment, and carrying out the format conversion of the original data, and obtaining recognizable experiment data; s2, analyzing the experimental data obtained in the step S1 through statistical analysis to extract key statistical information and an anomaly detection result; and S3, converting an analysis result in the S2 into a natural language text through a natural language processing technology NLP, and embedding the natural language text into a predefined report template through a template engine to form an experimental data report. According to the bridge experiment data report generation method and system based on natural language processing provided by the invention, by introducing a natural language processing technology and automatically generating a bridge experiment data report, the efficiency and accuracy of the report are improved, and meanwhile, the standardization and consistency of the report are realized; the method has remarkable technical progress and application value.

Description

Technical Field

[0001] The present invention relates to the field of automatic report generation, and more specifically, to a bridge test data report generation method and system based on natural language processing. Background Art

[0002] At present, most of the traditional report writing and production processes still rely on manual input and editing, which mainly has the following disadvantages: 1. It is time-consuming and inefficient. Different personnel have different understandings of relevant requirements, which makes it easy to make mistakes.

[0003] 2. Lack of consistency and standardization leads to inconsistent report formats and contents, affecting the readability and comparability of reports.

[0004] 3. Problems such as improper language expression and missing content are likely to occur, affecting the accuracy and professionalism of the report.

[0005] Even though there are report generation systems in the prior art to automatically generate reports, there are still the following deficiencies: lack of a complete quality review mechanism, and inability to guarantee the accuracy and completeness of the generated reports. Summary of the invention

[0006] An object of the present invention is to solve at least the above problems and / or disadvantages and to provide at least the advantages which will be described hereinafter.

[0007] In order to achieve these purposes and other advantages of the present invention, a method for generating a bridge test data report based on natural language processing is provided, comprising: S1. Collect the raw data from the bridge health detection experiment and convert its format to obtain recognizable experimental data; S2, analyzing the experimental data obtained in S1 through statistical analysis to extract key statistical information and anomaly detection results; S3, convert the analysis results of S2 into natural language text through natural language processing technology NLP, and embed the natural language text into a predefined report template through a template engine to form an experimental data report; S4. Perform automatic and / or manual review of the experimental data report generated by S3 through an automated quality inspection tool and output a report.

[0008] Preferably, in S1, the raw data includes strain data, displacement data, vibration data, and temperature data obtained by sensors; The format conversion is to process the original data into JSON or CSV format through a data conversion tool.

[0009] Preferably, in S3, the process of forming the experimental data report includes: S30, for each state, it is divided into i levels according to the specification requirements, and each level is determined by n parameters, denoted as S i ,and , Indicates i Under the state j The measured data of the parameters, i =1,2,… m, j =1,2,… n， m represents the number of data sets collected; S31, let the membership degree between each level of the corresponding state of each parameter be f jk ( x ), and 0≤ f jk ( x )≤1, j =1,2,…, n , k =1,2,…, i ; S32, standardize the parameter matrix corresponding to the m groups of collected data by the following formula: In the above formula, For the i Under the state j The data after parameter standardization; S33. Based on the standardized matrix, the fuzzy evaluation set is obtained by the following formula F : S34. Based on the fuzzy evaluation set, the membership degree of each level is determined by the following formula: S35. According to the maximum membership principle, the measured data is converted into corresponding descriptive text, imported into a pre-made template, and a report is generated.

[0010] Preferably, in S4, the automatic review method is: S40. The standard reference report and the report to be audited are processed in sections in the same way; S41, calculating the paragraph similarity of each paragraph in the standard reference report and the report to be reviewed to determine the paragraph similarity between the two; S42. When the proportion of paragraphs that meet the requirements for paragraph similarity is greater than 90%, the report to be reviewed is considered to meet the requirements.

[0011] Preferably, in S41, the paragraph similarity determination process is as follows: S410, calculate the frequency of important words in each paragraph of the standard reference report and the report to be reviewed , where w is the total number of text characters, w 0 is the number of word occurrences; S411. Arrange the words from high to low according to the frequency of the words to form a word vector ; S412, assuming that the word vectors of each section of the standard reference report and the report to be reviewed are and , the paragraph similarity corresponding to each paragraph is calculated by the following formula R : S413: Paragraph similarity R With a predetermined threshold r For comparison, when R ≥ r , then the paragraph similarity is considered to meet the requirements.

[0012] A system is applied to a method for automatically generating a bridge test data report based on natural language processing, the system comprising: Collect experimental data from bridge inspection experiments and input the data into a module in a certain format; A data analysis module that analyzes and processes the collected experimental data to extract key information and statistical data; A report generation module that uses natural language processing technology to convert analysis results into structured report text; Results review module for quality checking and optimization of automatically generated reports.

[0013] The present invention includes at least the following beneficial effects: the present invention automates the generation process of bridge test data reports by introducing natural language processing technology, thereby improving the efficiency and accuracy of reports, while achieving standardization and similarity of reports, and has significant technical progress and application value.

[0014] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The present invention is a flow chart of a method for generating a bridge test data report based on natural language processing. DETAILED DESCRIPTION

[0016] The present invention is further described in detail below in conjunction with the accompanying drawings so that those skilled in the art can implement the invention with reference to the description.

[0017] like Figure 1 As shown, a method for generating a bridge test data report based on natural language processing includes the following steps: 1. Data input Collect raw data from bridge health testing experiments (which can be collected manually or automatically), including sensor data such as strain, displacement, vibration, etc. Process the raw data into a format acceptable to the system, such as JSON or CSV, through data conversion tools.

[0018] The data input stage also includes the establishment of a template library, which can be established in two ways: system template and manual template.

[0019] 2. Data Analysis The data analysis mainly includes: chart drawing, model building, online analysis, etc.

[0020] Specifically, based on the comparison with existing data, the similarity coefficient R The method queries and verifies key data, uses statistical principles (such as translation-range transformation) to standardize the collected data, and compresses the data to facilitate the establishment of word vectors.

[0021] The system is directly connected to existing data analysis software (such as Excel, etc.), which can realize real-time analysis of text data, extract key statistical information and anomaly detection results, and then generate visual charts of experimental data to complete chart drawing, convert them into computer-processable formats such as tables, and combine the evaluation model construction based on the maximum membership principle of fuzzy sets to realize online analysis and processing of text.

[0022] 3. Report Generation Natural language processing (NLP) technology is used to convert the analysis results into natural language text, including data summary, trend analysis, anomaly detection results and recommended measures. The text generated by NLP is embedded into the predefined report template through the template engine to realize catalog arrangement and results summary, thus forming a complete experimental report.

[0023] The specific implementation process is as follows: Assume that a certain state is divided into five levels according to the specification requirements: "severe damage", "damage", "damage", "minor damage", and "normal use". n parameters are determined, and each parameter corresponds to i The status level is recorded as , Indicates i Under the state j The measured data of parameters; let the membership degree of each parameter corresponding to the 5 states be f jk (x ), and 0≤ f jk ( x )≤1, where j =1,2,…, n , k =1,2,3,4,5, corresponding to the above 5 status levels, such as f 12 ( x ) indicates the first parameter ( p 1) Membership function belonging to the second state ("destruction"), which can be determined according to relevant specifications.

[0024] Assume that a collection is obtained m Group data and obtain the parameter matrix S as follows: The data normalization method is: , , , For each group of data j The average value of the parameters, For the i Under the state j After the parameters are standardized, For each group of data j The standard deviation of the parameters; Based on the above matrix, obtain the fuzzy evaluation set F ,and , the final calculation F The intersection of the fuzzy sets in is: Determine the corresponding k =1,2,3,4,5, a total of 5 states of membership. According to the principle of maximum membership, the measured data is converted into corresponding description texts, imported into the corresponding positions of the pre-made templates, and a report is formed after cataloging to summarize the results.

[0025] The present invention uses a unified template to quantify standard requirements in the algorithm to achieve the professionalism and standardization required by the report. Based on computer technology, the efficiency of compiling reports is greatly improved. Based on the comparison of standard reports, the review of reports can be quickly carried out, and only a small amount of manual review is required, which greatly optimizes the workload.

[0026] 4. Results review Implement automated quality inspection tools to review the generated reports (including grammar checks, logic checks, etc.) to identify grammatical errors, logical inconsistencies, and data omissions in the text. Provide a manual review interface to allow experts to conduct a final review and modification of the automatically generated reports to ensure the quality and accuracy of the reports. The implementation steps are as follows: (1) Provide a standard reference report, pre-process the standard report P1 and the report to be reviewed P2 (such as word segmentation, removal of stop words, etc.), and then segment P1 and P2 using the same method; (2) Calculation of natural paragraph similarity (a) Calculate the frequency of important words (which can be set by yourself) in each paragraph of the two reports ,in, w is the total number of text characters, w 0 is the number of times a word appears. The words are arranged from high to low according to their frequency to form a word vector (b) Calculate the word vectors of the standard reference report and the report to be reviewed respectively and , through and Angle The cosine is calculated to obtain the paragraph similarity R, that is, , set the threshold according to the actual situation r (0≤ r ≤1), such as 0.85, when R ≥ r It is considered that the paragraph similarity meets the requirements.

[0027] (3) Document similarity calculation When the proportion of paragraphs that meet the requirements for paragraph similarity is greater than 90%, it can be considered that the report (document) to be reviewed meets the requirements.

[0028] 5. Report output After automated and manual review, export reports to a variety of formats, including PDF, Word, and HTML, for easy sharing and archiving.

[0029] Furthermore, the present invention relates to an automatic report generation system for bridge test data based on natural language processing (NLP), which aims to automatically generate bridge health detection test reports, reduce manual intervention, and improve the efficiency and accuracy of report generation. The system includes a data input module, a data analysis module, a report generation module, and a result review module. The data input module is responsible for collecting experimental data from bridge detection experiments and converting them into a format that can be processed by the system; the data analysis module analyzes and processes the experimental data to extract key information and statistical data; the report generation module uses natural language processing technology to convert the analysis results into structured report text; the result review module performs quality inspection and optimization on the automatically generated report to ensure the accuracy and completeness of the report. The present invention can effectively improve the efficiency of experimental report generation, reduce manual labor, and enhance the standardization and similarity of reports.

[0030] The above solution is only an illustration of a preferred embodiment, but is not limited thereto. When implementing the present invention, appropriate replacement and / or modification can be performed according to user needs.

[0031] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. A method for generating a bridge test data report based on natural language processing, characterized in that: include: S1. Collect the raw data from the bridge health detection experiment and convert its format to obtain recognizable experimental data; S2, analyzing the experimental data obtained in S1 through statistical analysis to extract key statistical information and anomaly detection results; S3, convert the analysis results of S2 into natural language text through natural language processing technology NLP, and embed the natural language text into a predefined report template through a template engine to form an experimental data report; S4. Perform automatic and / or manual review of the experimental data report generated by S3 through an automated quality inspection tool and output a report.

2. The method for generating a bridge test data report based on natural language processing according to claim 1, characterized in that: In S1, the raw data includes strain data, displacement data, vibration data, and temperature data obtained by sensors; The format conversion is to process the original data into JSON or CSV format through a data conversion tool.

3. The method for generating a bridge test data report based on natural language processing according to claim 1, characterized in that: In S3, the process of generating experimental data reports includes: S30, for each state, it is divided into i levels according to the specification requirements, and each level is determined by n parameters, denoted as S i ,and , Indicates i Under the state j The measured data of the parameters, i =1,2,… m, j =1,2,… n， m represents the number of data sets collected; S31, let the membership degree between each level of the corresponding state of each parameter be f jk ( x ), and 0≤ f jk ( x )≤1, j =1,2,… n , k =1,2,… i ; S32, standardize the parameter matrix corresponding to the m groups of collected data by the following formula: In the above formula, For the i Under the state j The data after parameter standardization; S33. Based on the standardized matrix, the fuzzy evaluation set F is obtained by the following formula: S34. Based on the fuzzy evaluation set, the membership degree of each level is determined by the following formula: S35. According to the maximum membership principle, the measured data is converted into corresponding descriptive text, imported into a pre-made template, and a report is generated.

4. The method for generating a bridge test data report based on natural language processing according to claim 1, characterized in that: In S4, the automatic review method is: S40. The standard reference report and the report to be reviewed are processed in sections in the same way; S41, calculating the paragraph similarity of each paragraph in the standard reference report and the report to be reviewed to determine the paragraph similarity between the two; S42. When the proportion of paragraphs that meet the requirements for paragraph similarity is greater than 90%, the report to be reviewed is considered to meet the requirements.

5. The method for generating a bridge test data report based on natural language processing according to claim 4, characterized in that: In S41, the paragraph similarity determination process is as follows: S410, calculate the frequency of important words in each paragraph of the standard reference report and the report to be reviewed , where w is the total number of text characters, w 0 is the number of word occurrences; S411. Arrange the words from high to low according to the frequency of the words to form a word vector ; S412, assuming that the word vectors of each section of the standard reference report and the report to be reviewed are and , the paragraph similarity corresponding to each paragraph is calculated by the following formula R : S413: Paragraph similarity R With a predetermined threshold r For comparison, when R ≥ r , then the paragraph similarity is considered to meet the requirements.

6. A system, applied to the bridge test data report generation method based on natural language processing as described in any one of claims 1 to 5, characterized in that: The system comprises: Collect experimental data from bridge inspection experiments and input the data into a module in a certain format; A data analysis module that analyzes and processes the collected experimental data to extract key information and statistical data; A report generation module that uses natural language processing technology to convert analysis results into structured report text; Results review module for quality checking and optimization of automatically generated reports.

Images