Municipal engineering inspection question and answer method and system based on AI large model

Abstract

The invention discloses a municipal engineering inspection question and answer method and system based on an AI large model, and relates to the technical field of artificial intelligence. Historical municipal engineering multi-source data are collected and processed, facility entities and relationships are extracted by using a BERT model, triples are screened in combination with a correct rate threshold, and real-time updating is realized based on Kafka + Flink, so that a dynamic municipal engineering knowledge graph is obtained; the YOLOv8 model is improved, question and answer pair data and historical perception data are combined with a wild goose algorithm for optimization, and a municipal engineering inspection large model is obtained; fusing inspection questions, images and sensor data in real time, calling a municipal engineering inspection large model to generate a decision scheme, and feeding back an effect; according to the method, the problems of multi-modal data islands, knowledge updating lag and low complex scene detection precision are solved, the data query efficiency and the complex environment recognition precision are improved, the capacity of processing unknown triads is achieved, the repeated failure rate is reduced, and an intelligent municipal facility operation and maintenance closed loop is formed.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence. Specifically, it particularly relates to a method and system for municipal engineering inspection and answering questions based on an AI large model. Background Art

[0002] With the acceleration of the urbanization process, the scale of municipal infrastructure (such as underground pipe networks, road manhole covers, and drainage systems) continues to expand, and its safe operation and maintenance face severe challenges. In recent years, smart cities and digital transformation have promoted the intelligent upgrading of municipal management; in this context, it is necessary to build an intelligent inspection system that integrates multi-modal data analysis, dynamic knowledge evolution, and high-precision detection to improve the scientific level of fault warning, disposal, and operation and maintenance management of municipal facilities and respond to the strategic needs of new infrastructure construction and urban safety risk prevention and control.

[0003] Existing municipal engineering inspection technologies rely on manual experience to process scattered text, images, and sensor data, resulting in low information utilization; static knowledge graphs are difficult to integrate new fault cases in a timely manner, and the value of decision-making reference is limited; visual detection models have poor adaptability to complex scenarios and lack the ability to jointly reason with knowledge in the field of municipal engineering, affecting the accuracy and timeliness of inspection decisions. Summary of the Invention

[0004] (I) Technical Problems to be Solved Aiming at the problems in the related technologies, the present invention provides a method for municipal engineering inspection and answering questions based on an AI large model to overcome the above-mentioned technical problems existing in the related technologies.

[0005] (II) Technical Solutions To solve the above technical problems, the present invention is realized through the following technical solutions: S1. Collect and process multi-source data of historical municipal engineering to obtain processed historical municipal engineering data; S2. Extract facility entities from the text data in the processed historical municipal engineering data to obtain historical triple data; Construct a dynamic municipal engineering knowledge graph according to the historical triple data and dynamic update rules; S3. Construct a pre-trained large model, embed the improved YOLOv8 model into the pre-trained large model to obtain the embedded pre-trained large model; Collect historical facility image data and sensor data corresponding to the historical triple data in the dynamic municipal engineering knowledge graph to obtain historical perception data; Convert the historical triple data in the dynamic municipal engineering knowledge graph into question-and-answer pair data; use the question-and-answer pair data and historical perception data in combination with an optimization algorithm to improve the pre-trained large model to obtain a municipal engineering inspection large model; S4. Input the real-time inspection personnel's question data, real-time facility image data, and sensor data into the municipal engineering inspection large model to obtain the municipal engineering inspection answers; Through artificial intelligence, deep learning, data fusion, and knowledge graphs, the present invention realizes cross-modal association of text, images, and sensor data, solves the data island problem, improves query efficiency relying on real-time update of stream computing to ensure that decisions are based on the latest knowledge base; improves recognition accuracy and complex scene detection accuracy; the closed-loop decision-making system integrates real-time perception and historical reasoning, generates interpretable disposal plans and feedback on maintenance effects, has the ability to process unknown triples, reduces the repeated failure rate, and forms an intelligent and sustainable closed-loop operation and maintenance of municipal facilities.

[0006] Preferably, the S1 includes the following steps: S11. Collect data through national standard documents, historical inspection reports, maintenance work orders, facility technical manuals, sensor monitoring data, facility parameter tables, facility damage photos, and inspection process videos to obtain historical municipal engineering multi-source data; S12. Clean, segment, and semantically annotate the unstructured text in the historical municipal engineering multi-source data, and standardize the structured data in the historical municipal engineering multi-source data to obtain the processed historical municipal engineering data; The above steps integrate various heterogeneous data to construct a multi-dimensional data set covering facility attributes and failure cases; clean and segment the unstructured text, fill in missing sensor values, unify units, and extract image feature vectors to achieve standardized fusion of multi-modal data.

[0007] Preferably, the S12 includes the following steps: S121. Denoise, segment, and perform part-of-speech and semantic annotation on the unstructured text in the historical municipal engineering multi-source data; S122. Fill in missing values, detect and remove outliers, unify units, and align time series for the sensor data in the historical municipal engineering multi-source data; S123. Use an image feature extraction model to perform object detection and feature extraction on the image data in the historical municipal engineering multi-source data; S124. Obtain the processed historical municipal engineering data through S121, S122, and S123; The above steps denoise and segment the text and perform annotation, fill in missing sensor values and unify units, detect image cracks and rust features, and finally integrate to form standardized historical municipal engineering data.

[0008] Preferably, the S2 includes the following steps: S21. Use a natural language processing model to automatically extract municipal facility entities, attributes, and relationships from the text in the processed historical municipal engineering data to obtain entity data; S22. Set a triple accuracy rate threshold; use a relationship extraction model to extract initial triple data from the entity data; combine the triple accuracy rate threshold to remove triples with fuzzy relationships to obtain historical triple data; S23. Set a knowledge graph node type set according to the historical triple data; the knowledge graph node type set includes facility nodes, fault nodes, and solution nodes; the attributes of the facility nodes include ID, type, coordinates, installation year, the attributes of the fault nodes include fault codes, severity levels, occurrence times, and the attributes of the solution nodes include method names, applicable scenarios, and cost estimates; Set a knowledge graph relationship type set according to the historical triple data, and the knowledge graph relationship type set includes facility - fault, fault - solution, and facility - facility; the facility - fault relationships such as cause, may trigger, the fault - solution relationships such as repair methods, reference cases, and the facility - facility relationships such as connection, adjacent; S24. Obtain a knowledge graph architecture according to the knowledge graph node type set and the knowledge graph relationship type set; S25. Use Neo4j to store historical triples, and set index optimization. Take the image features in the processed historical municipal engineering data as the attributes of the facility nodes, and associate the sensor rules with the facility nodes to obtain a multimodal data association graph database; S26. Set dynamic update rules, and the dynamic update rules are to use Kafka to receive new inspection data and use Flink to parse the data in real time, trigger the entity relationship extraction process to extract triples, and store and update the triples with a triple accuracy rate ≥ triple accuracy rate threshold; S27. Combine the knowledge graph architecture, the association graph database, and the dynamic update rules to obtain a dynamic municipal engineering knowledge graph; The above steps extract facility entities, attributes, and relationships from the text through a natural language processing model, filter out low - confidence triples by combining the accuracy rate threshold; define the knowledge graph node types and attributes, construct a "facility - fault - solution" relationship network; use Neo4j to store triples and associate image features with sensor rules to form a multimodal graph database; based on Kafka + Flink real - time stream processing of new inspection data, dynamically update the graph, realize the knowledge evolution of the dynamic municipal engineering knowledge graph, and finally support intelligent inspection decision - making.

[0009] Preferably, the S3 includes the following steps: S31, construct a YOLOv8 model, improve the YOLOv8 model, and obtain an improved YOLOv8 model; S32, construct a pre-trained large model, embed the YOLOv8 model into the pre-trained large model, obtain the embedded pre-trained large model, and set the network parameters of the embedded pre-trained large model; S33, collecting historical facility image data and sensor data corresponding to historical triple data in the dynamic municipal engineering knowledge graph in the dynamic municipal engineering knowledge graph to obtain historical perception data; converting the historical triple data in the dynamic municipal engineering knowledge graph into question-answer pair data; S34, setting a training accuracy threshold and a training accuracy of the network parameters of the embedded pre-trained large model, wherein the training accuracy is obtained by calculating the similarity between the answer output by the embedded pre-trained large model and the answer in the question-answer pair; Use historical perception data and questions and answers to train the embedded pre-trained large model; during the training process, combine the optimization algorithm to find the network parameters of the embedded pre-trained large model, obtain the optimal solution, and use the network parameters of the embedded pre-trained large model to obtain the municipal engineering inspection large model; The above steps improve YOLOv8; build a multimodal pre-training framework, embed the improved YOLOv8 as a visual module, and realize image-text feature alignment through cross-modal attention mechanism, such as associating image crack features with "leakage" text description; extract historical triples from the knowledge graph, such as "manhole cover-settlement-grouting repair", convert them into structured question-answer pairs, and associate the corresponding images and sensor data; set training goals, and use optimization algorithms to optimize the network parameters of the embedded pre-trained large model to output a large model for municipal engineering inspection that supports multi-task reasoning, which improves the detection accuracy and question-answer matching in dark light and occlusion scenes.

[0010] Preferably, the S31 comprises the following steps: S311. Replace the default CSPDarknet53 backbone network of YOLOv8 with ConvNeXt-Tiny; introduce dynamic snake convolution to replace some standard convolution layers; S312, embed bidirectional cross-scale attention in the neck network of YOLOv8, dynamically enhance the weight distribution of key areas through sparse global perception; replace the original CIoU loss function of YOLOv8 with EIoU Loss; S313, add dark light noise simulation, introduce CutMix-Fault enhancement strategy; Through the above steps, we can obtain a deeper receptive field and a cross-stage local attention module to enhance the ability to capture small targets (such as cracks and rust spots); improve the adaptability to irregular edge features of municipal facilities; The center point distance constraint and aspect ratio decoupling can be increased to improve the accuracy of bounding box regression; the inference speed is improved to meet the high-precision and high-efficiency requirements in complex scenarios.

[0011] In the training process of the above 34, the network parameters of the pre-trained large model after embedding are found by combining with an optimization algorithm to obtain the optimal solution, which includes the following steps: S341. Construct a wild goose population, set the scale of the wild goose population, and set the maximum number of training iterations; According to the network parameters of the pre-trained large model after embedding, randomly set the initial positions of the wild goose population to obtain the initial position set of the wild goose population; S342. Define a fitness function according to the training accuracy threshold and the training accuracy; S343. Perform iterative operations on the initial position set of the wild goose population. In each round of iteration, calculate the fitness value of each position in the initial position set of the wild goose population according to the fitness function, and update the positions of each wild goose in the initial position set of the wild goose population from high to low according to the fitness value. And in each round of iteration, obtain the best wild goose individual position and the global best wild goose position in the wild goose population; S344. Repeat S343. When the maximum number of optimization iterations is reached, stop the iteration, and use the global best wild goose position as the optimal solution; The above steps automatically search for the optimal model parameters by using the wild goose optimization algorithm; initialize and set wild goose individuals, representing different parameter combinations, and randomly generate initial positions; define a fitness function, with the answer similarity and detection accuracy weighted. In each round of iteration, calculate the fitness value of each parameter combination, sort the scores and update the wild goose positions. The wild geese move towards the high-score area, retain the best individual and the global best in each round, and output the global optimal parameters, avoiding the low efficiency problem of manual parameter tuning, improving the generalization ability in complex scenarios, and improving the accuracy of the model.

[0012] Preferably, the above S4 includes the following steps: S41. Input the real-time inspection personnel's question data, real-time facility image data, and sensor data into the municipal engineering inspection large model to obtain the municipal engineering inspection answer; S42. Set the answer accuracy threshold, take corresponding measures according to the municipal engineering inspection answer, evaluate the accuracy of the municipal engineering inspection answer, and convert the municipal engineering inspection answer with accuracy ≥ the answer accuracy threshold into a triple to obtain a real-time triple; store the real-time triple not in the dynamic municipal engineering knowledge graph into the dynamic municipal engineering knowledge graph; The above steps use multimodal input, such as the inspection staff's voice question "the reason for the abnormal manhole cover", the camera captures the crack image, and the tilt sensor alarm is input into the municipal engineering inspection large model. Combined with the knowledge graph, historical cases are matched to generate disposal suggestions; set the accuracy threshold of the answer. After the qualified plan is executed for maintenance, the verified effective triples are transmitted back to the knowledge graph in real time to form a closed-loop optimization, continuously enhancing the decision-making accuracy.

[0013] A municipal engineering inspection Q&A system based on an AI large model, which is used to implement the above-mentioned municipal engineering inspection Q&A method based on an AI large model, including a data collection and processing module, a knowledge graph construction module, a model construction and training module, and a multi-real-time application module; The data collection and processing module is used to construct a historical municipal engineering dataset through multi-source data, clean and semantically annotate unstructured text, fill in missing values and align time series for sensor data, and extract image features using an object detection model to finally obtain processed historical municipal engineering data; The knowledge graph construction module extracts facility entities and relationships from the text based on the BERT model, combines the accuracy threshold to screen high-confidence triples, defines node types and attributes, stores multi-modal associated data through Neo4j, and uses Kafka+Flink to update the graph in real time to obtain a dynamic municipal engineering knowledge graph; The model construction and training module is used to improve the YOLOv8 model; embed the improved model into a multi-modal pre-training framework, and use the Q&A pairs generated by the knowledge graph and an optimization algorithm to train the model parameters to make the answer similarity reach the threshold, and finally output a municipal engineering inspection large model that integrates visual and semantic reasoning; The real-time application module is used to receive inspection questions, real-time images and sensor data, call the knowledge graph through the model to associate historical cases, generate disposal suggestions in combination with real-time data analysis, and feedback the disposal results to the graph to achieve closed-loop optimization, forming a complete link of "perception - decision - verification".

[0014] (III) Beneficial effects The present invention has the following beneficial effects: The present invention realizes the cross-modal association of text, images and sensor data by processing historical multi-modal data and constructing a dynamic municipal knowledge graph, solves the data island problem, and improves the query efficiency. Relying on stream computing for real-time update, it ensures that the decision is based on the latest knowledge base; the improved YOLOv8 model combined with the optimization algorithm improves the recognition accuracy and the detection accuracy in complex scenarios; the closed-loop decision-making system integrates real-time perception and historical reasoning, generates an interpretable disposal plan and feedbacks the maintenance effect, has the ability to process unknown triples, reduces the repeated failure rate, and forms an intelligent and sustainable municipal facility operation and maintenance closed loop.

[0015] Through standardized cleaning, semantic annotation, and feature alignment, the present invention realizes cross-modal association of text, images, and sensor data and forms a joint index in the knowledge graph, solves the problem of data islands, and improves data query efficiency; based on the real-time triple extraction and threshold filtering mechanism of streaming computing (Kafka + Flink), the knowledge graph can dynamically incorporate new fault modes, and the update delay is reduced from the hour level to the minute level, supporting the inspection decision-making to always be based on the latest knowledge base.

[0016] The present invention obtains an improved YOLOv8 through dynamic snake-shaped convolution and the EIoU loss function, improving the recognition accuracy in the recognition tasks of municipal engineering compared with the original model; combined with the wild goose algorithm, the network parameters of the large model for municipal engineering inspection are optimized, improving the fault detection accuracy of the large model for municipal engineering inspection in complex scenarios and possessing the ability to process unknown triples.

[0017] Through real-time fusion of sensor alarms, image analysis, and knowledge reasoning, the present invention generates an interpretable disposal plan and feeds back the maintenance effect to the graph, forming a "perception - decision - verification" closed loop, reducing the repeated failure rate.

[0018] Of course, it is not necessary for any product implementing the present invention to achieve all the above-mentioned advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the invention, and those of ordinary skill in the art can also obtain other drawings according to these drawings without creative efforts.

[0020] Figure 1 It is a schematic flow chart of a method for answering questions about municipal engineering inspection based on an AI large model of the present invention; Figure 2 It is a schematic diagram of the modules of a system for answering questions about municipal engineering inspection based on an AI large model of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0021] The technical solutions in the embodiments of the invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the invention. Obviously, the described embodiments are only some embodiments of the invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the invention without creative efforts fall within the scope of protection of the invention.

[0022] In the description of the present invention, it should be understood that the terms "openings", "upper", "lower", "top", "middle", "inner", etc. indicating orientation or positional relationships are only for the convenience of describing the invention and simplifying the description, rather than indicating or implying that the components or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the invention.

[0023] Embodiment 1: Please refer to Figure 1 , the present invention discloses a method for municipal engineering inspection and answering based on an AI large model, including the following steps: S1. Collect and process historical multi-source data of municipal engineering to obtain processed historical municipal engineering data; The S1 includes the following steps: S11. Collect data through national standard documents, historical inspection reports, maintenance work orders, facility technical manuals, sensor monitoring data, facility parameter tables, facility damage photos, and inspection process videos to obtain historical multi-source data of municipal engineering; S12. Clean, segment, and semantically annotate the unstructured text in the historical multi-source data of municipal engineering, and standardize the structured data in the historical multi-source data of municipal engineering to obtain processed historical municipal engineering data; The S12 includes the following steps: S121. Denoise, segment and part-of-speech annotate, and semantically annotate the unstructured text in the historical multi-source data of municipal engineering; S122. Fill in missing values, detect and remove outliers, unify units, and align time series for the sensor data in the historical multi-source data of municipal engineering; S123. Use an image feature extraction model to perform object detection and feature extraction on the image data in the historical multi-source data of municipal engineering; S124. Obtain the processed historical municipal engineering data through S121, S122, and S123; S2. Extract facility entities from the text data in the processed historical municipal engineering data to obtain historical triple data; Construct a dynamic municipal engineering knowledge graph according to the historical triple data and dynamic update rules; The S2 includes the following steps: S11. Use a natural language processing model to automatically extract municipal facility entities (such as manhole covers, pipelines), attributes (materials, installation years), and relationships (such as "connection", "fault mode") from the text in the processed historical municipal engineering data to obtain entity data; The formula of the natural language processing model is as follows, ; Wherein, Represents the output entity data, such as "facilities (manhole covers, pipelines), faults (leaks, settlements), solutions (grouting repair)", x Represents the text data in the processed historical municipal engineering data, Soft max represents the activation function, W 、 b respectively represent the weight matrix and bias of the natural language processing model. BERT(x) represents the natural language processing model's processing of the text data x to obtain a vector containing rich semantic information of the input text; S12. Set the triple accuracy rate threshold; Use the relation extraction model to extract the initial triple data from the entity data; Combine the triple accuracy rate threshold to remove the triples with fuzzy relations to obtain the historical triple data; S13. Set the knowledge graph node type set according to the historical triple data , where, a 1, a 2 and a 3 respectively represent the facility node, the fault node, and the solution node; The attributes of the facility node include ID, type, coordinates, installation year, the attributes of the fault node include fault code, severity level, occurrence time, and the attributes of the solution node include method name, applicable scenario, cost estimate; Set the knowledge graph relation type set according to the historical triple data , where, b 1, b 2 and b 3 respectively represent facility - fault, fault - solution, and facility - facility; The facility - fault such as causes, may trigger, the fault - solution such as repair method, reference case, and the facility - facility such as connection, adjacent; S14. Obtain the knowledge graph architecture according to the knowledge graph node type set and the knowledge graph relation type set; S15. Use Neo4j to store the historical triples and set index optimization. Take the image features in the processed historical municipal engineering data as the attributes of the facility node, and associate the sensor rules with the facility node to obtain a multi - modal data associated graph database; S16. Set the dynamic update rule, and the dynamic update rule is to use Kafka to receive new inspection data and use Flink to parse the data in real - time, trigger the entity relation extraction process for triple extraction, and store and update the triples with triple accuracy rate ≥ triple accuracy rate threshold; S17. Combine the knowledge graph architecture, the associated graph database, and the dynamic update rule to obtain a dynamic municipal engineering knowledge graph; S3. Build a pre-trained large model, embed the improved YOLOv8 model into the pre-trained large model to obtain the embedded pre-trained large model; Collect historical facility image data and sensor data corresponding to historical triple data in the dynamic municipal engineering knowledge graph to obtain historical perception data; Convert the historical triple data in the dynamic municipal engineering knowledge graph into question-and-answer pair data; use the question-and-answer pair data and historical perception data in combination with an optimization algorithm to improve the pre-trained large model to obtain a municipal engineering inspection large model; The S3 includes the following steps: S31. Build a YOLOv8 model, improve the YOLOv8 model to obtain an improved YOLOv8 model; The S31 includes the following steps: S311. Replace the default CSPDarknet53 backbone network of YOLOv8 with ConvNeXt-Tiny, and introduce dynamic snake-shaped convolution to replace part of the standard convolution layer; S312. Embed bidirectional cross-scale attention in the neck network of YOLOv8, and dynamically enhance the weight distribution of key regions through sparse global perception; replace the original CIoU loss function of YOLOv8 with EIoU Loss; S313. Add low-light noise simulation and introduce the CutMix-Fault enhancement strategy; S32. Build a pre-trained large model, embed the YOLOv8 model into the pre-trained large model to obtain the embedded pre-trained large model, and set the network parameters of the embedded pre-trained large model; S33. Collect historical facility image data and sensor data corresponding to historical triple data in the dynamic municipal engineering knowledge graph to obtain historical perception data; convert the historical triple data in the dynamic municipal engineering knowledge graph into question-and-answer pair data; S34. Set the training accuracy threshold of the network parameters of the embedded pre-trained large model to c 1. The training accuracy is c 2, and the training accuracy is obtained by calculating the similarity between the answer output by the embedded pre-trained large model and the answer in the question-and-answer pair; Use the historical perception data and the pre-trained large model embedded with the question-and-answer pair for training; during the training process, combine the optimization algorithm to find the network parameters of the embedded pre-trained large model to obtain the optimal solution, and use the network parameters of the embedded pre-trained large model to obtain a municipal engineering inspection large model; The step of combining the optimization algorithm to find the network parameters of the embedded pre-trained large model to obtain the optimal solution in the 34 includes the following steps: S341. Construct a wild goose population and set the size of the wild goose population to d , then the wild goose population is represented as , where e i represents the i -th wild goose in the wild goose population; set the maximum number of training iterations; According to the network parameters of the pre-trained large model after embedding, randomly set the initial positions of the wild goose population to obtain the initial position set of the wild goose population as , where f i represents the position of the i -th wild goose in the wild goose population; S342. According to the training accuracy threshold c 1 and the training accuracy c 2, define a fitness function, and the formula of the fitness function is as follows, ; S343. Perform iterative operations on the initial position set of the wild goose population. The higher the fitness value, the better the position. In each round of iteration, according to the fitness function, calculate the fitness value of each position in the initial position set of the wild goose population, and update the positions of each wild goose in the initial position set of the wild goose population from high to low according to the fitness value. And in each round of iteration, obtain the best wild goose individual position and the global best wild goose position in the wild goose population; S344. Repeat S343. When the maximum number of optimization iterations is reached, stop the iteration and use the global best wild goose position as the optimal solution; S4. Input the real-time inspection personnel's question data, real-time facility image data, and sensor data into the municipal engineering inspection large model to obtain the municipal engineering inspection answers; The S4 includes the following steps: S41. Input the real-time inspection personnel's question data, real-time facility image data, and sensor data into the municipal engineering inspection large model to obtain the municipal engineering inspection answers; S42. Set an answer accuracy threshold, take corresponding measures according to the municipal engineering inspection answers, evaluate the accuracy of the municipal engineering inspection answers, and convert the municipal engineering inspection answers with accuracy ≥ answer accuracy threshold into triples to obtain real-time triples; Store the real-time triples not in the dynamic municipal engineering knowledge graph into the dynamic municipal engineering knowledge graph.

[0024] Embodiment 2: Please refer to Figure 2, a municipal engineering inspection Q&A system based on an AI large model, which is used to implement the above-mentioned municipal engineering inspection Q&A method based on an AI large model, including a data collection and processing module, a knowledge graph construction module, a model construction and training module, and a multi-real-time application module; The data collection and processing module is used to construct a historical municipal engineering dataset through multi-source data, clean and semantically annotate unstructured text, fill in missing values and align time series for sensor data, and extract image features using an object detection model, and finally obtain processed historical municipal engineering data; The knowledge graph construction module extracts facility entities and relationships from the text based on the BERT model, combines the accuracy threshold to screen high-confidence triples, defines node types and attributes, stores multi-modal associated data through Neo4j, and uses Kafka+Flink to update the graph in real time to obtain a dynamic municipal engineering knowledge graph; The model construction and training module is used to improve the YOLOv8 model; embed the improved model into a multi-modal pre-training framework, use the Q&A pairs generated by the knowledge graph and an optimization algorithm to train the model parameters to make the answer similarity reach the threshold, and finally output a municipal engineering inspection large model that integrates visual and semantic reasoning; The real-time application module is used to receive inspection questions, real-time images and sensor data, call the knowledge graph through the model to associate historical cases, generate disposal suggestions by combining real-time data analysis, and feedback the disposal results to the graph to achieve closed-loop optimization, forming a complete link of "perception - decision - verification".

[0025] In the description of this specification, the descriptions referring to terms such as "one embodiment", "example", "specific example", etc. mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0026] The preferred embodiments of the invention disclosed above are only used to help explain the invention. The preferred embodiments do not elaborate on all the details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principle and practical application of the invention, so that those skilled in the art in the relevant technical field can well understand and utilize the invention.

Claims

1. A municipal engineering inspection question-answering method based on an AI big model, characterized in that: The following steps are involved: S1. Collect and process multi-source data of historical municipal engineering projects to obtain processed historical municipal engineering project data; S2, extracting facility entities from the text data in the processed historical municipal engineering data to obtain historical triple data; Constructing a dynamic municipal engineering knowledge graph based on the historical triple data and dynamic update rules; S3. Build a pre-trained large model, embed the improved YOLOv8 model into the pre-trained large model, and obtain the embedded pre-trained large model; Collect historical facility image data and sensor data corresponding to historical triple data in the dynamic municipal engineering knowledge graph to obtain historical perception data; The historical triple data in the dynamic municipal engineering knowledge graph is converted into question-answer pair data; the pre-trained large model is improved by using the question-answer pair data and historical perception data combined with the optimization algorithm to obtain the municipal engineering inspection large model; S4. Input the real-time inspection personnel’s question data, real-time facility image data and sensor data into the municipal engineering inspection model to obtain the municipal engineering inspection answers.

2. According to the municipal engineering inspection question-answering method based on AI big model according to claim 1, it is characterized in that: The S1 comprises the following steps: S11. Collect data through national standard documents, historical inspection reports, maintenance work orders, facility technical manuals, sensor monitoring data, facility parameter tables, facility damage photos, and inspection process videos to obtain historical municipal engineering multi-source data; S12. Clean, segment and semantically annotate the unstructured text in the multi-source data of historical municipal engineering, and standardize the structured data in the multi-source data of historical municipal engineering to obtain the processed historical municipal engineering data.

3. According to claim 2, a municipal engineering inspection question-answering method based on an AI large model is characterized in that: The S12 comprises the following steps: S121. De-noising, word segmentation, part-of-speech tagging and semantic tagging of unstructured text in multi-source data of historical municipal engineering projects; S122, performing missing value filling, outlier detection and removal, unit unification, and time series alignment on sensor data in multi-source data of historical municipal engineering projects; S123, using an image feature extraction model to perform target detection and feature extraction on image data in multi-source data of historical municipal engineering projects; S124. Obtain processed historical municipal engineering data through S121, S122 and S123.

4. According to claim 1, a municipal engineering inspection question-answering method based on an AI big model is characterized in that: The S2 comprises the following steps: S21, using a natural language processing model to automatically extract municipal facility entities, attributes and relationships from the text in the processed historical municipal engineering data to obtain entity data; S22, setting a triple accuracy threshold; using a relation extraction model to extract initial triple data from entity data; removing triples with ambiguous relationships in combination with the triple accuracy threshold to obtain historical triple data; S23, setting a knowledge graph node type set according to the historical triple data; the knowledge graph node type set includes facility nodes, fault nodes and solution nodes; Setting a knowledge graph relationship type set according to the historical triple data, the knowledge graph relationship type set including facility-fault, fault-solution and facility-facility; S24, obtaining a knowledge graph architecture according to the knowledge graph node type set and the knowledge graph relationship type set; S25. Use Neo4j to store historical triples and set index optimization, use the image features in the processed historical municipal engineering data as attributes of the facility nodes, associate sensor rules with the facility nodes, and obtain a multimodal data association graph database; S26. Setting dynamic update rules, wherein the dynamic update rules are: using Kafka to receive new inspection data and using Flink to parse the data in real time, triggering the entity relationship extraction process to extract triples, and storing and updating triples whose triple accuracy is ≥ triple accuracy threshold; S27. Combine the knowledge graph architecture, the association graph database and the dynamic update rules to obtain a dynamic municipal engineering knowledge graph.

5. According to the municipal engineering inspection question-answering method based on AI big model in claim 1, it is characterized in that: The S3 comprises the following steps: S31, construct a YOLOv8 model, improve the YOLOv8 model, and obtain an improved YOLOv8 model; S32, construct a pre-trained large model, embed the YOLOv8 model into the pre-trained large model, obtain the embedded pre-trained large model, and set the network parameters of the embedded pre-trained large model; S33, collecting historical facility image data and sensor data corresponding to historical triple data in the dynamic municipal engineering knowledge graph in the dynamic municipal engineering knowledge graph to obtain historical perception data; converting the historical triple data in the dynamic municipal engineering knowledge graph into question-answer pair data; S34, setting a training accuracy threshold and a training accuracy of the network parameters of the embedded pre-trained large model, wherein the training accuracy is obtained by calculating the similarity between the answer output by the embedded pre-trained large model and the answer in the question-answer pair; The embedded pre-trained large model is trained using historical perception data and questions and answers. During the training process, the optimization algorithm is combined to find the network parameters of the embedded pre-trained large model to obtain the optimal solution. The network parameters of the embedded pre-trained large model are used to obtain the municipal engineering inspection large model.

6. According to claim 5, a municipal engineering inspection question-answering method based on an AI big model is characterized in that: The S31 comprises the following steps: S311. Replace the default CSPDarknet53 backbone network of YOLOv8 with ConvNeXt-Tiny; introduce dynamic snake convolution to replace some standard convolution layers; S312, embed bidirectional cross-scale attention in the neck network of YOLOv8, dynamically enhance the weight distribution of key areas through sparse global perception; replace the original CIoU loss function of YOLOv8 with EIoU Loss; S313, add dark light noise simulation and introduce CutMix-Fault enhancement strategy.

7. According to claim 1, a municipal engineering inspection question-answering method based on an AI big model is characterized in that: In the training process 34, the optimization algorithm is combined to find the network parameters of the embedded pre-trained large model, and the optimal solution includes the following steps: S341. Construct a wild goose population, set the size of the wild goose population, and set the maximum number of training iterations; According to the network parameters of the embedded pre-trained large model, the initial positions of the wild goose population are randomly set to obtain an initial position set of the wild goose population; S342, defining a fitness function according to the training accuracy threshold and the training accuracy; S343, performing an iterative operation on the initial position set of the wild goose population, calculating the fitness value of each position in the initial position set of the wild goose population according to the fitness function in each round of iteration, updating the position of each goose in the initial position set of the wild goose population according to the fitness value from high to low, and obtaining the best individual goose position in the wild goose population and the global best goose position in each round of iteration; S344, repeat S343, when the maximum number of optimization iterations is reached, stop the iteration, and take the global best goose position as the optimal solution.

8. According to claim 1, a municipal engineering inspection question-answering method based on an AI big model is characterized in that: The S4 comprises the following steps: S41, inputting the real-time inspection personnel question data, real-time facility image data and sensor data into the municipal engineering inspection large model to obtain the municipal engineering inspection answer; S42. Set an answer accuracy threshold, take corresponding measures according to the municipal engineering inspection answer, evaluate the accuracy of the municipal engineering inspection answer, and convert the municipal engineering inspection answer with an accuracy ≥ the answer accuracy into triples to obtain real-time triples; store the real-time triples that are not in the dynamic municipal engineering knowledge graph into the dynamic municipal engineering knowledge graph.

9. A safety and environmental protection system based on multi-pollutant coordinated governance, characterized in that: Implementing a municipal engineering inspection question-answering method based on an AI big model as described in any one of claims 1-8, the system comprising a data collection and processing module, a knowledge graph construction module, a model construction and training module, and a multi-real-time application module; The data collection and processing module is used to construct a historical municipal engineering data set through multi-source data, clean and semantically annotate unstructured text, fill missing values ​​and align time series of sensor data, extract image features using a target detection model, and finally obtain processed historical municipal engineering data; The knowledge graph construction module extracts facility entities and relationships from text based on the BERT model, filters high-confidence triples based on the accuracy threshold, defines node types and attributes, stores multimodal related data through Neo4j, and uses Kafka+Flink to update the graph in real time to obtain a dynamic municipal engineering knowledge graph; The model building and training module is used to improve the YOLOv8 model; the improved model is embedded in the multimodal pre-training framework, the question-answer pairs generated by the knowledge graph and the optimization algorithm are used to train the model parameters so that the answer similarity reaches the threshold, and finally a large municipal engineering inspection model that integrates visual and semantic reasoning is output; The real-time application module is used to receive inspection questions, real-time images and sensor data, call the knowledge graph through the model to associate historical cases, generate disposal suggestions in combination with real-time data analysis, and feed back the disposal results to the graph to achieve closed-loop optimization, forming a complete "perception-decision-verification" link.

10. A storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, a municipal engineering inspection question-and-answer method based on an AI large model as described in any one of claims 1-8 is implemented.

Images