Crack diagnosis reasoning method and system based on engineering inspection agent

Abstract

The invention provides a crack diagnosis reasoning method and system based on an engineering inspection agent in the technical field of engineering detection and artificial intelligence crossing. The method comprises the steps that S1, a mobile terminal collects multi-modal data and uploads the multi-modal data to a server; s2, the server inputs the multi-modal data into a crack recognition model to obtain crack masks and geometric features, physical scale parameters of cracks are calculated in combination with scale calibration information, and crack fact data are generated; s3, taking the crack fact data as input, performing two-stage retrieval in the engineering knowledge graph to construct an evidence sub-graph, and performing coding and reasoning on the evidence sub-graph by using a graph neural network to generate a sub-graph context vector; and S4, the crack fact data and the sub-graph context vectors are fused to obtain fusion features, the fusion features are input into the large language model, and a diagnosis conclusion is generated. The method has the advantages that the precision, the efficiency, the interpretability, the environmental adaptability and the generalization ability of crack diagnosis are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of engineering inspection and artificial intelligence, and in particular to a crack diagnosis reasoning method and system based on an engineering inspection intelligent agent. Background Technology

[0002] In civil engineering and building construction, safety inspections and condition assessments of engineering structures (such as buildings, bridges, tunnels, and slopes) are crucial for ensuring their safe operation. Among these, cracks, as one of the most common forms of structural defects, require accurate identification, quantitative measurement, and causal diagnosis as core tasks in assessing structural health and safety risks.

[0003] Currently, traditional crack detection methods mainly rely on manual inspection. Inspectors visually observe cracks and use tools such as steel rulers, crack width comparison cards, and crack width gauges to locate, record, and measure their width. This method has several significant drawbacks: 1. Highly subjective and with unstable accuracy: The test results are highly dependent on the personal experience and subjective judgment of the testers, and are easily affected by environmental factors such as lighting conditions, shooting angle, and on-site obstruction, resulting in poor repeatability and weak comparability of the measurement data.

[0004] 2. Low efficiency and difficult to apply on a large scale: The manual point-by-point measurement and recording method is time-consuming and labor-intensive. In large-scale structures or complex construction sites, it is difficult to achieve rapid, large-scale comprehensive detection and continuous tracking.

[0005] 3. Limited information dimensions and lack of engineering semantics: Traditional methods typically only record visual information and simple geometric dimensions of cracks, failing to effectively correlate the type of structural component (such as beams, columns, slabs, and walls), material properties (such as concrete, masonry, and wood), and the overall structural system of the building. This leaves diagnostic conclusions superficial, making it difficult to conduct in-depth, interpretable, engineering-level reasoning regarding the potential causes, development trends, and true impact of cracks on structural safety.

[0006] With the development of computer vision technology, some automated crack detection methods based on deep learning have emerged. These methods typically utilize image recognition models deployed on servers or mobile devices to automatically identify cracks in images. However, these methods still have significant limitations: 1. The contradiction between edge computing power and cloud collaboration: Mobile terminal devices have limited computing resources, making it difficult to run high-precision complex models locally, especially to process multimodal data (such as images, IMU attitude, environmental parameters, etc.). If all data is uploaded to the cloud for processing, there is a lack of adaptive transmission mechanisms for complex field network environments such as weak or no network, which can easily lead to data transmission interruptions or delays, affecting the real-time performance and reliability of detection.

[0007] 2. "Identification" rather than "Diagnosis": Most existing AI models focus on identifying and segmenting cracks at the pixel level, which is essentially "visual perception" rather than "engineering diagnosis." The models lack the support of engineering domain knowledge (such as design specifications, disease mechanisms, and historical cases), and cannot understand the engineering semantics behind the cracks. Therefore, they cannot answer key engineering questions such as "Why do cracks occur?", "What is the risk level of the cracks?", and "What measures should be taken?".

[0008] 3. Model is fixed and difficult to evolve: Once deployed, most systems have fixed recognition models and knowledge bases, which cannot effectively utilize the new data constantly generated on site (especially manual review results and difficult samples) for self-optimization and iteration. They are difficult to adapt to the emergence of new materials, new structural forms or new disease patterns, and the system's generalization ability and long-term effectiveness are insufficient.

[0009] Therefore, how to provide a crack diagnosis reasoning method and system based on engineering inspection intelligent agents to improve the accuracy, efficiency, interpretability, environmental adaptability and generalization ability of crack diagnosis has become an urgent technical problem to be solved. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to provide a crack diagnosis reasoning method and system based on an engineering inspection intelligent agent, so as to improve the accuracy, efficiency, interpretability, environmental adaptability and generalization ability of crack diagnosis.

[0011] In a first aspect, the present invention provides a crack diagnosis reasoning method based on an engineering inspection intelligent agent, comprising the following steps: Step S1: Collect multimodal data including crack images, environmental parameters, scale calibration information and engineering semantic priors through a mobile terminal. During the crack image acquisition process, a lightweight assistance model deployed on the mobile terminal is used to guide the shooting posture, and the lightweight assistance model is used to preprocess the acquired crack images. Step S2: The mobile terminal uploads the multimodal data to the server based on the preset upload priority and the breakpoint resume mechanism. Step S3: The server receives each of the multimodal data in real time, inputs each of the multimodal data into the pre-deployed crack recognition model to obtain pixel-level crack masks and geometric features, and calculates the physical scale parameters of the cracks in combination with the scale calibration information to generate structured crack fact data. Step S4: The server takes the crack fact data as input, performs a two-stage retrieval in the pre-created engineering knowledge graph to construct an evidence subgraph, and uses a graph neural network to encode and reason about the evidence subgraph to generate a subgraph context vector. Step S5: The server fuses the crack fact data with the subgraph context vector to obtain fused features, and inputs the fused features into a large language model fine-tuned by engineering domain knowledge. The large language model performs deep engineering semantic reasoning to generate a diagnostic conclusion that includes crack nature, impact degree, treatment suggestions and structured evidence chain. Step S6: The server sends the diagnostic conclusion back to the mobile terminal for visualization and obtains the diagnostic feedback sent by the mobile terminal. Step S7: Based on the diagnostic feedback, the server iteratively optimizes the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model.

[0012] Furthermore, in step S1, the lightweight assist model includes a super-resolution sub-model based on depthwise separable convolution, a deblurring sub-model based on a cropped U-Net structure, and a viewpoint guidance sub-model based on IMU and vision coupling. The super-resolution sub-model is used to improve the resolution of the crack image and reconstruct texture details; the deblurring sub-model is used to perform deconvolution processing on motion-blurred regions in the crack image; the viewpoint guidance sub-model is used to detect shooting viewpoint deviations of the mobile terminal and prompt the user to correct them. The lightweight assistance model is processed through model compression and acceleration technology to achieve real-time operation on the CPU / GPU of mobile terminals; The environmental parameters include at least GPS coordinates, IMU attitude, light intensity, and exposure parameters; the engineering semantic priors include at least building type, structural system, component category, and location information. Step S2 specifically involves: The mobile terminal performs hash calculation on each of the multimodal data using the SHA-256 algorithm to obtain a hash value, and then stores the hash value on the blockchain. The mobile terminal classifies the multimodal data according to a preset upload priority, encrypts the classified multimodal data using the SM4 algorithm, and then uploads it to the server in segments based on the CoAP Block-wise protocol, incorporating a breakpoint resume mechanism during the upload process.

[0013] Furthermore, step S3 specifically includes: The server receives the multimodal data in real time, decrypts the multimodal data using the SM4 algorithm, verifies the integrity of the multimodal data using the hash value stored in the blockchain, inputs the crack images, environmental parameters, and engineering semantic priors from the multimodal data into a pre-deployed crack recognition model to obtain pixel-level crack masks and geometric features, and combines the scale calibration information to calculate the physical scale parameters of the crack, including crack length, crack width, and crack direction, using the back projection method to generate structured crack fact data. The crack recognition model is constructed based on a cross-modal encoder, a feature fusion module, a temporal fusion module, a multi-branch decoder, and a cross-attention module. The cross-modal encoder encodes the input crack image, environmental parameters, and engineering semantic priors to obtain visual feature vectors, numerical feature vectors, and semantic feature vectors. The feature fusion module fuses the visual feature vectors, numerical feature vectors, and semantic feature vectors through a cross-attention mechanism to obtain a high-dimensional feature tensor. The temporal fusion module aligns and weights the spatial features in the continuous high-dimensional feature tensor through a temporal convolutional network to obtain a spatiotemporal feature tensor. The multi-branch decoder is constructed based on a semantic segmentation branch, a skeleton extraction branch, and a width regression branch. The semantic segmentation branch infers the spatiotemporal feature tensor through U-Net to obtain a crack mask. The skeleton extraction branch infers the spatiotemporal feature tensor through distance transform regression to obtain a crack centerline. The width regression branch infers the spatiotemporal feature tensor through a fully convolutional network to obtain a pixel-level width map. The cross-attention module mutually reinforces the crack mask, crack centerline, and pixel-level width map, outputting a crack mask and geometric features. The geometric features include the crack centerline and the pixel-level width map.

[0014] Furthermore, in step S4, the two-stage retrieval in the pre-created engineering knowledge graph to construct the evidence subgraph specifically involves: Based on the BM25 algorithm, the text field of the crack fact data is retrieved from the node attributes of a pre-created engineering knowledge graph to obtain the first-stage retrieval results; the numerical feature vector of the crack fact data is retrieved from the vector embedding representation of the engineering knowledge graph based on vector similarity to obtain the second-stage retrieval results; the first-stage retrieval results and the second-stage retrieval results are merged to locate key evidence nodes, and then an evidence subgraph related to the crack fact data is constructed. The evidence subgraph is constructed using a k-hop expansion strategy, and the edges in the evidence subgraph are assigned time decay weights. The graph neural network uses graph attention networks or GraphSAGE, and introduces an evidence consistency loss function during training to improve the interpretability of reasoning.

[0015] Furthermore, in step S5, the reasoning mechanism of the large language model is as follows: Engineering domain knowledge is converted into semantic fragments, which serve as the context of the large language model. Using a pre-set reasoning template, the large language model is guided to perform engineering semantic reasoning in the logical order of "evidence-fact-goal" to obtain the nature of the crack, the degree of impact, and the treatment suggestions. It also actively references the relevant engineering domain knowledge as a structured evidence chain and outputs a diagnostic conclusion that includes the nature of the crack, the degree of impact, the treatment suggestions, and the structured evidence chain. In step S7, the iterative optimization of the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model specifically involves: The crack identification model is fine-tuned using mini-batch incremental tuning, and the large language model is fine-tuned using low-rank adaptation techniques. A domain-dependent strategy is used to locate graph subgraphs affected by new samples or new knowledge, and the vector embedding representation of the engineering knowledge graph is locally updated. After the iteratively optimized crack identification model, engineering knowledge graph, and large language model are verified offline in a sandbox environment, they are gradually deployed online through A / B testing or Canary release mechanisms, while retaining version control and rollback capabilities.

[0016] Secondly, the present invention provides a crack diagnosis reasoning system based on an engineering inspection intelligent agent, comprising the following modules: The multimodal data acquisition module is used to acquire multimodal data including crack images, environmental parameters, scale calibration information and engineering semantic priors through a mobile terminal. During the crack image acquisition process, a lightweight assistance model deployed on the mobile terminal is used to guide the shooting posture, and the lightweight assistance model is used to preprocess the acquired crack images. The multimodal data upload module is used by the mobile terminal to upload the multimodal data to the server by combining the preset upload priority and the breakpoint resume mechanism. The crack fact data generation module is used to receive the multimodal data in real time from the server, input the multimodal data into a pre-deployed crack identification model to obtain pixel-level crack masks and geometric features, and calculate the physical scale parameters of the cracks in combination with the scale calibration information to generate structured crack fact data. The knowledge graph retrieval module is used by the server to perform a two-stage retrieval in a pre-created engineering knowledge graph to construct an evidence subgraph, and to use a graph neural network to encode and reason about the evidence subgraph to generate a subgraph context vector. The diagnostic conclusion generation module is used by the server to fuse the crack fact data with the subgraph context vector to obtain fused features, input the fused features into a large language model fine-tuned with engineering domain knowledge, and perform deep engineering semantic reasoning through the large language model to generate a diagnostic conclusion that includes the nature of the crack, the degree of impact, treatment suggestions, and a structured chain of evidence. The diagnostic conclusion feedback and display module is used by the server to send the diagnostic conclusion back to the mobile terminal for visualization display and to obtain the diagnostic feedback sent by the mobile terminal. The iterative optimization module is used by the server to iteratively optimize the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model based on the diagnostic feedback.

[0017] Furthermore, in the multimodal data acquisition module, the lightweight assistance model includes a super-resolution sub-model based on depthwise separable convolution, a deblurring sub-model based on a cropped U-Net structure, and a viewpoint guidance sub-model based on IMU and vision coupling. The super-resolution sub-model is used to improve the resolution of the crack image and reconstruct texture details; the deblurring sub-model is used to perform deconvolution processing on motion-blurred regions in the crack image; the viewpoint guidance sub-model is used to detect shooting viewpoint deviations of the mobile terminal and prompt the user to correct them. The lightweight assistance model is processed through model compression and acceleration technology to achieve real-time operation on the CPU / GPU of mobile terminals; The environmental parameters include at least GPS coordinates, IMU attitude, light intensity, and exposure parameters; the engineering semantic priors include at least building type, structural system, component category, and location information. The multimodal data upload module is specifically used for: The mobile terminal performs hash calculation on each of the multimodal data using the SHA-256 algorithm to obtain a hash value, and then stores the hash value on the blockchain. The mobile terminal classifies the multimodal data according to a preset upload priority, encrypts the classified multimodal data using the SM4 algorithm, and then uploads it to the server in segments based on the CoAP Block-wise protocol, incorporating a breakpoint resume mechanism during the upload process.

[0018] Furthermore, the crack fact data generation module is specifically used for: The server receives the multimodal data in real time, decrypts the multimodal data using the SM4 algorithm, verifies the integrity of the multimodal data using the hash value stored in the blockchain, inputs the crack images, environmental parameters, and engineering semantic priors from the multimodal data into a pre-deployed crack recognition model to obtain pixel-level crack masks and geometric features, and combines the scale calibration information to calculate the physical scale parameters of the crack, including crack length, crack width, and crack direction, using the back projection method to generate structured crack fact data. The crack recognition model is constructed based on a cross-modal encoder, a feature fusion module, a temporal fusion module, a multi-branch decoder, and a cross-attention module. The cross-modal encoder encodes the input crack image, environmental parameters, and engineering semantic priors to obtain visual feature vectors, numerical feature vectors, and semantic feature vectors. The feature fusion module fuses the visual feature vectors, numerical feature vectors, and semantic feature vectors through a cross-attention mechanism to obtain a high-dimensional feature tensor. The temporal fusion module aligns and weights the spatial features in the continuous high-dimensional feature tensor through a temporal convolutional network to obtain a spatiotemporal feature tensor. The multi-branch decoder is constructed based on a semantic segmentation branch, a skeleton extraction branch, and a width regression branch. The semantic segmentation branch infers the spatiotemporal feature tensor through U-Net to obtain a crack mask. The skeleton extraction branch infers the spatiotemporal feature tensor through distance transform regression to obtain a crack centerline. The width regression branch infers the spatiotemporal feature tensor through a fully convolutional network to obtain a pixel-level width map. The cross-attention module mutually reinforces the crack mask, crack centerline, and pixel-level width map, outputting a crack mask and geometric features. The geometric features include the crack centerline and the pixel-level width map.

[0019] Furthermore, in the knowledge graph retrieval module, the two-stage retrieval within the pre-created engineering knowledge graph to construct the evidence subgraph specifically involves: Based on the BM25 algorithm, the text field of the crack fact data is retrieved from the node attributes of a pre-created engineering knowledge graph to obtain the first-stage retrieval results; the numerical feature vector of the crack fact data is retrieved from the vector embedding representation of the engineering knowledge graph based on vector similarity to obtain the second-stage retrieval results; the first-stage retrieval results and the second-stage retrieval results are merged to locate key evidence nodes, and then an evidence subgraph related to the crack fact data is constructed. The evidence subgraph is constructed using a k-hop expansion strategy, and the edges in the evidence subgraph are assigned time decay weights. The graph neural network uses graph attention networks or GraphSAGE, and introduces an evidence consistency loss function during training to improve the interpretability of reasoning.

[0020] Furthermore, in the diagnostic conclusion generation module, the reasoning mechanism of the large language model is as follows: Engineering domain knowledge is converted into semantic fragments, which serve as the context of the large language model. Using a pre-set reasoning template, the large language model is guided to perform engineering semantic reasoning in the logical order of "evidence-fact-goal" to obtain the nature of the crack, the degree of impact, and the treatment suggestions. It also actively references the relevant engineering domain knowledge as a structured evidence chain and outputs a diagnostic conclusion that includes the nature of the crack, the degree of impact, the treatment suggestions, and the structured evidence chain. In the iterative optimization module, the iterative optimization of the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model specifically involves: The crack identification model is fine-tuned using mini-batch incremental tuning, and the large language model is fine-tuned using low-rank adaptation techniques. A domain-dependent strategy is used to locate graph subgraphs affected by new samples or new knowledge, and the vector embedding representation of the engineering knowledge graph is locally updated. After the iteratively optimized crack identification model, engineering knowledge graph, and large language model are verified offline in a sandbox environment, they are gradually deployed online through A / B testing or Canary release mechanisms, while retaining version control and rollback capabilities.

[0021] The advantages of this invention are: 1. Multimodal data, including crack images, environmental parameters, scale calibration information, and engineering semantic priors, is collected via mobile terminals. During crack image acquisition, a lightweight assistive model guides the shooting posture and preprocesses the acquired crack images. Combined with preset upload priorities and a breakpoint resume mechanism, the multimodal data is uploaded to the server. The server inputs the multimodal data into a crack recognition model to obtain pixel-level crack masks and geometric features, and calculates the physical scale parameters of the cracks based on scale calibration information, generating structured crack factual data. Using the crack factual data as input, the server performs a two-stage retrieval in the engineering knowledge graph to construct an evidence subgraph. A graph neural network is then used to encode and infer the evidence subgraph to generate a subgraph context vector. The crack factual data and the subgraph context vector are fused to obtain fused features. These fused features are input into a large language model fine-tuned with engineering domain knowledge to generate diagnostic conclusions, which are then transmitted back to the mobile terminal for visualization. The system demonstrates and acquires diagnostic feedback from mobile terminals. Based on this feedback, the server iteratively optimizes the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model. Specifically, it integrates a lightweight assistance model with the mobile terminal to guide shooting and perform image preprocessing, combining multimodal data acquisition and reliable transmission mechanisms to improve data quality and acquisition efficiency from the source. On the server, the crack identification model, which integrates cross-modal information, achieves pixel-level precise quantization. By retrieving the engineering knowledge graph and using graph neural network inference, it associates crack factual data with engineering domain knowledge, providing evidence support for the deep diagnosis of the large language model and generating interpretable diagnostic conclusions containing structured evidence chains. Finally, based on user feedback, the crack identification model, engineering knowledge graph, and large language model are continuously iteratively optimized to adapt to complex field environments and continuously improve their generalization ability to new scenarios, ultimately greatly improving the accuracy, efficiency, interpretability, environmental adaptability, and generalization ability of crack diagnosis.

[0022] 2. By innovatively integrating multimodal data such as crack images, environmental parameters, scale calibration information, and engineering semantic priors during the data acquisition phase, the richness and contextual relevance of the input information are ensured. Through lightweight assistance models deployed on mobile terminals (such as super-resolution, deblurring, and viewpoint guidance sub-models), real-time guidance of shooting posture and image preprocessing are achieved, effectively improving the quality and efficiency of on-site data acquisition. This design solves the problems of reliance on manual experience, single data, and susceptibility to environmental interference in traditional engineering inspection, providing a reliable data foundation for subsequent accurate diagnosis and enhancing the robustness and practicality of the system.

[0023] 3. By using preset upload priorities and breakpoint resume mechanisms, combined with SM4 encryption algorithm and CoAP Block-wise protocol for segmented uploads, reliable data transmission is ensured in unstable network environments. At the same time, the SHA-256 hash algorithm is used to store data on the blockchain, realizing the immutability and integrity verification of the data. These technical means significantly improve the applicability of the system in engineering sites (such as remote areas or complex network conditions), reduce the risk of data loss or leakage, and meet the high standards of data security and real-time performance required by modern engineering inspection.

[0024] 4. The server adopts a crack recognition model based on a cross-modal encoder, feature fusion module, and multi-branch decoder, which can collaboratively process visual, numerical, and semantic features to generate pixel-level crack masks and geometric features (such as centerline and width maps). Combined with the back projection method to calculate physical scale parameters, it realizes automated and high-precision quantification of crack length, width, and other parameters, avoiding the errors caused by relying on a single modality in traditional image processing. This multi-source fusion and spatiotemporal feature alignment design improves the accuracy and robustness of crack detection, and is especially suitable for the identification of fine cracks in complex engineering structures.

[0025] 5. By performing two-stage retrieval (text recall based on BM25 and vector recall based on vector similarity) in a pre-created engineering knowledge graph, and combining it with a k-hop expansion strategy to construct an evidence subgraph, efficient and comprehensive evidence mining is achieved. Graph neural networks (such as graph attention networks) are used to encode and reason about the evidence subgraph, and an evidence consistency loss function is introduced to enhance the logic and interpretability of the reasoning process. This design enables the system to quickly locate relevant cases and rules from massive amounts of engineering knowledge, providing structured evidence support for diagnosis and improving the scientific nature and transparency of decision-making.

[0026] 6. After fusing the factual data of cracks with the context vector of the subgraph, the data is input into a large language model finely tuned with engineering domain knowledge. Using a preset "evidence-fact-target" reasoning template, deep-level engineering semantic reasoning is achieved. This model can automatically generate diagnostic conclusions that include the nature of cracks, degree of impact, treatment suggestions, and a structured chain of evidence. It not only provides professional judgment but also outputs explanations and suggestions in a human-like manner. This solves the problem of traditional expert systems relying on hard-coded rules and lacking flexibility, enabling the system to have human-like reasoning capabilities and significantly improving user experience and diagnostic practicality.

[0027] 7. Based on diagnostic feedback sent from mobile terminals, the crack identification model, the vector embedding representation of the engineering knowledge graph, and the large language model are iteratively optimized, such as by using small-batch incremental fine-tuning, low-rank adaptation technology, and local graph updates. Through mechanisms such as sandbox environment verification and A / B testing, the system is gradually deployed to ensure its stability and adaptability when introducing new data or knowledge. This closed-loop optimization design enables the system to continuously learn new cases in engineering practice, continuously improve diagnostic accuracy, achieve long-term effectiveness and scalability, and reduce maintenance costs.

[0028] 8. From data collection, uploading, processing, inference to feedback optimization, a complete end-to-end automated closed-loop process is formed; the modules work closely together, such as the mobile terminal being responsible for lightweight preprocessing and interaction, the server performing heavy computation and inference, and the final results being visualized and feedback collected; this integrated design realizes the potential for full-process intelligentization and unattended operation of engineering inspection, greatly improving diagnostic efficiency (such as reducing manual inspection time) and consistency, which is in line with the development trend of modern intelligent manufacturing and infrastructure maintenance.

[0029] 9. The lightweight assist model achieves real-time operation on mobile terminal CPUs / GPUs through model compression and acceleration technologies, reducing hardware dependence and energy consumption; the server side uses efficient algorithms (such as temporal convolutional networks and graph neural networks) for parallel processing, supporting high-concurrency data streams; these optimizations ensure that the system can be stably deployed in resource-constrained engineering sites (such as handheld devices) and the cloud, balancing performance and cost. Attached Figure Description

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0031] Figure 1 This is a flowchart of a crack diagnosis reasoning method based on an engineering inspection intelligent agent according to the present invention.

[0032] Figure 2 This is a schematic diagram of the structure of a crack diagnosis reasoning system based on an engineering inspection intelligent agent according to the present invention. Detailed Implementation

[0033] The overall approach of the technical solution in this application is as follows: A lightweight assistance model is integrated into the mobile terminal to achieve shooting guidance and image preprocessing. Combined with multimodal data acquisition and reliable transmission mechanisms, data quality and acquisition efficiency are improved from the source. On the server, a crack identification model that integrates cross-modal information achieves pixel-level precise quantization. Furthermore, by retrieving engineering knowledge graphs and using graph neural network inference, crack factual data is associated with engineering domain knowledge, providing evidence support for deep diagnosis using a large language model and generating interpretable diagnostic conclusions containing structured evidence chains. Finally, based on user feedback, the crack identification model, engineering knowledge graph, and large language model are continuously iterated and optimized to adapt to complex field environments and continuously improve their generalization ability to new scenarios, thereby enhancing the accuracy, efficiency, interpretability, environmental adaptability, and generalization ability of crack diagnosis.

[0034] Please refer to Figures 1 to 2 As shown, a preferred embodiment of the crack diagnosis reasoning method based on an engineering inspection intelligent agent of the present invention includes the following steps: Step S1: Collect multimodal data including crack images, environmental parameters, scale calibration information and engineering semantic priors through a mobile terminal. During the crack image acquisition process, a lightweight assistance model deployed on the mobile terminal is used to guide the shooting posture, and the lightweight assistance model is used to preprocess the acquired crack images. The lightweight assistance model is used to enhance and control the image quality of crack images in real time, so as to solve the problem of image blurring caused by shaking, insufficient lighting or angular deviation at the engineering site. In practice, the mobile terminal also performs distortion correction and perspective correction on the crack image. If the mobile terminal can provide camera intrinsic parameters, these parameters are used first for accurate distortion correction. If not, multi-frame self-calibration combined with reference object detection (such as a ruler, brick seam, or standard paper) is used to fit the homography matrix to recover the local planar viewpoint, outputting a pixel-to-physical scale mapping (pixel-per-mm) and the corresponding scale confidence. The resulting scale mapping and camera pose are uploaded as metadata along with the thumbnail and low-dimensional features. Upon receiving the metadata, the server immediately uses it to perform physical scale association on the crack image, providing a benchmark for subsequent pixel-to-millimeter conversion.

[0035] Step S2: The mobile terminal uploads the multimodal data to the server based on the preset upload priority and the breakpoint resume mechanism. Step S3: The server receives each of the multimodal data in real time, inputs each of the multimodal data into the pre-deployed crack recognition model to obtain pixel-level crack masks and geometric features, and calculates the physical scale parameters of the cracks in combination with the scale calibration information to generate structured crack fact data. Step S4: The server takes the crack fact data as input, performs a two-stage retrieval in the pre-created engineering knowledge graph to construct an evidence subgraph, and uses a graph neural network to encode and reason about the evidence subgraph to generate a subgraph context vector. Step S5: The server fuses the crack fact data with the subgraph context vector to obtain fused features, and inputs the fused features into a large language model fine-tuned by engineering domain knowledge. The large language model performs deep engineering semantic reasoning to generate a diagnostic conclusion that includes crack nature, impact degree, treatment suggestions and structured evidence chain. The server organizes the output of each detection into a standardized data structure for subsequent engineering knowledge graph retrieval and inference. This data structure includes: a unique crack identifier (crack_id), timestamp and session ID, centerline point set (a sequence of three-dimensional coordinate arrays), length_mm, width_max_mm, width_avg_mm, orientation_deg, pixelPerMm_source and confidence level, component_type and material_type, a list of participating frames and the scale source of each frame, key evidence indexes (such as the top-level frame number and its hash fingerprint used for assertions), and source information (uploader ID, algorithm version, model confidence level, etc.). All fields are encapsulated in JSON / CBOR format and written to a temporary fact store (session-level factstore), and a message queue ensures consistency with asynchronous consumption of the engineering knowledge graph, thereby achieving reliable transmission from pixels to structured engineering facts for direct use by the engineering knowledge graph retrieval and graph neural network inference modules.

[0036] Step S6: The server sends the diagnostic conclusion back to the mobile terminal for visualization and obtains the diagnostic feedback sent by the mobile terminal. When on-site technicians have doubts about the diagnostic conclusions or lack sufficient evidence, they can directly trigger a supplementary sampling operation. The system will automatically locate the minimum necessary supplementary sampling area and guide the user to supplement the video frames or environmental information, thus forming a data-driven "real-time correction" capability. Diagnostic feedback (the reviewed results, the supplemented multimodal data, and manually annotated key samples) will be automatically transmitted back to the server. After data quality filtering and scene consistency verification, it enters the incremental learning pool. Based on these high-quality samples, the system periodically iterates the crack detection model, the map update logic, and the reasoning preferences of the large language model, making the recognition capability continuously closer to the real distribution in actual engineering scenarios. At the same time, the review path, evidence adoption status, and manual correction content are recorded, so that the subsequent reasoning process can better reflect the real judgment logic of engineering experts. Through this continuous data-model-reasoning feedback mechanism, the intelligent agent and human experience are co-evolved, enabling the system to achieve higher stability, reliability, and engineering adaptability in long-term operation.

[0037] Step S7: Based on the diagnostic feedback, the server iteratively optimizes the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model.

[0038] The diagnostic inference process incorporates several operational techniques to improve recognition rate and efficiency: First, edge-cloud collaborative inference division of labor, with lightweight enhancement and feature extraction completed on the edge to reduce upload volume and latency, while complex multi-frame optimization and 3D reconstruction are executed in parallel on the server to utilize stronger computing power; Second, model training and deployment employ mixed-precision inference, pruning and quantization techniques, and a GPU / TPU parallel inference framework to improve throughput and maintain model real-time performance; Third, a streaming incremental update strategy is adopted in the online phase: when the server discovers new samples that deviate significantly from the existing model distribution, it can send differential fine-tuning packages to mobile terminals for local adaptation, improving the on-site adaptability of recognition.

[0039] In step S1, the lightweight assist model includes a super-resolution sub-model based on depthwise separable convolution (Mobile-SR), a deblurring sub-model based on cropped U-Net structure (Mobile-Deblur), and a view guidance sub-model based on IMU and vision coupling (Pose-GuideNet). The super-resolution sub-model is used to improve the resolution of the crack image and reconstruct texture details; the deblurring sub-model is used to perform deconvolution processing on motion-blurred regions in the crack image; the viewpoint guidance sub-model is used to detect shooting viewpoint deviations of the mobile terminal and prompt the user to correct them. The lightweight assist model is processed by model compression (distillation compression) and TensorRT-Android acceleration technology to achieve real-time operation on the CPU / GPU of mobile terminals; In specific implementation, based on the sharpness index, blurred area mask, and viewing angle deviation output by the lightweight assistance model, the following processing steps are executed in real time: ① If the blurriness is higher than the threshold, the deblurring sub-model is called to deblur the crack image and generate enhanced keyframes; ② If the resolution is insufficient or texture details are lost, the super-resolution sub-model is called to improve local sharpness and compensate for texture details; ③ If the shooting angle deviates from the component normal vector by more than the set range, the viewing angle guidance sub-model prompts the user on the interface to guide the correction of the shooting posture; ④ For areas with insufficient lighting, local brightness compensation is performed according to the exposure parameters and brightness histogram to improve crack visibility.

[0040] The environmental parameters include at least GPS coordinates, IMU attitude, light intensity, and exposure parameters; the engineering semantic priors include at least building type, structural system, component category, and location information; in specific implementation, when acquiring the crack image, data such as image frame, video frame, focal length, white balance, and timestamp are included; the multimodal data is used to supplement the geometric, optical, and engineering semantic information for crack identification.

[0041] Step S2 specifically involves: The mobile terminal performs hash calculation on each of the multimodal data using the SHA-256 algorithm to obtain a hash value, and then stores the hash value on the blockchain. The mobile terminal classifies the multimodal data according to a preset upload priority, encrypts the classified multimodal data using the SM4 algorithm, and then uploads it to the server in segments based on the CoAP Block-wise protocol, incorporating a breakpoint resume mechanism during the upload process.

[0042] To ensure data uploadability in weak or no network environments, a block-level slice CoAP segmented upload mechanism is adopted: high-priority data (such as thumbnails and low-dimensional feature vectors processed by a lightweight assist model) is uploaded first, and low-priority data (such as the original high-resolution frames) is automatically re-uploaded when the network recovers, thus realizing simultaneous shooting and transmission and breakpoint resumption.

[0043] In practice, high-priority data is sent first. This data is divided into small blocks ranging from 512B to 2KB and sent block by block through CoAP Block 1, maintaining transmission continuity even with signal fluctuations. All blocks use Confirmable mode, requiring the server to return confirmation. If a block is lost, the mobile terminal automatically retransmits the corresponding part, preventing the entire packet from being retransmitted. If the mobile terminal is completely offline, the system writes the upload progress to a local state cache. After the network is restored, the CoAP Block 2 process automatically resumes the upload based on the cache records, ensuring no blocks are missed or duplicated. Low-priority data is only supplemented when network conditions permit.

[0044] To enable the server to continue processing even when the original images are not fully uploaded, the mobile terminal generates a multimodal synchronization description file (M-Sync Descriptor) after each batch of data is uploaded. This file records the timestamp chain, feature index, and current data integrity status. Based on this, the server can perform crack prediction, scale estimation, and component classification in advance without waiting for all the original frames to arrive.

[0045] Step S3 specifically involves: The server receives the multimodal data in real time, decrypts the multimodal data using the SM4 algorithm, and verifies the integrity of the multimodal data using the hash value stored in the blockchain. It then inputs the crack images, environmental parameters, and engineering semantic priors from the multimodal data into a pre-deployed crack recognition model to obtain pixel-level crack masks and geometric features. Combined with the scale calibration information, the server calculates the physical scale parameters of the crack, including crack length, crack width, and crack direction, using a back-projection method. This generates structured crack fact data (including unique crack identifier, centerline point set, length / width sequence, direction, three-dimensional location, component type, material category, scale confidence, participating frame index, and algorithm tracing, among other structured fields). The back-projection method maps discrete points on the centerline to a local 3D coordinate system and uses multi-view triangulation or bundle adjustment to optimize the crack endpoint positions and centerline surface on the server, calculating geometric quantities such as the actual crack length, maximum / average / minimum width, and main crack direction. To quantify measurement uncertainty, a confidence score is output for each measurement. This confidence score is synthesized by weighted rules from reprojection error, scale confidence, model prediction confidence, and multi-frame consistency score. Measurements with low confidence are marked and trigger re-collection or manual review processes. Simultaneously, the component and material identification module uses semantic segmentation combined with a texture classifier and engineering prior rules to determine the component type (wall, beam, column, slab, foundation) and material category (brick, concrete, stone, wood, etc.) where the crack is located, and couples the determination results with the crack's geometric features for subsequent engineering semantic interpretation.

[0046] The crack recognition model is constructed based on a cross-modal encoder, a feature fusion module, a temporal fusion module, a multi-branch decoder, and a cross-attention module. The cross-modal encoder encodes the input crack image, environmental parameters, and engineering semantic priors to obtain visual feature vectors, numerical feature vectors, and semantic feature vectors. The feature fusion module fuses the visual feature vectors, numerical feature vectors, and semantic feature vectors through a cross-attention mechanism to obtain a high-dimensional feature tensor. The temporal fusion module aligns and weights the spatial features in the continuous high-dimensional feature tensor through a temporal convolutional network to obtain a spatiotemporal feature tensor. The multi-branch decoder is constructed based on a semantic segmentation branch, a skeleton extraction branch, and a width regression branch. The semantic segmentation branch infers the spatiotemporal feature tensor through U-Net to obtain a crack mask. The skeleton extraction branch infers the spatiotemporal feature tensor through distance transform regression to obtain a crack centerline. The width regression branch infers the spatiotemporal feature tensor through a fully convolutional network to obtain a pixel-level width map. The cross-attention module mutually reinforces the crack mask, crack centerline, and pixel-level width map, outputting a crack mask and geometric features. The geometric features include the crack centerline and the pixel-level width map.

[0047] In practice, to improve the accuracy of identifying minute cracks, the crack identification model training phase employs synthetic data augmentation (simulating different lighting, blurring, occlusion, and various material textures), adversarial enhancement, and self-supervised pre-training (such as visual pre-training based on mask reconstruction). A small number of manually labeled "hard samples" are used for online hard sample mining and fine-tuning to significantly improve the model's generalization ability in complex scenarios.

[0048] In step S4, the two-stage retrieval in the pre-created engineering knowledge graph to construct the evidence subgraph specifically involves: Based on the BM25 algorithm, the text field of the crack fact data is retrieved from the node attributes of a pre-created engineering knowledge graph to obtain the first-stage retrieval results; the numerical feature vector of the crack fact data is retrieved from the vector embedding representation of the engineering knowledge graph based on vector similarity to obtain the second-stage retrieval results; the first-stage retrieval results and the second-stage retrieval results are merged to locate key evidence nodes, and then an evidence subgraph related to the crack fact data is constructed. The evidence subgraph is constructed using a k-hop expansion strategy, and the edges in the evidence subgraph are assigned time decay weights. The graph neural network uses graph attention network or GraphSAGE, and introduces evidence consistency loss function during training (in addition to conventional cross-entropy or regression loss, it also compares the evidence ranking output by the model with the evidence importance of expert annotation / historical case, using InfoNCE or contrastive loss, so that the model learns to assign more attention to subgraph nodes with stronger evidence) to improve the interpretability of reasoning.

[0049] In step S5, the reasoning mechanism of the large language model is as follows: Engineering domain knowledge is converted into semantic fragments, which serve as the context of the large language model. Using a pre-defined reasoning template, the large language model is guided to perform engineering semantic reasoning in a logical order of "evidence-fact-objective" to obtain the nature of the crack, its impact level, and treatment suggestions. It also actively references relevant engineering domain knowledge as a structured evidence chain, outputting a diagnostic conclusion that includes the crack's nature (contraction, penetration, stress type, etc.), impact level, treatment suggestions, and a structured evidence chain (generated synchronously for subsequent auditing, expert review, and system self-learning; the data source consists of the referenced normative clauses or case sources, the graph node IDs involved in the reasoning, the similarity scores of each piece of evidence, and the image frame number and time information on which the reasoning is based; attention weights are mapped back to specific nodes, allowing users to clearly understand the source of the model's judgment, rather than simply providing a conclusion; when the confidence level of a judgment is insufficient, manual review is automatically triggered). In step S7, the iterative optimization of the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model specifically involves: The crack identification model is fine-tuned using mini-batch incremental tuning, and the large language model is fine-tuned using low-rank adaptation techniques. A domain-dependent strategy is used to locate graph subgraphs affected by new samples or new knowledge, and the vector embedding representation of the engineering knowledge graph is locally updated. After the iteratively optimized crack identification model, engineering knowledge graph, and large language model are verified offline in a sandbox environment, they are gradually deployed online through A / B testing or Canary release mechanisms, while retaining version control and rollback capabilities.

[0050] When new samples or specifications are introduced and their association strength with certain nodes / edges in the engineering knowledge graph exceeds a preset threshold, or when entirely new entities appear, the affected subgraph is located using a "dependency domain" strategy. This typically employs one-hop or two-hop diffusion to define update boundaries, allowing subsequent processing only on the affected subgraph rather than reconstructing the entire graph. At the model level, different model update paths are differentiated: for visual perception and crack measurement modules, incremental training or fine-tuning (e.g., freezing the backbone and fine-tuning only the head layer) is prioritized; for the engineering semantic capabilities of large models, low-rank fine-tuning such as LoRA / Adapter is used to achieve rapid adaptation while preserving the stability of the large model; for graph embedding, online incremental embedding methods (e.g., local retraining or fast approximation algorithms using incremental TransE / GraphSAGE) are employed to recalculate only the embedding vectors of affected nodes and update the index as needed. All updates undergo sandboxed training and offline validation on the server. Validation includes accuracy, recall, evidence consistency testing, and regression risk detection. An A / B or Canary release mechanism is used to gradually roll out to online traffic, while retaining rollback snapshots and complete model / graph version control records.

[0051] During the graph structure update process, new nodes and relationships are written into a temporary change set in the form of triples. Changes are merged according to time windows and priorities, and conflicts (such as different definitions of the same entity) are handled through rule-based processing or marked as "pending manual judgment" before the change to maintain data consistency. After the change is applied, local index reconstruction and subgraph-level retrieval performance testing are triggered to ensure that the downstream retrieval and GNN encoding modules maintain real-time performance and accuracy after the update. To ensure auditability and compliance, all incremental update processes record detailed traceability information (reason for change, initiator of change, verification results, and timestamp), and the model output is monitored online. Key indicator alarms are set (such as declining evidence matching rate, increased manual review rate, etc.). Once the indicator exceeds the threshold, it automatically reverts to the previous stable version and abnormal samples are labeled as pending analysis. The entire process supports event-driven (e.g., new specification release, centralized uploading of major manual reviews) and periodic scheduling (e.g., batch merging of minor changes at night), and can be updated in segments according to business area or building type to achieve optimal matching of computing resources and business scenarios. This enables low-cost, high-frequency and operable incremental learning and dynamic updates of knowledge graphs while ensuring engineering-level credibility and traceability.

[0052] A preferred embodiment of the crack diagnosis reasoning system based on an engineering inspection intelligent agent according to the present invention includes the following modules: The multimodal data acquisition module is used to acquire multimodal data including crack images, environmental parameters, scale calibration information and engineering semantic priors through a mobile terminal. During the crack image acquisition process, a lightweight assistance model deployed on the mobile terminal is used to guide the shooting posture, and the lightweight assistance model is used to preprocess the acquired crack images. The lightweight assistance model is used to enhance and control the image quality of crack images in real time, so as to solve the problem of image blurring caused by shaking, insufficient lighting or angular deviation at the engineering site. In practice, the mobile terminal also performs distortion correction and perspective correction on the crack image. If the mobile terminal can provide camera intrinsic parameters, these parameters are used first for accurate distortion correction. If not, multi-frame self-calibration combined with reference object detection (such as a ruler, brick seam, or standard paper) is used to fit the homography matrix to recover the local planar viewpoint, outputting a pixel-to-physical scale mapping (pixel-per-mm) and the corresponding scale confidence. The resulting scale mapping and camera pose are uploaded as metadata along with the thumbnail and low-dimensional features. Upon receiving the metadata, the server immediately uses it to perform physical scale association on the crack image, providing a benchmark for subsequent pixel-to-millimeter conversion.

[0053] The multimodal data upload module is used by the mobile terminal to upload the multimodal data to the server by combining the preset upload priority and the breakpoint resume mechanism. The crack fact data generation module is used to receive the multimodal data in real time from the server, input the multimodal data into a pre-deployed crack identification model to obtain pixel-level crack masks and geometric features, and calculate the physical scale parameters of the cracks in combination with the scale calibration information to generate structured crack fact data. The knowledge graph retrieval module is used by the server to perform a two-stage retrieval in a pre-created engineering knowledge graph to construct an evidence subgraph, and to use a graph neural network to encode and reason about the evidence subgraph to generate a subgraph context vector. The diagnostic conclusion generation module is used by the server to fuse the crack fact data with the subgraph context vector to obtain fused features, input the fused features into a large language model fine-tuned with engineering domain knowledge, and perform deep engineering semantic reasoning through the large language model to generate a diagnostic conclusion that includes the nature of the crack, the degree of impact, treatment suggestions, and a structured chain of evidence. The server organizes the output of each detection into a standardized data structure for subsequent engineering knowledge graph retrieval and inference. This data structure includes: a unique crack identifier (crack_id), timestamp and session ID, centerline point set (a sequence of three-dimensional coordinate arrays), length_mm, width_max_mm, width_avg_mm, orientation_deg, pixelPerMm_source and confidence level, component_type and material_type, a list of participating frames and the scale source of each frame, key evidence indexes (such as the top-level frame number and its hash fingerprint used for assertions), and source information (uploader ID, algorithm version, model confidence level, etc.). All fields are encapsulated in JSON / CBOR format and written to a temporary fact store (session-level factstore), and a message queue ensures consistency with asynchronous consumption of the engineering knowledge graph, thereby achieving reliable transmission from pixels to structured engineering facts for direct use by the engineering knowledge graph retrieval and graph neural network inference modules.

[0054] The diagnostic conclusion feedback and display module is used by the server to send the diagnostic conclusion back to the mobile terminal for visualization display and to obtain the diagnostic feedback sent by the mobile terminal. When on-site technicians have doubts about the diagnostic conclusions or lack sufficient evidence, they can directly trigger a supplementary sampling operation. The system will automatically locate the minimum necessary supplementary sampling area and guide the user to supplement the video frames or environmental information, thus forming a data-driven "real-time correction" capability. Diagnostic feedback (the reviewed results, the supplemented multimodal data, and manually annotated key samples) will be automatically transmitted back to the server. After data quality filtering and scene consistency verification, it enters the incremental learning pool. Based on these high-quality samples, the system periodically iterates the crack detection model, the map update logic, and the reasoning preferences of the large language model, making the recognition capability continuously closer to the real distribution in actual engineering scenarios. At the same time, the review path, evidence adoption status, and manual correction content are recorded, so that the subsequent reasoning process can better reflect the real judgment logic of engineering experts. Through this continuous data-model-reasoning feedback mechanism, the intelligent agent and human experience are co-evolved, enabling the system to achieve higher stability, reliability, and engineering adaptability in long-term operation.

[0055] The iterative optimization module is used by the server to iteratively optimize the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model based on the diagnostic feedback.

[0056] The diagnostic inference process incorporates several operational techniques to improve recognition rate and efficiency: First, edge-cloud collaborative inference division of labor, with lightweight enhancement and feature extraction completed on the edge to reduce upload volume and latency, while complex multi-frame optimization and 3D reconstruction are executed in parallel on the server to utilize stronger computing power; Second, model training and deployment employ mixed-precision inference, pruning and quantization techniques, and a GPU / TPU parallel inference framework to improve throughput and maintain model real-time performance; Third, a streaming incremental update strategy is adopted in the online phase: when the server discovers new samples that deviate significantly from the existing model distribution, it can send differential fine-tuning packages to mobile terminals for local adaptation, improving the on-site adaptability of recognition.

[0057] In the multimodal data acquisition module, the lightweight assistance model includes a super-resolution sub-model based on depthwise separable convolution (Mobile-SR), a deblurring sub-model based on cropped U-Net structure (Mobile-Deblur), and a view guidance sub-model based on IMU and vision coupling (Pose-GuideNet). The super-resolution sub-model is used to improve the resolution of the crack image and reconstruct texture details; the deblurring sub-model is used to perform deconvolution processing on motion-blurred regions in the crack image; the viewpoint guidance sub-model is used to detect shooting viewpoint deviations of the mobile terminal and prompt the user to correct them. The lightweight assist model is processed by model compression (distillation compression) and TensorRT-Android acceleration technology to achieve real-time operation on the CPU / GPU of mobile terminals; In specific implementation, based on the sharpness index, blurred area mask, and viewing angle deviation output by the lightweight assistance model, the following processing steps are executed in real time: ① If the blurriness is higher than the threshold, the deblurring sub-model is called to deblur the crack image and generate enhanced keyframes; ② If the resolution is insufficient or texture details are lost, the super-resolution sub-model is called to improve local sharpness and compensate for texture details; ③ If the shooting angle deviates from the component normal vector by more than the set range, the viewing angle guidance sub-model prompts the user on the interface to guide the correction of the shooting posture; ④ For areas with insufficient lighting, local brightness compensation is performed according to the exposure parameters and brightness histogram to improve crack visibility.

[0058] The environmental parameters include at least GPS coordinates, IMU attitude, light intensity, and exposure parameters; the engineering semantic priors include at least building type, structural system, component category, and location information; in specific implementation, when acquiring the crack image, data such as image frame, video frame, focal length, white balance, and timestamp are included; the multimodal data is used to supplement the geometric, optical, and engineering semantic information for crack identification.

[0059] The multimodal data upload module is specifically used for: The mobile terminal performs hash calculation on each of the multimodal data using the SHA-256 algorithm to obtain a hash value, and then stores the hash value on the blockchain. The mobile terminal classifies the multimodal data according to a preset upload priority, encrypts the classified multimodal data using the SM4 algorithm, and then uploads it to the server in segments based on the CoAP Block-wise protocol, incorporating a breakpoint resume mechanism during the upload process.

[0060] To ensure data uploadability in weak or no network environments, a block-level slice CoAP segmented upload mechanism is adopted: high-priority data (such as thumbnails and low-dimensional feature vectors processed by a lightweight assist model) is uploaded first, and low-priority data (such as the original high-resolution frames) is automatically re-uploaded when the network recovers, thus realizing simultaneous shooting and transmission and breakpoint resumption.

[0061] In practice, high-priority data is sent first. This data is divided into small blocks ranging from 512B to 2KB and sent block by block through CoAP Block 1, maintaining transmission continuity even with signal fluctuations. All blocks use Confirmable mode, requiring the server to return confirmation. If a block is lost, the mobile terminal automatically retransmits the corresponding part, preventing the entire packet from being retransmitted. If the mobile terminal is completely offline, the system writes the upload progress to a local state cache. After the network is restored, the CoAP Block 2 process automatically resumes the upload based on the cache records, ensuring no blocks are missed or duplicated. Low-priority data is only supplemented when network conditions permit.

[0062] To enable the server to continue processing even when the original images are not fully uploaded, the mobile terminal generates a multimodal synchronization description file (M-Sync Descriptor) after each batch of data is uploaded. This file records the timestamp chain, feature index, and current data integrity status. Based on this, the server can perform crack prediction, scale estimation, and component classification in advance without waiting for all the original frames to arrive.

[0063] The crack fact data generation module is specifically used for: The server receives the multimodal data in real time, decrypts the multimodal data using the SM4 algorithm, and verifies the integrity of the multimodal data using the hash value stored in the blockchain. It then inputs the crack images, environmental parameters, and engineering semantic priors from the multimodal data into a pre-deployed crack recognition model to obtain pixel-level crack masks and geometric features. Combined with the scale calibration information, the server calculates the physical scale parameters of the crack, including crack length, crack width, and crack direction, using a back-projection method. This generates structured crack fact data (including unique crack identifier, centerline point set, length / width sequence, direction, three-dimensional location, component type, material category, scale confidence, participating frame index, and algorithm tracing, among other structured fields). The back-projection method maps discrete points on the centerline to a local 3D coordinate system and uses multi-view triangulation or bundle adjustment to optimize the crack endpoint positions and centerline surface on the server, calculating geometric quantities such as the actual crack length, maximum / average / minimum width, and main crack direction. To quantify measurement uncertainty, a confidence score is output for each measurement. This confidence score is synthesized by weighted rules from reprojection error, scale confidence, model prediction confidence, and multi-frame consistency score. Measurements with low confidence are marked and trigger re-collection or manual review processes. Simultaneously, the component and material identification module uses semantic segmentation combined with a texture classifier and engineering prior rules to determine the component type (wall, beam, column, slab, foundation) and material category (brick, concrete, stone, wood, etc.) where the crack is located, and couples the determination results with the crack's geometric features for subsequent engineering semantic interpretation.

[0064] The crack recognition model is constructed based on a cross-modal encoder, a feature fusion module, a temporal fusion module, a multi-branch decoder, and a cross-attention module. The cross-modal encoder encodes the input crack image, environmental parameters, and engineering semantic priors to obtain visual feature vectors, numerical feature vectors, and semantic feature vectors. The feature fusion module fuses the visual feature vectors, numerical feature vectors, and semantic feature vectors through a cross-attention mechanism to obtain a high-dimensional feature tensor. The temporal fusion module aligns and weights the spatial features in the continuous high-dimensional feature tensor through a temporal convolutional network to obtain a spatiotemporal feature tensor. The multi-branch decoder is constructed based on a semantic segmentation branch, a skeleton extraction branch, and a width regression branch. The semantic segmentation branch infers the spatiotemporal feature tensor through U-Net to obtain a crack mask. The skeleton extraction branch infers the spatiotemporal feature tensor through distance transform regression to obtain a crack centerline. The width regression branch infers the spatiotemporal feature tensor through a fully convolutional network to obtain a pixel-level width map. The cross-attention module mutually reinforces the crack mask, crack centerline, and pixel-level width map, outputting a crack mask and geometric features. The geometric features include the crack centerline and the pixel-level width map.

[0065] In practice, to improve the accuracy of identifying minute cracks, the crack identification model training phase employs synthetic data augmentation (simulating different lighting, blurring, occlusion, and various material textures), adversarial enhancement, and self-supervised pre-training (such as visual pre-training based on mask reconstruction). A small number of manually labeled "hard samples" are used for online hard sample mining and fine-tuning to significantly improve the model's generalization ability in complex scenarios.

[0066] In the knowledge graph retrieval module, the two-stage retrieval in the pre-created engineering knowledge graph to construct the evidence subgraph specifically involves: Based on the BM25 algorithm, the text field of the crack fact data is retrieved from the node attributes of a pre-created engineering knowledge graph to obtain the first-stage retrieval results; the numerical feature vector of the crack fact data is retrieved from the vector embedding representation of the engineering knowledge graph based on vector similarity to obtain the second-stage retrieval results; the first-stage retrieval results and the second-stage retrieval results are merged to locate key evidence nodes, and then an evidence subgraph related to the crack fact data is constructed. The evidence subgraph is constructed using a k-hop expansion strategy, and the edges in the evidence subgraph are assigned time decay weights. The graph neural network uses graph attention network or GraphSAGE, and introduces evidence consistency loss function during training (in addition to conventional cross-entropy or regression loss, it also compares the evidence ranking output by the model with the evidence importance of expert annotation / historical case, using InfoNCE or contrastive loss, so that the model learns to assign more attention to subgraph nodes with stronger evidence) to improve the interpretability of reasoning.

[0067] In the diagnostic conclusion generation module, the reasoning mechanism of the large language model is as follows: Engineering domain knowledge is converted into semantic fragments, which serve as the context of the large language model. Using a pre-defined reasoning template, the large language model is guided to perform engineering semantic reasoning in a logical order of "evidence-fact-objective" to obtain the nature of the crack, its impact level, and treatment suggestions. It also actively references relevant engineering domain knowledge as a structured evidence chain, outputting a diagnostic conclusion that includes the crack's nature (contraction, penetration, stress type, etc.), impact level, treatment suggestions, and a structured evidence chain (generated synchronously for subsequent auditing, expert review, and system self-learning; the data source consists of the referenced normative clauses or case sources, the graph node IDs involved in the reasoning, the similarity scores of each piece of evidence, and the image frame number and time information on which the reasoning is based; attention weights are mapped back to specific nodes, allowing users to clearly understand the source of the model's judgment, rather than simply providing a conclusion; when the confidence level of a judgment is insufficient, manual review is automatically triggered). In the iterative optimization module, the iterative optimization of the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model specifically involves: The crack identification model is fine-tuned using mini-batch incremental tuning, and the large language model is fine-tuned using low-rank adaptation techniques. A domain-dependent strategy is used to locate graph subgraphs affected by new samples or new knowledge, and the vector embedding representation of the engineering knowledge graph is locally updated. After the iteratively optimized crack identification model, engineering knowledge graph, and large language model are verified offline in a sandbox environment, they are gradually deployed online through A / B testing or Canary release mechanisms, while retaining version control and rollback capabilities.

[0068] When new samples or specifications are introduced and their association strength with certain nodes / edges in the engineering knowledge graph exceeds a preset threshold, or when entirely new entities appear, the affected subgraph is located using a "dependency domain" strategy. This typically employs one-hop or two-hop diffusion to define update boundaries, allowing subsequent processing only on the affected subgraph rather than reconstructing the entire graph. At the model level, different model update paths are differentiated: for visual perception and crack measurement modules, incremental training or fine-tuning (e.g., freezing the backbone and fine-tuning only the head layer) is prioritized; for the engineering semantic capabilities of large models, low-rank fine-tuning such as LoRA / Adapter is used to achieve rapid adaptation while preserving the stability of the large model; for graph embedding, online incremental embedding methods (e.g., local retraining or fast approximation algorithms using incremental TransE / GraphSAGE) are employed to recalculate only the embedding vectors of affected nodes and update the index as needed. All updates undergo sandboxed training and offline validation on the server. Validation includes accuracy, recall, evidence consistency testing, and regression risk detection. An A / B or Canary release mechanism is used to gradually roll out to online traffic, while retaining rollback snapshots and complete model / graph version control records.

[0069] During the graph structure update process, new nodes and relationships are written into a temporary change set in the form of triples. Changes are merged according to time windows and priorities, and conflicts (such as different definitions of the same entity) are handled through rule-based processing or marked as "pending manual judgment" before the change to maintain data consistency. After the change is applied, local index reconstruction and subgraph-level retrieval performance testing are triggered to ensure that the downstream retrieval and GNN encoding modules maintain real-time performance and accuracy after the update. To ensure auditability and compliance, all incremental update processes record detailed traceability information (reason for change, initiator of change, verification results, and timestamp), and the model output is monitored online. Key indicator alarms are set (such as declining evidence matching rate, increased manual review rate, etc.). Once the indicator exceeds the threshold, it automatically reverts to the previous stable version and abnormal samples are labeled as pending analysis. The entire process supports event-driven (e.g., new specification release, centralized uploading of major manual reviews) and periodic scheduling (e.g., batch merging of minor changes at night), and can be updated in segments according to business area or building type to achieve optimal matching of computing resources and business scenarios. This enables low-cost, high-frequency and operable incremental learning and dynamic updates of knowledge graphs while ensuring engineering-level credibility and traceability.

[0070] In summary, the advantages of this invention are as follows: 1. Multimodal data, including crack images, environmental parameters, scale calibration information, and engineering semantic priors, is collected via mobile terminals. During crack image acquisition, a lightweight assistive model guides the shooting posture and preprocesses the acquired crack images. Combined with preset upload priorities and a breakpoint resume mechanism, the multimodal data is uploaded to the server. The server inputs the multimodal data into a crack recognition model to obtain pixel-level crack masks and geometric features, and calculates the physical scale parameters of the cracks based on scale calibration information, generating structured crack factual data. Using the crack factual data as input, the server performs a two-stage retrieval in the engineering knowledge graph to construct an evidence subgraph. A graph neural network is then used to encode and infer the evidence subgraph to generate a subgraph context vector. The crack factual data and the subgraph context vector are fused to obtain fused features. These fused features are input into a large language model fine-tuned with engineering domain knowledge to generate diagnostic conclusions, which are then transmitted back to the mobile terminal for visualization. The system demonstrates and acquires diagnostic feedback from mobile terminals. Based on this feedback, the server iteratively optimizes the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model. Specifically, it integrates a lightweight assistance model with the mobile terminal to guide shooting and perform image preprocessing, combining multimodal data acquisition and reliable transmission mechanisms to improve data quality and acquisition efficiency from the source. On the server, the crack identification model, which integrates cross-modal information, achieves pixel-level precise quantization. By retrieving the engineering knowledge graph and using graph neural network inference, it associates crack factual data with engineering domain knowledge, providing evidence support for the deep diagnosis of the large language model and generating interpretable diagnostic conclusions containing structured evidence chains. Finally, based on user feedback, the crack identification model, engineering knowledge graph, and large language model are continuously iteratively optimized to adapt to complex field environments and continuously improve their generalization ability to new scenarios, ultimately greatly improving the accuracy, efficiency, interpretability, environmental adaptability, and generalization ability of crack diagnosis.

[0071] 2. By innovatively integrating multimodal data such as crack images, environmental parameters, scale calibration information, and engineering semantic priors during the data acquisition phase, the richness and contextual relevance of the input information are ensured. Through lightweight assistance models deployed on mobile terminals (such as super-resolution, deblurring, and viewpoint guidance sub-models), real-time guidance of shooting posture and image preprocessing are achieved, effectively improving the quality and efficiency of on-site data acquisition. This design solves the problems of reliance on manual experience, single data, and susceptibility to environmental interference in traditional engineering inspection, providing a reliable data foundation for subsequent accurate diagnosis and enhancing the robustness and practicality of the system.

[0072] 3. By using preset upload priorities and breakpoint resume mechanisms, combined with SM4 encryption algorithm and CoAP Block-wise protocol for segmented uploads, reliable data transmission is ensured in unstable network environments. At the same time, the SHA-256 hash algorithm is used to store data on the blockchain, realizing the immutability and integrity verification of the data. These technical means significantly improve the applicability of the system in engineering sites (such as remote areas or complex network conditions), reduce the risk of data loss or leakage, and meet the high standards of data security and real-time performance required by modern engineering inspection.

[0073] 4. The server adopts a crack recognition model based on a cross-modal encoder, feature fusion module, and multi-branch decoder, which can collaboratively process visual, numerical, and semantic features to generate pixel-level crack masks and geometric features (such as centerline and width maps). Combined with the back projection method to calculate physical scale parameters, it realizes automated and high-precision quantification of crack length, width, and other parameters, avoiding the errors caused by relying on a single modality in traditional image processing. This multi-source fusion and spatiotemporal feature alignment design improves the accuracy and robustness of crack detection, and is especially suitable for the identification of fine cracks in complex engineering structures.

[0074] 5. By performing two-stage retrieval (text recall based on BM25 and vector recall based on vector similarity) in a pre-created engineering knowledge graph, and combining it with a k-hop expansion strategy to construct an evidence subgraph, efficient and comprehensive evidence mining is achieved. Graph neural networks (such as graph attention networks) are used to encode and reason about the evidence subgraph, and an evidence consistency loss function is introduced to enhance the logic and interpretability of the reasoning process. This design enables the system to quickly locate relevant cases and rules from massive amounts of engineering knowledge, providing structured evidence support for diagnosis and improving the scientific nature and transparency of decision-making.

[0075] 6. After fusing the factual data of cracks with the context vector of the subgraph, the data is input into a large language model finely tuned with engineering domain knowledge. Using a preset "evidence-fact-target" reasoning template, deep-level engineering semantic reasoning is achieved. This model can automatically generate diagnostic conclusions that include the nature of cracks, degree of impact, treatment suggestions, and a structured chain of evidence. It not only provides professional judgment but also outputs explanations and suggestions in a human-like manner. This solves the problem of traditional expert systems relying on hard-coded rules and lacking flexibility, enabling the system to have human-like reasoning capabilities and significantly improving user experience and diagnostic practicality.

[0076] 7. Based on diagnostic feedback sent from mobile terminals, the crack identification model, the vector embedding representation of the engineering knowledge graph, and the large language model are iteratively optimized, such as by using small-batch incremental fine-tuning, low-rank adaptation technology, and local graph updates. Through mechanisms such as sandbox environment verification and A / B testing, the system is gradually deployed to ensure its stability and adaptability when introducing new data or knowledge. This closed-loop optimization design enables the system to continuously learn new cases in engineering practice, continuously improve diagnostic accuracy, achieve long-term effectiveness and scalability, and reduce maintenance costs.

[0077] 8. From data collection, uploading, processing, inference to feedback optimization, a complete end-to-end automated closed-loop process is formed; the modules work closely together, such as the mobile terminal being responsible for lightweight preprocessing and interaction, the server performing heavy computation and inference, and the final results being visualized and feedback collected; this integrated design realizes the potential for full-process intelligentization and unattended operation of engineering inspection, greatly improving diagnostic efficiency (such as reducing manual inspection time) and consistency, which is in line with the development trend of modern intelligent manufacturing and infrastructure maintenance.

[0078] 9. The lightweight assist model achieves real-time operation on mobile terminal CPUs / GPUs through model compression and acceleration technologies, reducing hardware dependence and energy consumption; the server side uses efficient algorithms (such as temporal convolutional networks and graph neural networks) for parallel processing, supporting high-concurrency data streams; these optimizations ensure that the system can be stably deployed in resource-constrained engineering sites (such as handheld devices) and the cloud, balancing performance and cost.

[0079] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A crack diagnosis reasoning method based on an engineering inspection intelligent agent, characterized in that: Includes the following steps: Step S1: Collect multimodal data including crack images, environmental parameters, scale calibration information and engineering semantic priors through a mobile terminal. During the crack image acquisition process, a lightweight assistance model deployed on the mobile terminal is used to guide the shooting posture, and the lightweight assistance model is used to preprocess the acquired crack images. Step S2: The mobile terminal uploads the multimodal data to the server based on the preset upload priority and the breakpoint resume mechanism. Step S3: The server receives each of the multimodal data in real time, inputs each of the multimodal data into the pre-deployed crack recognition model to obtain pixel-level crack masks and geometric features, and calculates the physical scale parameters of the cracks in combination with the scale calibration information to generate structured crack fact data. Step S4: The server takes the crack fact data as input, performs a two-stage retrieval in the pre-created engineering knowledge graph to construct an evidence subgraph, and uses a graph neural network to encode and reason about the evidence subgraph to generate a subgraph context vector. Step S5: The server fuses the crack fact data with the subgraph context vector to obtain fused features, and inputs the fused features into a large language model fine-tuned by engineering domain knowledge. The large language model performs deep engineering semantic reasoning to generate a diagnostic conclusion that includes crack nature, impact degree, treatment suggestions and structured evidence chain. Step S6: The server sends the diagnostic conclusion back to the mobile terminal for visualization and obtains the diagnostic feedback sent by the mobile terminal. Step S7: Based on the diagnostic feedback, the server iteratively optimizes the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model.

2. The crack diagnosis reasoning method based on an engineering inspection intelligent agent as described in claim 1, characterized in that: In step S1, the lightweight assist model includes a super-resolution sub-model based on depthwise separable convolution, a deblurring sub-model based on cropped U-Net structure, and a viewpoint guidance sub-model based on IMU and vision coupling. The super-resolution sub-model is used to improve the resolution of the crack image and reconstruct texture details; the deblurring sub-model is used to perform deconvolution processing on the motion-blurred regions in the crack image. The perspective guidance sub-model is used to detect the shooting perspective deviation of the mobile terminal and prompt the user to correct it; The lightweight assistance model is processed through model compression and acceleration technology to achieve real-time operation on the CPU / GPU of mobile terminals; The environmental parameters include at least GPS coordinates, IMU attitude, light intensity, and exposure parameters; the engineering semantic priors include at least building type, structural system, component category, and location information. Step S2 specifically involves: The mobile terminal performs hash calculation on each of the multimodal data using the SHA-256 algorithm to obtain a hash value, and then stores the hash value on the blockchain. The mobile terminal classifies the multimodal data according to a preset upload priority, encrypts the classified multimodal data using the SM4 algorithm, and then uploads it to the server in segments based on the CoAP Block-wise protocol, incorporating a breakpoint resume mechanism during the upload process.

3. The crack diagnosis reasoning method based on an engineering inspection intelligent agent as described in claim 1, characterized in that: Step S3 specifically involves: The server receives the multimodal data in real time, decrypts the multimodal data using the SM4 algorithm, verifies the integrity of the multimodal data using the hash value stored in the blockchain, inputs the crack images, environmental parameters, and engineering semantic priors from the multimodal data into a pre-deployed crack recognition model to obtain pixel-level crack masks and geometric features, and combines the scale calibration information to calculate the physical scale parameters of the crack, including crack length, crack width, and crack direction, using the back projection method to generate structured crack fact data. The crack recognition model is constructed based on a cross-modal encoder, a feature fusion module, a temporal fusion module, a multi-branch decoder, and a cross-attention module. The cross-modal encoder encodes the input crack image, environmental parameters, and engineering semantic priors to obtain visual feature vectors, numerical feature vectors, and semantic feature vectors. The feature fusion module fuses the visual feature vectors, numerical feature vectors, and semantic feature vectors through a cross-attention mechanism to obtain a high-dimensional feature tensor. The temporal fusion module aligns and weights the spatial features in the continuous high-dimensional feature tensor through a temporal convolutional network to obtain a spatiotemporal feature tensor. The multi-branch decoder is constructed based on a semantic segmentation branch, a skeleton extraction branch, and a width regression branch. The semantic segmentation branch infers the spatiotemporal feature tensor through U-Net to obtain a crack mask. The skeleton extraction branch infers the spatiotemporal feature tensor through distance transform regression to obtain a crack centerline. The width regression branch infers the spatiotemporal feature tensor through a fully convolutional network to obtain a pixel-level width map. The cross-attention module mutually reinforces the crack mask, crack centerline, and pixel-level width map, outputting a crack mask and geometric features. The geometric features include the crack centerline and the pixel-level width map.

4. The crack diagnosis reasoning method based on an engineering inspection intelligent agent as described in claim 1, characterized in that: In step S4, the two-stage retrieval in the pre-created engineering knowledge graph to construct the evidence subgraph specifically involves: Based on the BM25 algorithm, the text field of the crack fact data is retrieved from the node attributes of a pre-created engineering knowledge graph to obtain the first-stage retrieval results; the numerical feature vector of the crack fact data is retrieved from the vector embedding representation of the engineering knowledge graph based on vector similarity to obtain the second-stage retrieval results. The first-stage retrieval results and the second-stage retrieval results are merged to locate key evidence nodes, and then an evidence subgraph related to the crack fact data is constructed. The evidence subgraph is constructed using a k-hop expansion strategy, and the edges in the evidence subgraph are assigned time decay weights. The graph neural network uses graph attention networks or GraphSAGE, and introduces an evidence consistency loss function during training to improve the interpretability of reasoning.

5. The crack diagnosis reasoning method based on an engineering inspection intelligent agent as described in claim 1, characterized in that: In step S5, the reasoning mechanism of the large language model is as follows: Engineering domain knowledge is converted into semantic fragments, which serve as the context of the large language model. Using a pre-set reasoning template, the large language model is guided to perform engineering semantic reasoning in the logical order of "evidence-fact-goal" to obtain the nature of the crack, the degree of impact, and the treatment suggestions. It also actively references the relevant engineering domain knowledge as a structured evidence chain and outputs a diagnostic conclusion that includes the nature of the crack, the degree of impact, the treatment suggestions, and the structured evidence chain. In step S7, the iterative optimization of the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model specifically involves: The crack identification model is fine-tuned using mini-batch incremental tuning, and the large language model is fine-tuned using low-rank adaptation techniques. A domain-dependent strategy is used to locate graph subgraphs affected by new samples or new knowledge, and the vector embedding representation of the engineering knowledge graph is locally updated. After the iteratively optimized crack identification model, engineering knowledge graph, and large language model are verified offline in a sandbox environment, they are gradually deployed online through A / B testing or Canary release mechanisms, while retaining version control and rollback capabilities.

6. A crack diagnosis reasoning system based on an engineering inspection intelligent agent, characterized in that: Includes the following modules: The multimodal data acquisition module is used to acquire multimodal data including crack images, environmental parameters, scale calibration information and engineering semantic priors through a mobile terminal. During the crack image acquisition process, a lightweight assistance model deployed on the mobile terminal is used to guide the shooting posture, and the lightweight assistance model is used to preprocess the acquired crack images. The multimodal data upload module is used by the mobile terminal to upload the multimodal data to the server by combining the preset upload priority and the breakpoint resume mechanism. The crack fact data generation module is used to receive the multimodal data in real time from the server, input the multimodal data into a pre-deployed crack identification model to obtain pixel-level crack masks and geometric features, and calculate the physical scale parameters of the cracks in combination with the scale calibration information to generate structured crack fact data. The knowledge graph retrieval module is used by the server to perform a two-stage retrieval in a pre-created engineering knowledge graph to construct an evidence subgraph, and to use a graph neural network to encode and reason about the evidence subgraph to generate a subgraph context vector. The diagnostic conclusion generation module is used by the server to fuse the crack fact data with the subgraph context vector to obtain fused features, input the fused features into a large language model fine-tuned with engineering domain knowledge, and perform deep engineering semantic reasoning through the large language model to generate a diagnostic conclusion that includes the nature of the crack, the degree of impact, treatment suggestions, and a structured chain of evidence. The diagnostic conclusion feedback and display module is used by the server to send the diagnostic conclusion back to the mobile terminal for visualization display and to obtain the diagnostic feedback sent by the mobile terminal. The iterative optimization module is used by the server to iteratively optimize the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model based on the diagnostic feedback.

7. The crack diagnosis reasoning system based on an engineering inspection intelligent agent as described in claim 6, characterized in that: In the multimodal data acquisition module, the lightweight assistance model includes a super-resolution sub-model based on depthwise separable convolution, a deblurring sub-model based on a cropped U-Net structure, and a viewpoint guidance sub-model based on IMU and vision coupling. The super-resolution sub-model is used to improve the resolution of the crack image and reconstruct texture details; the deblurring sub-model is used to perform deconvolution processing on the motion-blurred regions in the crack image. The perspective guidance sub-model is used to detect the shooting perspective deviation of the mobile terminal and prompt the user to correct it; The lightweight assistance model is processed through model compression and acceleration technology to achieve real-time operation on the CPU / GPU of mobile terminals; The environmental parameters include at least GPS coordinates, IMU attitude, light intensity, and exposure parameters; the engineering semantic priors include at least building type, structural system, component category, and location information. The multimodal data upload module is specifically used for: The mobile terminal performs hash calculation on each of the multimodal data using the SHA-256 algorithm to obtain a hash value, and then stores the hash value on the blockchain. The mobile terminal classifies the multimodal data according to a preset upload priority, encrypts the classified multimodal data using the SM4 algorithm, and then uploads it to the server in segments based on the CoAP Block-wise protocol, incorporating a breakpoint resume mechanism during the upload process.

8. The crack diagnosis reasoning system based on an engineering inspection intelligent agent as described in claim 6, characterized in that: The crack fact data generation module is specifically used for: The server receives the multimodal data in real time, decrypts the multimodal data using the SM4 algorithm, verifies the integrity of the multimodal data using the hash value stored in the blockchain, inputs the crack images, environmental parameters, and engineering semantic priors from the multimodal data into a pre-deployed crack recognition model to obtain pixel-level crack masks and geometric features, and combines the scale calibration information to calculate the physical scale parameters of the crack, including crack length, crack width, and crack direction, using the back projection method to generate structured crack fact data. The crack recognition model is constructed based on a cross-modal encoder, a feature fusion module, a temporal fusion module, a multi-branch decoder, and a cross-attention module. The cross-modal encoder encodes the input crack image, environmental parameters, and engineering semantic priors to obtain visual feature vectors, numerical feature vectors, and semantic feature vectors. The feature fusion module fuses the visual feature vectors, numerical feature vectors, and semantic feature vectors through a cross-attention mechanism to obtain a high-dimensional feature tensor. The temporal fusion module aligns and weights the spatial features in the continuous high-dimensional feature tensor through a temporal convolutional network to obtain a spatiotemporal feature tensor. The multi-branch decoder is constructed based on a semantic segmentation branch, a skeleton extraction branch, and a width regression branch. The semantic segmentation branch infers the spatiotemporal feature tensor through U-Net to obtain a crack mask. The skeleton extraction branch infers the spatiotemporal feature tensor through distance transform regression to obtain a crack centerline. The width regression branch infers the spatiotemporal feature tensor through a fully convolutional network to obtain a pixel-level width map. The cross-attention module mutually reinforces the crack mask, crack centerline, and pixel-level width map, outputting a crack mask and geometric features. The geometric features include the crack centerline and the pixel-level width map.

9. The crack diagnosis reasoning system based on an engineering inspection intelligent agent as described in claim 6, characterized in that: In the knowledge graph retrieval module, the two-stage retrieval in the pre-created engineering knowledge graph to construct the evidence subgraph specifically involves: Based on the BM25 algorithm, the text field of the crack fact data is retrieved from the node attributes of a pre-created engineering knowledge graph to obtain the first-stage retrieval results; the numerical feature vector of the crack fact data is retrieved from the vector embedding representation of the engineering knowledge graph based on vector similarity to obtain the second-stage retrieval results. The first-stage retrieval results and the second-stage retrieval results are merged to locate key evidence nodes, and then an evidence subgraph related to the crack fact data is constructed. The evidence subgraph is constructed using a k-hop expansion strategy, and the edges in the evidence subgraph are assigned time decay weights. The graph neural network uses graph attention networks or GraphSAGE, and introduces an evidence consistency loss function during training to improve the interpretability of reasoning.

10. The crack diagnosis reasoning system based on an engineering inspection intelligent agent as described in claim 6, characterized in that: In the diagnostic conclusion generation module, the reasoning mechanism of the large language model is as follows: Engineering domain knowledge is converted into semantic fragments, which serve as the context of the large language model. Using a pre-set reasoning template, the large language model is guided to perform engineering semantic reasoning in the logical order of "evidence-fact-goal" to obtain the nature of the crack, the degree of impact, and the treatment suggestions. It also actively references the relevant engineering domain knowledge as a structured evidence chain and outputs a diagnostic conclusion that includes the nature of the crack, the degree of impact, the treatment suggestions, and the structured evidence chain. In the iterative optimization module, the iterative optimization of the crack identification model, the vector embedding representation of the engineering knowledge graph, and the inference parameters of the large language model specifically involves: The crack identification model is fine-tuned using mini-batch incremental tuning, and the large language model is fine-tuned using low-rank adaptation techniques. A domain-dependent strategy is used to locate graph subgraphs affected by new samples or new knowledge, and the vector embedding representation of the engineering knowledge graph is locally updated. After the iteratively optimized crack identification model, engineering knowledge graph, and large language model are verified offline in a sandbox environment, they are gradually deployed online through A / B testing or Canary release mechanisms, while retaining version control and rollback capabilities.

Images