Tunnel three-dimensional disease intelligent identification system and method based on large model

Abstract

The invention relates to a tunnel three-dimensional disease intelligent identification system and method based on a large model, the system comprises a point cloud data acquisition module and a processor, and the processor comprises a data processing module, a three-dimensional tile optimization module and a disease identification module. The data processing module carries out standardization, noise reduction and registration processing on the received point cloud data; the three-dimensional tile optimization module constructs a multi-level tile pyramid structure based on the registered point cloud data, establishes a mapping relation between a space coordinate and a tile index, compresses tile data based on a curvature point cloud simplification algorithm and adjusts texture quality to form a three-dimensional tile image; calculating a comprehensive score of the tile quality to verify the quality of the three-dimensional tile image; a disease identification module extracts multi-modal fusion features of the preprocessed three-dimensional tile image and geometric features corresponding to disease types; fusing the multi-modal fusion feature and the geometric feature to obtain a joint fusion feature; performing field fine tuning on the joint fusion features; and obtaining a disease identification result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tunnel disease identification, and particularly relates to a tunnel three-dimensional disease intelligent identification system and method based on a large model. BACKGROUND

[0002] With the rapid development of China's transportation infrastructure construction, the number of tunnel projects has increased dramatically, and the safety and durability of tunnel structures have become a major concern. The traditional tunnel disease detection methods mainly have the key problems of low detection efficiency, insufficient detection accuracy, limited detection coverage, insufficient data processing capacity and poor real-time performance. Specifically, the traditional manual detection method requires detection personnel to enter the tunnel for on-site inspection, which has a long detection cycle, low efficiency and safety risks. Manual detection is greatly influenced by subjective factors, and the detection results lack objectivity and consistency, making it difficult to accurately quantify the disease degree. Manual detection is difficult to achieve full-section and full-coverage detection, and there are detection blind spots. Traditional methods lack intelligent analysis and processing capacity for a large amount of detection data, and it is difficult to form a systematic evaluation report. Traditional detection methods cannot realize real-time monitoring, and it is difficult to discover and handle sudden diseases in time.

[0003] At present, the tunnel disease detection technology mainly includes traditional manual detection, image processing-based detection method, deep learning-based detection method and three-dimensional laser scanning technology. Traditional manual detection relies on the experience and judgment of detection personnel, and the detection result is highly subjective and low in efficiency; the image processing-based detection method uses computer vision technology to analyze tunnel images, but lacks understanding of three-dimensional spatial information; the deep learning-based detection method has made some progress in certain specific disease identification, but lacks comprehensive understanding ability for complex scenes; the three-dimensional laser scanning technology can obtain tunnel three-dimensional point cloud data, but lacks intelligent disease identification and analysis ability. These existing technologies all have their own limitations and cannot meet the demand of modern tunnel engineering for efficient and intelligent detection.

[0004] CN120070333A discloses a rail transit tunnel disease detection method based on multi-source data features, which comprises: acquiring global visible light images, infrared images and laser radar point cloud data of multi-period rail transit tunnels; according to the visible light images, an image recognition algorithm is used to identify suspected water leakage areas and suspected pipe piece misalignment areas in the tunnel; based on the infrared images, temperature difference judgment is performed to refine the suspected water leakage areas, and the clear water leakage areas are mapped to the visible light images; according to the water leakage areas in the multi-period visible light images, the water leakage development trend is judged; according to the laser radar point cloud data, the tunnel contour deformation, ground subsidence, multi-period tunnel deformation and rail component installation are identified, the water accumulation in the tunnel is determined, and the pipe piece misalignment information is obtained in combination with the suspected pipe piece misalignment areas; and according to the multi-disease detection results, a more comprehensive rail transit tunnel safety evaluation is realized.

[0005] CN119625713A discloses a laser-based tunnel detection and early warning system, which comprises a data acquisition module, a data preprocessing module, a feature extraction and disease detection module, and a disease evaluation and early warning module; the data acquisition module uses a laser scanner to emit a laser beam, which is reflected back after irradiating the surface of a target object such as a tunnel wall, to determine the distance from the scanner to the surface of the target object; the laser scanner scans in the horizontal and vertical directions to obtain three-dimensional point cloud data of the inner surface of the tunnel; due to the influence of environmental noise during laser scanning, noise points may be contained in the collected point cloud data, and statistical filtering and other methods are used to remove these noise points and statistically filter the local statistical characteristics based on the point cloud data.

[0006] The above two tunnel disease detection methods are both based on point cloud data for detection. The defects of such detection include: lack of multi-source information fusion and mutual verification capability, difficult to fully reflect the multi-dimensional characteristics of tunnel diseases. Relying only on point cloud data can obtain geometric deformation information, but cannot effectively identify the material and temperature abnormal characteristics of apparent diseases such as water leakage, cracks and spalling, and is prone to misjudgment in complex lighting or shielding environments; at the same time, point cloud data itself is not sensitive to subtle diseases, the resolution is limited by the accuracy of the equipment, and it is difficult to realize fine identification of early diseases. In addition, the existing methods generally lack intelligent evolution analysis capability for multi-time series data, and cannot accurately judge the development trend of the disease, which restricts the accuracy of the detection result and the timeliness of the early warning. Therefore, there is an urgent need for a tunnel disease detection method that fuses multi-source heterogeneous data and has intelligent identification and comprehensive analysis capabilities to improve the comprehensiveness, accuracy and intelligent level of detection.

[0007] In view of the defects of the prior art, the present application provides a tunnel three-dimensional disease intelligent identification system and method based on a large model, which adds three-dimensional tile technology to solve the problems of low detection efficiency and limited coverage range in the prior art, and realizes intelligent and automatic detection and intelligent evaluation of tunnel diseases. SUMMARY

[0008] Currently, the detection means of tunnel diseases can be roughly divided into four categories: traditional manual detection method, image processing detection method, deep learning detection method and three-dimensional laser scanning technology. Among them, the traditional manual detection mainly relies on the practical experience and subjective judgment of the detection personnel, not only the objectivity of the detection result is difficult to guarantee, but also the overall detection efficiency is relatively low; the image processing detection method analyzes the tunnel related images with the help of computer vision technology, but this method has obvious shortcomings in the perception and understanding of three-dimensional space information; the deep learning detection method has made certain breakthroughs in the identification of some specific types of diseases, but it still does not have the ability to comprehensively and comprehensively interpret complex scenes; the three-dimensional laser scanning technology can successfully collect three-dimensional point cloud data of the tunnel, but it still has shortcomings in the intelligent identification and deep analysis of diseases. It is not difficult to see that these existing technologies each have their own shortcomings and cannot meet the actual needs of modern tunnel engineering for efficient and intelligent detection.

[0009] In view of the shortcomings of the prior art, the present application provides a tunnel three-dimensional disease intelligent identification system based on a large model from a first aspect, comprising a point cloud data acquisition module and a processor, the processor comprising a data processing module, a three-dimensional tile optimization module and a disease identification module. The data processing module standardizes, denoises and registers the received point cloud data; the three-dimensional tile optimization module constructs a multi-level tile pyramid structure based on the registered point cloud data, establishes a mapping relationship between the spatial coordinates and the tile index, compresses the tile data and adjusts the texture quality based on the curvature of the point cloud simplification algorithm to form a three-dimensional tile image, and calculates the comprehensive score of the tile quality to verify the quality of the three-dimensional tile image; the disease identification module extracts the multi-modal fusion features of the preprocessed three-dimensional tile image and the geometric features corresponding to the disease type; the multi-modal fusion features and the geometric features are fused to obtain joint fusion features; the joint fusion features are fine-tuned based on the domain fine-tuning model; and the recognition result of the disease is obtained. The present application overcomes the defects of low efficiency, weak three-dimensional perception and insufficient intelligence of traditional methods through point cloud data acquisition and multi-module collaborative processing, realizes efficient, accurate and objective identification of tunnel diseases, and improves the automation and intelligence level of detection.

[0010] The tunnel three-dimensional disease intelligent identification system and method based on a large model of the present application realizes intelligent and automatic detection and evaluation of tunnel diseases by combining three-dimensional laser scanning technology with a visual large language model. The main innovations of the present application include: for the first time, three-dimensional point cloud technology is integrated with a visual large language model to realize deep integration of multi-modal information; a complete intelligent detection process is established, with high-efficiency disease identification capability. In terms of application value, the present application can significantly improve tunnel detection efficiency, reduce detection cost and safety risk, provide a scientific basis for tunnel safety management, and thus promote the intelligent development of tunnel detection technology.

[0011] The system further comprises a confidence assessment module, a verification module, an optimization module, and a report generation module. The confidence assessment module is used for confidence assessment of the identification result; the verification module is used for multi-scale identification result verification; the optimization module is used for processing optimization of the identification result; and the report generation module is used for integrating detection result data to generate an intelligent report.

[0012] Through confidence assessment, multi-scale verification, and result optimization, the reliability and accuracy of the identification result are improved, and an intelligent report is automatically generated, realizing a closed loop of the detection process and solving the problem of single result output and lack of credibility quantification in the prior art. According to one preferred embodiment, the data processing steps of the data processing module include: converting the received point cloud data into point cloud data in a standard format; performing noise reduction processing on the point cloud data in the standard format based on a statistical filtering algorithm to remove abnormal points that do not conform to statistical distribution; and performing point cloud registration based on an ICP algorithm to generate a coordinate conversion matrix.

[0013] Through format standardization, statistical filtering noise reduction, and ICP registration, point cloud noise and coordinate deviation are effectively eliminated, data consistency and quality are improved, and the technical defect of inaccurate identification caused by disordered original point cloud data and large errors is solved.

[0014] According to one preferred embodiment, the steps of constructing a multi-level tile pyramid structure by the three-dimensional tile optimization module based on the registered point cloud data include: calculating the spatial range based on the point cloud bounding box, setting the target tile size to a preset size, calculating the number of tiles and generating a tile grid, and adaptively adjusting based on the point cloud density, so as to realize adaptive grid division of the tiles; the 0th level of the multi-level tile pyramid is the top layer, and the resolution of each level of tiles is gradually increased from the 0th level; or, the resolution of each level of tiles is gradually decreased from the maximum level. Through adaptive grid division and multi-level tile pyramid construction, the present application realizes efficient hierarchical management and multi-resolution expression of point cloud data, and solves the problems of low processing efficiency of large-scale three-dimensional data and difficulty in balancing global and local details.

[0015] According to a preferred embodiment, the multi-level tile pyramid is a 5-level tile pyramid, and the 0th level is the top layer; the 0th level provides overview information, and the 5th level provides the highest precision details; the resolution of each level tile is halved from the 5th level.

[0016] The present application uses a 5-level progressive resolution tile structure to gradually refine from overview to details, supports on-demand loading and multi-scale analysis, significantly improves system response speed and disease identification accuracy, and meets the efficient detection needs of complex tunnel scenes.

[0017] According to a preferred embodiment, the step of establishing a mapping relationship between the spatial coordinates and the tile index by the three-dimensional tile optimization module comprises: calculating the tile index position of the point in the three-dimensional space in the multi-level tile pyramid, and generating a unique tile ID based on the tile index position; the tile ID adopts an encoding format of "level x 1000000 + row number x 1000 + column number", realizing the mapping from the spatial coordinates to the tile index, thereby establishing a unique correspondence between the level, row, column and spatial position.

[0018] The present application establishes a unique mapping between the spatial coordinates and the tile ID through level-row-column encoding, realizes fast positioning and efficient retrieval of point cloud data, and solves the technical bottleneck of three-dimensional data index confusion and low access efficiency.

[0019] According to a preferred embodiment, the step of extracting multi-modal fusion features of the three-dimensional tile image by the disease identification module comprises: receiving the preprocessed three-dimensional tile image and the disease description text, extracting the visual feature vector and the text feature vector, and outputting the multi-modal fusion features.

[0020] Fusion of visual and text semantic features enhances the model's contextual understanding ability of diseases, solves the semantic missing problem of pure image methods, and improves the recognition accuracy and interpretability of disease types in complex environments.

[0021] According to a preferred embodiment, the step of extracting geometric features corresponding to the disease type of the three-dimensional tile image by the disease identification module comprises: determining the crack direction based on principal component analysis; calculating the crack length based on the minimum spanning tree algorithm; calculating the curvature based on local geometric changes; calculating the water seepage area based on connected component analysis; calculating the boundary complexity based on the edge detection algorithm; and calculating the concrete spalling area based on the morphological operation algorithm.

[0022] Through algorithms such as principal component analysis and minimum spanning tree, the geometric features such as cracks, water seepage and spalling are accurately quantified, the three-dimensional information deficiency of image method and the deep learning feature black box problem are solved, and the objectivity and accuracy of identification are improved.

[0023] According to a preferred embodiment, the step of fusing the multi-modal fusion features and the geometric features by the disease identification module comprises: mapping all the geometric features to the same dimension as the multi-modal fusion features through a fully connected layer, and fusing the features based on a weighted average formula; the weighted average formula is: fusion features = W A x multi-modal fusion features + W B x geometric features; wherein, W A represents the weight of the multi-modal fusion features, W B represents the weight of the geometric features.

[0024] The weighted fusion of the multi-modal semantic features and the geometric morphological features fully utilizes the advantages of data and knowledge double driving, and improves the comprehensive discrimination ability of the model in complex scenes, and solves the problem of poor robustness of single feature recognition.

[0025] The application provides a tunnel three-dimensional disease intelligent identification method based on a large model from a second aspect, and the method comprises the following steps: standardizing, denoising and registering received point cloud data; constructing a multi-level tile pyramid structure based on the registered point cloud data, establishing a mapping relationship between a spatial coordinate and a tile index, compressing tile data and adjusting texture quality based on a curvature-based point cloud simplification algorithm to form a three-dimensional tile image, and calculating a comprehensive score of tile quality to check the quality of the three-dimensional tile image; extracting multi-modal fusion features of the preprocessed three-dimensional tile image and geometric features corresponding to a disease type; fusing the multi-modal fusion features and the geometric features to obtain joint fusion features; performing domain fine-tuning on the joint fusion features based on a domain fine-tuning model; and obtaining a disease recognition result.

[0026] The method overcomes the defects of weak three-dimensional perception of the image method, insufficient generalization of deep learning and lack of intelligence of laser scanning, and realizes high-precision, interpretable and full-space coverage intelligent identification of tunnel diseases.

[0027] According to a preferred embodiment, the method further comprises: confidence evaluation on the recognition result; and multi-scale recognition result verification.

[0028] The recognition result is processed and optimized, the detection result data is integrated, and an intelligent report is generated. The introduction of the confidence evaluation and the multi-scale verification mechanism enhances the reliability and robustness of the recognition result, and the combination of the result optimization and the intelligent report generation solves the problems of untrusted result and single output form of the existing method, and realizes the closed-loop intelligentization of the detection process. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a simplified module connection relationship schematic diagram of a tunnel three-dimensional disease intelligent identification system based on a large model provided by the application;

[0030] Figure 2 is a logical schematic diagram of a tunnel three-dimensional disease intelligent identification system based on a large model provided by the application;

[0031] Figure 3 is a flow schematic diagram of a tunnel three-dimensional disease intelligent identification method based on a large model provided by the application;

[0032] Figure 4 is a first schematic diagram of an appearance of a tunnel movable three-dimensional laser scanning device provided by the application;

[0033] Figure 5 is a second schematic diagram of an appearance of a tunnel movable three-dimensional laser scanning device provided by the application;

[0034] Figure 6 is a flow schematic diagram of three-dimensional point cloud visualization provided by the application;

[0035] Figure 7 is a three-dimensional point cloud image provided by the application;

[0036] Figure 8 is a system framework schematic diagram of a tunnel three-dimensional disease intelligent identification system based on a large model provided by the application;

[0037] Figure 9 is a distributed database framework schematic diagram of a tunnel three-dimensional disease intelligent identification system based on a large model provided by the application;

[0038] Figure 10 is a tunnel concrete spalling disease identification image provided by the application;

[0039] Figure 11 is a tunnel water seepage disease identification image provided by the application;

[0040] Figure 12 is a tunnel wall crack disease identification image provided by the application;

[0041] Figure 13 is an example diagram of a tunnel three-dimensional structure detection report provided by the application;

[0042] Figure 14 is a collection flow schematic diagram of a tunnel movable three-dimensional laser scanning device provided by the application.

[0043] LIST OF REFERENCE NUMERALS

[0044] 100: point cloud data acquisition module; 200: processor; 210: data processing module; 211: standardization module; 212: noise reduction module; 213: registration module; 220: three-dimensional tile optimization module; 221: construction module; 222: mapping module; 223: compression module; 224: texture adjustment module; 225: evaluation module; 230: disease identification module; 231: first feature extraction module; 232: second feature extraction module; 233: fusion module; 234: domain fine-tuning module; 235: classification module; 400: confidence evaluation module; 500: verification module; 600: optimization module; 700: report generation module. DETAILED DESCRIPTION

[0045] The following will be described in detail with reference to the accompanying drawings.

[0046] In the field of tunnel structure health assessment, the current detection methods include manual visual inspection, computer vision analysis, deep learning algorithm application, and three-dimensional laser scanning technology. Manual visual inspection mainly relies on the personal experience of on-site engineers, and the results are significantly affected by subjective factors, and the operation period is long and the efficiency is low; computer vision analysis can process tunnel surface image data, but cannot effectively analyze three-dimensional spatial structure characteristics; deep learning models have certain effect in identifying specific diseases (such as cracks or water seepage points), but it is difficult to cope with the complex and variable changes in the tunnel environment; three-dimensional laser scanning technology can generate high-precision tunnel point cloud models, but lacks automatic disease classification and intelligent diagnosis functions. These methods each have technical bottlenecks and are difficult to meet the high standard requirements of real-time and automated detection in modern tunnel engineering.

[0047] Therefore, the present application provides a tunnel three-dimensional disease intelligent identification system and method based on a large model, which is used to improve the intelligent identification of tunnel diseases. The present application can also provide a processor 200 for running the tunnel three-dimensional disease intelligent identification method of the present application. The present application can also provide an electronic device including a terminal device and a processor 200 for implementing the tunnel three-dimensional disease intelligent identification system and method of the present application. The present application can also provide a terminal device connected with a processor 200 capable of running a tunnel three-dimensional disease intelligent identification system. The present application can also provide a storage medium storing an encoding program of a tunnel three-dimensional disease intelligent identification system based on a large model. The storage medium is connected with the processor 200, and is used to call the encoding program of the tunnel three-dimensional disease intelligent identification system based on a large model by the processor 200, so as to run the tunnel three-dimensional disease intelligent identification system based on a large model of the present application.

[0048] Embodiment 1

[0049] In view of the deficiencies of the prior art, the present application provides a tunnel three-dimensional disease intelligent identification system based on a large model, comprising a point cloud data acquisition module 100 and a processor 200. The point cloud data acquisition module 100 and the processor 200 realize communication in a wired or wireless manner.

[0050] Preferably, the point cloud data acquisition module 100 is preferably a mobile three-dimensional laser scanning device. Preferably, the point cloud data acquisition module 100 is not limited to one, but can also be two or even more.

[0051] The traditional manual detection method has the defects of low efficiency, limited coverage range, etc., and the fixed detection equipment is difficult to realize the full-face detection requirement of the tunnel. Therefore, the present scheme adopts a mobile three-dimensional laser scanning device as the point cloud data acquisition module 100, and implements full-coverage scanning through a multi-station layout strategy to ensure blind area detection and effectively avoid the omission of key disease information. Figure 4 And Figure 5 A schematic view of two angles of the same three-dimensional laser scanning device is shown.

[0052] Preferably, in the three-dimensional scanning detection of the tunnel structure, the present application scientifically lays out multiple scanning detection stations in the tunnel to ensure that the detection area is covered without omission. Each scanning detection station performs 360-degree full-circle scanning to collect complete and high-precision three-dimensional point cloud data (referred to as point cloud data). Based on target point positioning, the point cloud data between multiple stations is accurately registered and fused. Figure 7 A three-dimensional point cloud image is shown.

[0053] Preferably, the present application adopts a high-frequency, high-precision, intelligent laser sensor system, which directs the laser beam to the inner wall surface of the tunnel through the laser emitter, captures the reflected signal in real time through the receiver, and generates high-precision three-dimensional point cloud data after signal processing by the recording unit.

[0054] As Figure 14 shown, the collection process of the tunnel mobile three-dimensional laser scanning device is as follows.

[0055] System startup phase: laser emitter initialization, scanning device initialization, receiver initialization, and recording unit initialization.

[0056] Start scanning phase: the laser emitter generates a laser beam, and the scanning device controls the direction of the laser beam. The laser beam irradiates the inner wall surface of the tunnel. It is judged whether the laser beam covers the target area, if not, the scanning angle is adjusted. If yes, the laser beam covers the target area according to the preset scanning mode. The target surface reflects the laser beam. The receiver receives the reflected laser beam. The recording unit receives the signal.

[0057] Signal preprocessing stage: signal preprocessing, distance calculation, coordinate conversion, and point cloud data generation. Determine whether the scanning is complete. If not, move to the next scanning position and return to the step of generating a laser beam by the laser emitter.

[0058] Data post-processing stage: point cloud data optimization, then data quality inspection, determine whether the data quality is qualified. If not, rescan the problem area and return to the step of generating a laser beam by the laser emitter. If yes, generate high-precision three-dimensional point cloud data, then data storage, and the acquisition process is completed.

[0059] Compared with traditional photogrammetry and ultrasonic detection technology, laser sensors have higher measurement accuracy and stronger environmental adaptability. Laser as a detection medium does not depend on external light conditions and can achieve stable work in all weather, especially suitable for long-term monitoring of underground tunnels with insufficient light and complex environment. In addition, intelligent sensors have self-adaptive parameter adjustment function, which can dynamically optimize working parameters according to the actual environmental changes in the tunnel (such as temperature, humidity, dust concentration, etc.), ensuring reliable detection performance under different working conditions.

[0060] Preferably, the processor 200 includes a data processing module 210, a three-dimensional tile optimization module 220, and a disease identification module 230. The data processing module 210 standardizes, denoises, and registers the received point cloud data.

[0061] Preferably, as shown in Figure 1 , the data processing module 210 includes a standardization module 211, a denoising module 212, and a registration module 213. The standardization module 211 is connected with the denoising module 212 and sends the point cloud data in a standard format to the denoising module 212. The denoising module 212 is connected with the registration module 213 and sends the denoised point cloud module to the registration module 213.

[0062] As shown in Figure 2 , the data processing steps of the data processing module 210 include: the standardization module 211 converts the received point cloud data into point cloud data in a standard format; the denoising module 212 performs denoising processing on the point cloud data in a standard format based on a statistical filtering algorithm to remove abnormal points that do not conform to statistical distribution; and the registration module 213 performs point cloud registration based on an ICP algorithm to generate a coordinate conversion matrix.

[0063] Preferably, as shown in Figure 1As shown, the 3D tile optimization module 220 includes a construction module 221, a mapping module 222, a compression module 223, a texture adjustment module 224, and an evaluation module 225. The construction module 221 connects to the mapping module 222 and sends the multi-level tile pyramid structure to the mapping module 222. The mapping module 222 connects to the compression module 223 and sends the mapping relationship between spatial coordinates and tile indices to the compression module 223. The compression module 223 connects to the texture adjustment module 224 and sends compressed tile data to the texture adjustment module 224. The texture adjustment module 224 connects to the evaluation module 225 and sends the texture-optimized 3D tile image to the evaluation module 225 to calculate a comprehensive score for tile quality.

[0064] like Figure 2 The 3D tile optimization module 220 shown constructs a multi-level tile pyramid structure based on the registered point cloud data, establishes a mapping relationship between spatial coordinates and tile indices, compresses the tile data based on a curvature-based point cloud simplification algorithm, and adjusts the texture quality to form a 3D tile image. It then calculates a comprehensive score for the tile quality to verify the quality of the 3D tile image.

[0065] Preferably, such as Figure 1 As shown, the disease identification module 230 includes a first feature extraction module 231, a second feature extraction module 232, a fusion module 233, and a neighborhood fine-tuning module 234. The first feature extraction module 231 is connected to the fusion module 233 and sends the multimodal fused features to the fusion module 233. The second feature extraction module 232 is connected to the fusion module 233 and sends the geometric features to the fusion module 233. The fusion module 233 is connected to the neighborhood fine-tuning module 234 and sends the joint fused features to the neighborhood fine-tuning module 234.

[0066] like Figure 2 As shown, the disease identification module 230 extracts the multimodal fusion features and geometric features corresponding to the disease type from the preprocessed three-dimensional tile image; it fuses the multimodal fusion features with the geometric features to obtain joint fusion features; it then performs domain fine-tuning on the joint fusion features based on the domain fine-tuning model to obtain the disease identification result.

[0067] Preferably, such as Figure 3 As shown, the processing steps of the processor 200 for point cloud data are as follows.

[0068] S100: The data processing module 210 performs standardization, noise reduction, and registration processing on the received point cloud data.

[0069] S110: Standardization module 211 converts the received point cloud data into point cloud data in a standard format.

[0070] The data formats output by laser scanning devices of different manufacturers are different. If the original format is directly used, compatibility problems will be caused for subsequent data processing and system integration. To ensure the universality and interchangeability of the data, the standardization module 211 uses the.las / .laz format as the standard storage format. This format has been widely adopted as an industry standard for point cloud data and has good software compatibility and support for rich metadata.

[0071] The standardization module 211 uniformly converts the originally collected point cloud data into the.las / .laz standard format. Preferably, the spatial resolution is set to 1-5 mm to meet the accuracy requirements of fine detection of tunnel diseases. The amount of data generated per kilometer of tunnel is about 1-2 GB; the coordinate system uses the UTM (Universal Transverse Mercator) projection coordinate system to ensure the spatial positioning accuracy and consistency of the point cloud data in a large range.

[0072] S120: The noise reduction module 212 performs noise reduction processing on the point cloud data in the standard format based on a statistical filtering algorithm to remove abnormal points that do not conform to the statistical distribution.

[0073] During laser scanning, due to environmental interference, sensor noise, and multi-path reflection, the collected point cloud data often contains a large number of noise points. Such abnormal points can significantly reduce the accuracy of subsequent disease identification. The statistical filtering algorithm has good edge preservation characteristics and can effectively remove outliers while maximizing the preservation of the geometric structure characteristics of the point cloud.

[0074] The noise reduction module 212 uses a statistical filtering algorithm to perform noise reduction processing on the point cloud data. Specifically, the mean_k parameter of the neighborhood point number is set to 50, and the std_dev_mult threshold of the standard deviation multiple is set to 1.0. By analyzing the distance statistical distribution of each point and its neighborhood points, abnormal points deviating from the normal distribution are identified and removed, thereby suppressing noise while ensuring the integrity of the overall structure of the point cloud.

[0075] S130: The registration module 213 performs point cloud registration based on the ICP algorithm to generate a coordinate transformation matrix.

[0076] The calculation and evaluation of point cloud registration error is a key link to ensure the accuracy of multi-station data fusion. Only when the registration error is controlled within a reasonable range, the subsequent disease identification and spatial analysis results have reliability. Since the point cloud data obtained by multi-station scanning is located in different local coordinate systems, if accurate registration is not performed, direct merging will cause spatial position deviation, affecting the continuity and accuracy of the overall geometric structure.

[0077] ICP (Iterative Closest Point) algorithm, as the most mature and high-precision point cloud registration method at present, has good convergence performance and registration stability, and can effectively realize high-precision spatial alignment. Therefore, the ICP algorithm is selected for point cloud data registration in this step.

[0078] The registration module 213 uses the ICP algorithm to register the point clouds of adjacent stations, and minimizes the distance error between the corresponding point pairs through iterative optimization. The registration error is calculated in the form of root mean square error (RMSE), and the formula is:

[0079] Specifically,

[0080] In the above formula, p i and q i are matching point pairs, and n is the number of point pairs.

[0081] The ICP algorithm finally generates a unified coordinate transformation matrix, including a rotation matrix, a translation vector, and an optional scaling factor, to realize high-precision alignment of point clouds in multiple stations in a unified coordinate system.

[0082] S200: The three-dimensional tile optimization module 220 processes the coordinate transformation matrix to form a three-dimensional tile image and performs evaluation.

[0083] As shown in Figure 6 , the point cloud tile data generation stage includes inputting point cloud data into SuperMapIDeskTop, and then outputting three-dimensional tile data (three-dimensional tile image). In the point cloud tile publishing stage, the three-dimensional tile data (three-dimensional tile image) is input into SuperMapiServer. In the point cloud visualization stage, SuperMapiServer sends the optimized three-dimensional tile image to the browser, and the browser displays the optimized three-dimensional tile image of the three-dimensional disease.

[0084] S210: The construction module 221 constructs a multi-level tile pyramid structure based on the registered point cloud data.

[0085] The point cloud data obtained by tunnel three-dimensional laser scanning is large in scale, usually containing hundreds of millions to hundreds of millions of points, and direct storage and processing can easily cause memory overflow, resulting in significant decrease in calculation efficiency. Traditional data processing methods are difficult to efficiently cope with the management and analysis needs of TB-level three-dimensional point cloud data. Therefore, the three-dimensional tile technology can effectively solve this problem: this technology divides massive point cloud data into multiple logically independent and appropriately sized data units (i.e. "tiles") according to a spatial division strategy, each tile carries an appropriate amount of point cloud, significantly reducing the amount of data loaded at a time, improving memory utilization, and supporting parallel processing and distributed computing.

[0086] In addition, the three-dimensional tile technology has a special effect on large model-driven intelligent disease identification. By constructing a standardized tile structure, raw point cloud data of different sources, uneven density, and different precision can be unified into standardized data units with consistent spatial size, resolution, and organization format, thereby providing input data with uniform format and standardized structure for deep learning models, effectively reducing recognition bias caused by data heterogeneity, and improving the generalization ability and recognition stability of the model.

[0087] The construction module 221 adopts the SuperMap S3M three-dimensional tile technology to construct a multi-level tile pyramid structure. This technology is compatible with international mainstream standards such as OSGB and 3D-Tiles, and combines with the WebGL hardware accelerated rendering mechanism, having good cross-platform compatibility and visualization performance.

[0088] The multi-level tile pyramid structure supports LOD (Level of Detail) dynamic loading, realizing multi-scale data expression from macro to micro. At the same time, a tile quality evaluation system is established to quantitatively evaluate the spatial division rationality, data integrity, and geometric fidelity of the tile, ensuring that no key structural information is lost during the tiling process. Finally, an efficient, stable, and standardized three-dimensional data service architecture is formed, providing reliable data support for subsequent large model training and reasoning.

[0089] S220: The construction module 221 performs hierarchical division on the multi-level tile pyramid structure based on an adaptive grid division strategy.

[0090] Traditional fixed grid division strategies are difficult to adapt to the density differences and structural complexities of point cloud data in spatial distribution, which can easily lead to insufficient information in sparse areas or redundant data in dense areas. Therefore, an adaptive grid division method is used, which can dynamically adjust the division scale of the tile according to the local point cloud density, ensuring that each tile contains sufficient and balanced geometric feature information, improving the rationality of data organization and the reliability of subsequent analysis.

[0091] Through a large number of experiments, it is verified that a tile size of 1.0m x 1.0m x 2.0m can achieve an optimal balance between data processing efficiency and feature integrity: this size can effectively control the data volume of a single tile, avoiding memory pressure, and is sufficient to retain key geometric details of the tunnel structure (such as cracks, deformations, etc.), meeting the needs of high-precision disease identification.

[0092] The construction module 221 calculates the spatial range based on the point cloud bounding box, sets the target tile size to the preset size, calculates the number of tiles and generates the tile grid, and performs adaptive adjustment based on the point cloud density, thereby realizing adaptive grid division of the tile.

[0093] Due to the difference in the requirement of detail accuracy of the tunnel point cloud data at different observation distances, a single fixed resolution cannot meet the application requirement of multi-scale analysis. To realize efficient and flexible data expression, a tile pyramid structure is adopted, a multi-level resolution hierarchy is constructed, and multi-level data representation from coarse to fine is provided. The structure supports the LOD (Level of Detail) mechanism, and can dynamically load tile data of corresponding accuracy according to the viewing distance or analysis requirement, thereby significantly improving the rendering efficiency and system response speed while ensuring the visualization quality.

[0094] Preferably, the 0th level of the multi-level tile pyramid constructed by the construction module 221 is the top layer, and the resolution of each level tile is gradually increased from the 0th level. Alternatively, the resolution of each level tile is gradually decreased from the maximum level.

[0095] Further preferably, the multi-level tile pyramid is a 5-level tile pyramid, and the 0th level is the top layer. The 0th level provides overview information, and the 5th level provides the highest accuracy details. The resolution of each level tile is halved from the 5th level. The resolution reduction strategy (halving each time) can ensure smooth transition between different levels and avoid visual jumps. Further preferably, from the 0th level to the 5th level, the resolutions of each level are 5 mm, 2.5 mm, 1.25 mm, 0.625 mm and 0.3125 mm, respectively, supporting LOD (Level of Detail) dynamic adjustment.

[0096] S230: The mapping module 222 establishes the mapping relationship between the spatial coordinates and the tile index.

[0097] S231: The mapping module 222 calculates the tile index position of the point in the three-dimensional space in the multi-level tile pyramid. The calculation formula of the tile index position is:

[0098]

[0099] S232: A unique tile ID is generated based on the tile index position.

[0100] Preferably, the tile ID adopts the encoding format of “level × 1000000 + row number × 1000 + column number”, realizing the mapping from the spatial coordinates to the tile index, thereby establishing the unique correspondence between the level, row, column and the spatial position.

[0101] Massive tile data needs an efficient indexing mechanism to support fast positioning and access. Through spatial indexing, three-dimensional spatial coordinates can be mapped to the corresponding tile index, achieving O(1) time complexity of tile lookup and access. The tile ID adopts a unique coding design, with a format of "level x 1000000 + row number x 1000 + column number", which not only ensures the expansion capacity of accommodating large-scale tiles under a multi-level structure, but also takes into account the readability and parsing efficiency of the ID. This indexing mechanism provides basic support for subsequent spatial analysis operations, enabling efficient implementation of common spatial operations such as range query and neighborhood search.

[0102] S240: The compression module 223 compresses the tile data based on the curvature-based point cloud simplification algorithm.

[0103] The amount of original point cloud data is extremely large, and direct storage and transmission will consume a large amount of resources. Using a curvature-based point cloud simplification algorithm can effectively preserve important geometric features while reducing data volume, and this algorithm performs better in feature preservation than other simplification methods.

[0104] The compression module 223 compresses the tile data based on the curvature-based point cloud simplification algorithm. Preferably, the compression ratio is set to 0.3, i.e., 30% of the key points are retained. This can maintain key geometric features while reducing data volume.

[0105] The compression ratio is determined according to the specific accuracy requirements, and this ratio can achieve an ideal balance between significantly reducing data volume and the required accuracy. This can significantly reduce the storage and transmission pressure, and ensure that the key features of the point cloud data are completely preserved.

[0106] S250: The texture module adjusts the texture quality of the tile.

[0107] Large models have strict requirements on the size and format of input data, and different resolutions of textures may lead to unstable recognition results. Choosing 512x512 pixels as the uniform resolution is based on the consideration of model input specifications, which can meet the recognition accuracy requirements and effectively control the computational complexity. In the resampling process, the Lanczos interpolation algorithm is used because it can better preserve image details and effectively reduce blurring and artifacts compared to other interpolation methods. In addition, through texture quality assessment, the optimized texture is ensured to meet the model recognition requirements.

[0108] Preferably, the texture module uniformly resamples all textures to 512x512 pixels, uses the Lanczos interpolation algorithm to maintain image quality, and performs texture quality assessment and necessary optimization to ensure the consistency and effectiveness of the input.

[0109] S260: The evaluation module 225 performs quality evaluation and verification of the tile data.

[0110] The quality of the tiling process directly affects the accuracy of subsequent identification, so establishing a scientific quality evaluation system is a key link to ensure the processing effect. The three indicators of compression ratio, geometric accuracy and texture quality can comprehensively reflect the overall quality of the tile. The weight distribution (compression ratio 30%, geometric accuracy 40%, texture quality 30%) comprehensively considers the influence degree of each indicator on the identification accuracy, among which the geometric accuracy is given a higher weight because it plays a more critical role in disease identification. By setting a quality threshold, only high-quality data that meets the standard can enter the subsequent processing flow.

[0111] The evaluation module 225 builds a comprehensive quality evaluation system including compression ratio (30%), geometric accuracy (40%) and texture quality (30%), and calculates the comprehensive score by weighting to quantitatively evaluate the quality of the tiling process and ensure the reliability and consistency of the output results.

[0112] The calculation formula of the comprehensive score is:

[0113] Preferably, Q = W c ·C + W g ·G + W t ·T.

[0114] In the above formula, Q represents the comprehensive score, with a value range of [0, 1], which is used to judge whether the tile meets the quality threshold for entering the subsequent process (for example, if the threshold is set to 0.7, then Q ≥ 0.7 is qualified). C represents the compression ratio score. G represents the geometric accuracy score. T represents the texture quality score. W c represents the weight of the compression ratio; W g represents the weight of the geometric accuracy; W t represents the weight of the texture quality.

[0115] If the compression ratio score of a tile is 0.8, the geometric accuracy score is 0.9, and the texture quality score is 0.7, then its comprehensive score is: Q = 0.3 × 0.8 + 0.4 × 0.9 + 0.3 × 0.7 = 0.24 + 0.36 + 0.21 = 0.81.

[0116] If the quality threshold is set to 0.7, then the tile passes the quality evaluation.

[0117] S300: The disease identification module 230 extracts the multi-modal fusion features of the pre-processed three-dimensional tile image and the geometric features corresponding to the disease types; and fuses the multi-modal fusion features and the geometric features to obtain joint fusion features.

[0118] Preferably, the disease identification module 230 contains a disease identification model. Preferably, the disease identification model is a 14-parameter QwenVL-Max model, using the PyTorch 2.1.0 inference framework, configured with an NVIDIA RTX 4090 hardware environment, and employing 8-bit quantization to optimize memory usage.

[0119] Compared to traditional machine learning methods, large models offer significant advantages in understanding and generalization performance. This approach selects the QwenVL-Max model with 14 bytes of parameters, as its sufficient model capacity effectively captures and understands complex disease features while maintaining a reasonable balance between performance and computational cost, avoiding a significant increase in inference burden due to excessive model size. Training and inference are implemented using the PyTorch 2.1.0 framework, which provides excellent support and efficient optimization capabilities for large models. On the hardware side, an NVIDIA RTX 4090 is used, providing ample computing power to ensure efficient inference for large models. Furthermore, 8-bit quantization technology is introduced to significantly reduce memory usage and improve inference efficiency while maintaining model accuracy as much as possible.

[0120] S310: The first feature extraction module 231 preprocesses the three-dimensional tile image and extracts multimodal fusion features.

[0121] S311: The first feature extraction module 231 preprocesses and standardizes the three-dimensional tile image sent by the texture module.

[0122] The QwenVL-Max model has strict requirements on the format and size of input data. Data from different modalities must undergo unified preprocessing before they can be effectively processed by the model. This technical solution focuses on cracks in tunnel defect detection (see...). Figure 12 ), seepage (see) Figure 11 ), concrete spalling (see) Figure 10 Taking three typical diseases as examples, we conducted multimodal fusion feature extraction and recognition. Visual data needs to be standardized in size and format, geometric data needs to be converted into a representation that the model can understand, and text data needs to be standardized and encoded. Such preprocessing measures help the model efficiently fuse multimodal information and improve recognition accuracy.

[0123] The first feature extraction module 231 receives the preprocessed 3D tile image and the disease description text, and uniformly adjusts the 3D tile image to 512×512 pixels. At the same time, it performs standardization and cleaning on the text description, including removing special characters and unifying the format, to ensure the consistency of the input.

[0124] S312: The first feature extraction module 231 extracts visual feature vectors and text feature vectors, and outputs multimodal fusion features.

[0125] The QwenVL-Max model possesses powerful multimodal understanding capabilities, enabling it to process various types of information simultaneously, including images and text. Compared to traditional hand-designed feature extraction methods, this large model can automatically learn richer and more abstract feature representations, including high-level semantic features and cross-modal correlation features. This end-to-end feature extraction approach effectively overcomes the subjectivity and limitations of hand-designed features, significantly improving feature expressiveness and generalization performance.

[0126] The first feature extraction module 231 receives the preprocessed 512×512 tile image and the standardized disease description text, automatically extracts the corresponding visual feature vectors and text feature vectors based on the QwenVL-Max model, and generates a unified multimodal fusion feature representation through an internal fusion mechanism for subsequent recognition and analysis tasks.

[0127] S3121: The calculation process of the first feature extraction module 231 to extract visual feature vectors is as follows.

[0128] A 512×512 pixel tile image is input into the QwenVL-Max visual encoder. The visual encoder uses the Vision Transformer (ViT) architecture.

[0129] The specific calculation formula is as follows:

[0130] patch emb =Linear(patch) size 2 ×3,d);

[0131]

[0132] patch pos =patch emb +pos emb ;

[0133] V = ViT encoder (patch pos ).

[0134] In the above formula, patch size Indicates the image patch size; d represents the model dimension; pos represents the position index; i represents the dimension index; Represents the position-coded sine component; Represents the position-coded cosine component; patch emb Indicates image patch embedding; patch pos V represents an image patch with location information; V represents a visual feature.

[0135] The shape of the input data for the first feature extraction module 231 is:

[0136] Input image: (512, 512, 3)

[0137] ↓ Block processing

[0138] Image patch: (16,16,32,32,3) → flattened to (256,3072)

[0139] ↓Linear projection

[0140] patch emb :(256,768)

[0141] ↓Location code

[0142] pos emb :(256,768)

[0143] ↓ Add

[0144] patch pos :(256,768)

[0145] ↓ViT Encoder

[0146] V:(256,768)←Visual features

[0147] The following is an example of the calculation of the visual feature vector extracted by the first feature extraction module 231.

[0148] The 512×512 image is divided into 16 small 32×32 pixel patches.

[0149] patch size =32, d=768;

[0150] patch emb =Linear(32 2 ×768)=Linear(3072×768).

[0151] Input: A 32×32×3 = 3072-dimensional vector;

[0152] Output: 768-dimensional feature vector.

[0153]

[0154] pos = 10, i = 0, d = 768;

[0155]

[0156]

[0157] V = ViT encoder (patch pos ).

[0158] The input to the first feature extraction module 231 is a 512×512×3 image.

[0159] Divide into blocks: 16×16=256 32×32 blocks.

[0160] Output of the first feature extraction module 231: V∈R (256×768) .

[0161] S3122: The process of extracting text features by the first feature extraction module 231 is shown below.

[0162] The first feature extraction module 231 performs word segmentation on the disease description text.

[0163] Preferably, the text encoder adopts the Transformer architecture, and the calculation formula is as follows:

[0164] word emb =EMbedding(vocab size ,d);

[0165] T = Transformer encoder (word emb ).

[0166] In the above formula, word emb Indicates word embedding; vocab size T represents the vocabulary size; T represents the text feature vector.

[0167] The shape of the input data for the first feature extraction module 231 is as follows.

[0168] Input text: "This is a 3D point cloud map of a tunnel used for crack detection."

[0169] ↓Tokenization

[0170] token ids :(1,512) ← Fill to the maximum length

[0171] ↓Word embedding

[0172] word emb :(512,768)

[0173] ↓Transformer encoder

[0174] T:(512,768)←Text Features

[0175] S3123: The first feature extraction module 231 performs multimodal feature fusion to obtain multimodal fused features.

[0176] Preferably, a cross-attention mechanism is used to fuse visual vectors and text feature vectors:

[0177] Q = V × W q ;

[0178] K = T × W k ;

[0179] V att =T×W v ;

[0180]

[0181] F = V + att × V att .

[0182] In the above formula, W q W k and W v Let V represent the learnable weight matrix; Q represent the query matrix; K represent the key matrix; and V represent the key matrix. att Represents a value matrix; d k represents the key dimension; at represents the attention weight; F represents the fused feature vector.

[0183] The first feature extraction module 231 receives input data in the following form during this process:

[0184] V:(256,768)

[0185] T:(512,768)

[0186] ↓Calculate attention

[0187] Q = V × W q :(256,64)

[0188] K = T × W k :(512,64)

[0189] V att =T×W v :(512,64)

[0190] ↓Attention Calculation

[0191]

[0192] ↓Feature fusion

[0193] F = V + att × V att :(256,768)←Fused feature vector.

[0194] The following is an example of the calculation of multimodal feature fusion performed by the first feature extraction module 231.

[0195] V∈R (256×768) ,T∈R (512×768) ;

[0196] W q W k W v ∈R (768×64) ;

[0197] Q = V × W q = (256 × 768) × (768 × 64) = (256 × 64);

[0198] K = T × W k = (512 × 768) × (768 × 64) = (512 × 64);

[0199] V att =T×W v = (512 × 768) × (768 × 64) = (512 × 64);

[0200] Q×K T = (256 × 64) × (64 × 512) = (256 × 512);

[0201] d k =64,

[0202] att = softmax( ( 256×512) / 8)=softmax((256×512)).

[0203] S320: The second feature extraction module 232 extracts the geometric features of the three-dimensional tile image corresponding to the disease type.

[0204] Geometric features play a crucial role in the accurate quantification and severity assessment of diseases. Although the QwenVL-Max model is capable of processing visual and textual information, geometric features in point cloud data still require extraction using specialized algorithms. To fully leverage the advantages of different feature types, this solution independently fuses the extracted geometric features with the multimodal fusion features generated by the model, achieving information complementarity, enhancing feature representation capabilities, and thus improving the accuracy of identification and its practical application value.

[0205] According to a preferred embodiment, the second feature extraction module 232 in the defect identification module 230 extracts geometric features of the three-dimensional tile image corresponding to the defect type in the following steps: determining the crack direction based on principal component analysis; calculating the crack length based on the minimum spanning tree algorithm; calculating the curvature based on local geometric changes; calculating the seepage area based on connected component analysis; calculating the boundary complexity based on the edge detection algorithm; and calculating the concrete spalling area based on the morphological operation algorithm.

[0206] S321: The process by which the second feature extraction module 232 calculates the geometric features of the crack is as follows.

[0207] cov=(1 / n)×∑(x i -μ)(x i -μ) T ;

[0208] eigenvals,eigenvecs=eig(cov);

[0209] dir = eigenvecs[0];

[0210]

[0211] In the above formula, cov represents the covariance matrix; n represents the sample size; x i and y i represents the point coordinates; μ represents the mean vector; eigenvals and eigenvecs represent the eigenvalues ​​and eigenvectors, respectively; dir represents the principal direction vector; len represents the crack length; curv represents the curvature; x′ and y′ represent the first derivatives; x″ and y″ represent the second derivatives; and perim represents the perimeter.

[0212] The second feature extraction module 232 calculates the geometric features of the crack as shown in the following example.

[0213] Set n = 100 points. i =[10,20,30],μ=[15,25,35];

[0214] x i -μ = [-5, -5, -5].

[0215] (x i -μ)(x i -μ) T = [-5, -5, -5] × [-5, -5, -5] T =25.

[0216] cov = (1 / 100) × 25 = 0.25.

[0217] S322: The second feature extraction module 232 calculates the geometric features of the area.

[0218] area=∑(conn pixels )×pixel area ;

[0219] comp=perim 2 / (4π×area).

[0220] In the above formula, conn pixels Indicates the number of connected pixels; pixel area represents the area of ​​a single pixel; comp represents the complexity.

[0221] The second feature extraction module 232 calculates the area geometric features as shown in the following example.

[0222] Set conn pixels = 1500 connected pixels.

[0223] pixel area =0.25mm 2 (0.5mm×0.5mm);

[0224] area = 1500 × 0.25 = 375 mm 2 ;

[0225] perim = 50mm;

[0226] comp=50 2 / (4π×375)=2500 / (4×3.14159×375)=0.530.

[0227] S330: The disease identification module 230 fuses multimodal fusion features with geometric features.

[0228] Specifically, the process by which the fusion module 233 fuses multimodal fusion features with geometric features is as follows.

[0229] All geometric features are mapped to the same dimension as the multimodal fusion features through a fully connected layer, and feature fusion is performed based on a weighted average formula.

[0230] The weighted average formula is: Fusion Feature = W A ×Multimodal fusion features + W B ×Geometric features.

[0231] W A W represents the weights of the multimodal fusion features. B This represents the weight of the geometric features. For example, fusion feature = 0.7 × multimodal fusion feature + 0.3 × geometric feature.

[0232] Fusion module 233 maps all geometric features to 768 dimensions through a fully connected layer:

[0233] geom feat =[len,width,area,comp,…];

[0234] mapped = RELU(W×geom) feat +b).

[0235] In the above formula, geom feat "represents the geometric feature vector"; "mapped" represents the mapped feature vector.

[0236] The input data shape of the fusion module 233 is as follows:

[0237] Point cloud data: (N,3)

[0238] ↓Principal Component Analysis

[0239] Covariance matrix: (3,3)

[0240] Eigenvalue: (3,)

[0241] Eigenvector: (3,3)

[0242] Main direction:(3,)

[0243] ↓ Length calculation

[0244] Crack length: (1,) ← scalar

[0245] ↓Curvature Calculation

[0246] Curvature: (N,) ← Curvature at each point

[0247] ↓Area Calculation

[0248] Area of ​​connected region: (1,) ← scalar

[0249] ↓Complexity Calculation

[0250] Boundary complexity: (1,) ← scalar

[0251] ↓Feature combination

[0252] geom_feat:(5,)←[length, width, area, complexity, mean curvature]

[0253] ↓Fully connected layer

[0254] W:(5,768),b:(768,)

[0255] mapped:(768,) ← Mapped to the same dimension as F.

[0256] The feature fusion process of fusion module 233 based on the weighted average formula is as follows.

[0257] final = 0.7 × F + 0.3 × mapped.

[0258] In the above formula, final represents the final fused feature.

[0259] Input data shape:

[0260] F:(256,768)←From S15

[0261] mapped:(768,)←Geometric features

[0262] ↓Broadcast addition

[0263] final:(256,768) ← Joint fusion feature

[0264] S400: The disease identification module 230 will perform domain fine-tuning based on the domain fine-tuning model using the combined and fused features.

[0265] While QwenVL-Max possesses general multimodal capabilities, tunnel scenes exhibit significant domain bias (low illumination, slippery reflections, smoke, and repetitive textures) and marked class imbalance (relatively abundant crack samples, scarce samples of seepage and peeling). Directly using a general model can lead to insufficient recall of minor classes and false positives in the scene.

[0266] The present invention selects the LoRA adapter as the domain fine-tuning module 234 for domain fine-tuning because: under the premise of freezing most of the weights, only the low-rank adaptation parameters of attention and multimodal projection are trained, which avoids catastrophic forgetting and significantly reduces memory usage, making it easier to achieve efficient convergence on a single card / small cluster.

[0267] Preferably, the fine-tuning principle of the domain fine-tuning module 234 is as follows:

[0268] First, a training set including three types of defects—cracks, seepage, and spalling—is constructed based on the tiled 512×512 3D tile images and their geometric attributes. A two-layer annotation method of "main bounding box + optional polygon" is adopted to accommodate different fine-grained annotation requirements; millimeter / pixel ratio and tile ID are recorded for subsequent 3D spatial reprojection. The data is organized in JSONL format, and each sample includes key fields such as image path, tile ID, defect type, bounding box coordinates, polygon coordinates, millimeter / pixel ratio, instruction text, and output result.

[0269] Subsequently, the extracted geometric statistics (such as crack skeleton length, maximum width, seepage connectivity area, spalling area and boundary complexity) are injected into the text prompt as "external evidence" to construct an instruction-answer pair of "image + geometric evidence → structured answer" and output in JSON format to reduce decoding noise caused by free text generation.

[0270] Next, a multi-objective training strategy was designed: with generative cross-entropy loss as the main objective, a class imbalance loss function was introduced to weight the disease types (the lower the frequency of the category, the higher the weight), and a contrastive loss for visual-text alignment was added. The alignment quality of multimodal features was improved by learning through comparison of positive and negative samples.

[0271] In terms of model configuration, the weights of the base model are frozen, and LoRA adapters are injected into the query projection, value projection, output projection, multimodal projection layers, and cross-modal attention layers of the language decoder. Appropriate rank parameters and scaling factors are set, and 4-bit quantization is used to significantly reduce memory usage. During training, only a small number of parameters of the LoRA module and related normalization layers are updated to ensure stable and efficient training.

[0272] For optimization, the AdamW optimizer is used, with appropriate learning rate and weight decay configured. Combined with cosine annealing learning rate scheduling and warm-up strategies, mixed-precision training and gradient pruning are employed. Large-scale training is achieved through gradient accumulation and distributed training. The model is trained for multiple epochs, with early stopping based on validation set performance.

[0273] Finally, the model was validated and evaluated according to the type of disease, and the model with the best overall performance was selected for online deployment.

[0274] Preferably, an example of JSONL format data for geometric features is shown below.

[0275] {

[0276] "image_path":" / tiles / tile_001_512x512.jpg",

[0277] "tile_id":"1001001",

[0278] "disease_type":"crack",

[0279] "bbox":[120,80,200,150],

[0280] "polygon":[[120,80],[200,80],[200,150],[120,150]],

[0281] "mm_pixel_ratio":0.5,

[0282] "instruction":"Please identify the type of tunnel defects in the image",

[0283] "output":"{"type":"crack","confidence":0.95,"area":4000}"

[0284] }

[0285] Preferably, the LoRA adapter is configured as follows.

[0286] LoRA parameter settings:

[0287] rank = 16 / / rank of low-rank decomposition

[0288] alpha = 32 / / Scaling factor

[0289] dropout = 0.1 / / Dropout rate

[0290] The LoRA weights are updated as follows: W = W0 + (B × A) × (α / γ).

[0291] In the above formula, B and A represent low-rank matrices, respectively; α represents the scaling factor; γ represents the rank; W0 represents the original weight matrix; and W represents the updated weight matrix.

[0292] The input data shape of the domain fine-tuning module 234 is as follows.

[0293] Original weights: W0:(768,768);

[0294] LoRA matrix: B:(768,16), A:(16,768)

[0295] ↓Low-rank decomposition

[0296] B×A:(768,768)

[0297] ↓Weight Update

[0298] W = W0 + (B × A) × 2: (768, 768)

[0299] The following is an example of how the domain fine-tuning module 234 calculates weights.

[0300] W0 = 768 × 768.

[0301] Weight matrix:

[0302] B = 768 × 16, A = 16 × 768.

[0303] α = 32, γ = 16.

[0304] B×A=(768×16)×(16×768)=(768×768).

[0305] W=W0+(B×A)×(α / γ)=(768×768)+(768×768)×2.

[0306] Domain fine-tuning module 234 calculates the multi-objective loss function:

[0307] L ce =-∑logP(y i |final);

[0308] w i =1 / freq(y i );

[0309] L imb =-∑w i ×logP(y i );

[0310] sim=v i T ×t i / (‖v i ‖×‖t i ||);

[0311] L cont =-log(exp(sim / t) / ∑exp(sim / τ));

[0312] L total =L ce +0.1×L imb +0.05×L cont ;

[0313] tuned model =train model (W,train data ,L total ).

[0314] In the above formula, y i L represents the i-th predicted label; ce Represents cross-entropy loss; freq(y) i ) indicates category frequency; w i Indicates category weight; L imb Represents the imbalance loss; sim represents the similarity; v i Represents a visual feature vector; t i Represents the text feature vector; τ represents the temperature parameter; L cont Indicates comparative loss; Ltotal Indicates total loss; tuned model This indicates a fine-tuned model.

[0315] The shape of the input data in the step of calculating the multi-objective loss function in the domain fine-tuning module 234 is as follows.

[0316] Predicted probability: (batch_size, num_classes);

[0317] Real label:(batch_size,)

[0318] ↓ Cross-entropy loss

[0319] L ce :(1,)←Scalar

[0320] ↓ Imbalanced loss

[0321] L imb :(1,)←Scalar

[0322] ↓Compare the losses

[0323] L cont :(1,)←Scalar

[0324] Total losses

[0325] L total :(1,)←Scalar

[0326] S500: Disease identification module 230 directly identifies and classifies diseases.

[0327] Compared to the existing "detector + rule" cascaded approach, this invention employs a finely tuned QwenVL-Max model for end-to-end structured recognition, simultaneously fusing visual semantics and geometric evidence to effectively reduce the propagation of boundary errors caused by inter-module segmentation. By introducing JSON pattern constraints and few-sample examples, this invention significantly reduces illusions and formatting errors during the free generation process, improving the usability and auditability of the output results.

[0328] Preferably, the disease identification module 230 further includes a classification module 235.

[0329] The classification module 235 uses a template of "contextual few-shot examples + tile images + geometric evidence" to construct structured prompts, guiding the model to output fixed JSON key-value pairs containing key information such as disease type, bounding box coordinates, confidence level, tile ID, and millimeter-pixel ratio. Specific examples are used to demonstrate the expected output format to ensure that the model understands the task requirements.

[0330] Then, the classification module 235 enables JSON pattern constraints and invalid tag masking, sets appropriate decoding parameters including sampling strategy, maximum generation length, and bundle search size to prevent the generation of free text, and applies tag bias to key enumeration fields to suppress illegal words and ensure the standardization of the output format.

[0331] Next, the classification module 235 pre-caches the visual encoding features and sends them in batches according to the tile ID for the same tunnel segment. It enables key-value caching and tensor parallel technology, and the geometric features are concatenated into the prompt words in the form of natural language "auxiliary parameters" to make full use of multimodal information.

[0332] Finally, the classification module 235 adopts a non-maximum suppression and voting fusion strategy for the output from multiple sampling or multiple prompt word perspectives. It processes overlapping detection boxes through soft non-maximum suppression and sets a consistency voting threshold to ensure the reliability of the results.

[0333] The JSON schema constraint process is as follows.

[0334] Decoding parameter settings:

[0335] max_length = 512

[0336] temperature = 0.7

[0337] top_p = 0.9

[0338] beam_size = 4

[0339] repetition_penalty = 1.1

[0340] Nonmaximum suppression

[0341] pred=tuned model (final);

[0342] overlap=IoU=Area(bbox1∩bbox2) / Area(bbox1∪bbox2);

[0343] s i =s i ×exp(-(IoU 2 / σ));

[0344] filtered output =NMS(structured output ,s i ).

[0345] In the above formula, IoU represents the intersection-union ratio; overlap represents the degree of overlap; and bbox represents the degree of overlap. iRepresents the bounding box i; Area represents the area function; s i σ represents the confidence score of the i-th box; σ represents the soft NMS parameter (0.5).

[0346] The input data for classification module 235 has the following shape.

[0347] The model after fine-tuning: tuned model

[0348] Input: (256,768) ← final from S16

[0349] Output: {

[0350] "type":(1,),←Disease type

[0351] "bbox":(4,),←boundary box

[0352] "confidence":(1,)←Confidence level

[0353] }

[0354] pred: original prediction

[0355] ↓Structured processing

[0356] structured_output:{

[0357] "type":(1,),

[0358] "bbox":(4,),

[0359] "confidence":(1,)

[0360] }

[0361] Multiple bounding boxes: (N,4) ← N bounding boxes

[0362] Confidence score: (N,) ← N confidence scores

[0363] ↓NMS processing

[0364] filtered output :(M,4) ← M reserved bounding boxes, M≤N

[0365] The calculation example for classification module 235 is as follows.

[0366] bbox1=[100,100,200,200], bbox2=[150,150,250,250];

[0367] bbox1∩bbox2=[150,150,200,200];

[0368] bbox1∪bbox2=[100,100,250,250];

[0369] Area(bbox1∩bbox2)=(200-150)×(200-150)=50×50=2500;

[0370] Area(bbox1∪bbox2)=(250-100)×(250-100)=150×150=22500;

[0371] IoU = 2500 / 22500 = 0.111;

[0372] s i =0.9, σ=0.5;

[0373] s i =s i ×exp(-(IoU 2 / σ))=0.9×exp(-0.0246)=0.878.

[0374] S600: Confidence assessment module 400 assesses the confidence of the recognition results.

[0375] Preferably, such as Figure 1 As shown, the processor 200 of the present invention further includes a confidence assessment module 400. The disease identification module 230 is connected to the confidence assessment module 400. Specifically, the confidence assessment module 400 is connected to the classification module 235 in the disease identification module 230.

[0376] Directly using decoding probabilities often leads to overconfidence or underconfidence. Integrating the confidence levels of three sources—sequence probability, cross-modal matching, and geometric consistency—can more reliably reflect the credibility of conclusions and facilitate the formulation of threshold grading and manual review strategies.

[0377] First, the confidence assessment module 400 calculates the sequence confidence, takes the average labeled log probability of the structured output, and obtains a standardized confidence score after length normalization, which ranges from 0 to 1.

[0378] Then, the confidence assessment module 400 calculates the cross-modal matching score, paraphrases the predicted JSON results into summary text, performs visual text matching scoring with the original image, uses the same type or lightweight matching head for similarity calculation, and normalizes to the range of 0 to 1.

[0379] Next, the confidence assessment module 400 calculates the geometric consistency score, estimating geometric features (such as crack width, seepage area, and spalling area) from the bounding box or polygon based on the millimeter-pixel ratio, and verifying the consistency with the disease type features given by the model. The deviation is mapped to a score of 0 to 1 by the Sigmoid function.

[0380] Finally, the confidence assessment module 400 weights and fuses the three confidence scores according to their respective weights: 50% for sequence confidence, 30% for cross-modal matching, and 20% for geometric consistency. Temperature calibration and equal-division box calibration are performed on the validation set to optimize calibration error. Four confidence threshold levels are set, and results with extremely low confidence scores automatically enter the manual review process.

[0381] The confidence assessment module 400 performs the following process for calculating the confidence of a sequence.

[0382] Mean log probability:

[0383] seq conf = (1*L)×∑logP(y i |filtered output ).

[0384] In the above formula, seq conf P(y) represents the sequence confidence level; L represents the sequence length; P(y) represents the sequence confidence level. i |filtered output ) represents conditional probability.

[0385] The input data for the confidence assessment module 400 has the following shape.

[0386] filtered output :(M,4)←From S18

[0387] geom_feat:(5,)←From S16

[0388] Predicted sequence: (L,) ← L predicted labels

[0389] ↓ Logarithmic probability calculation

[0390] log probs :(L,)

[0391] ↓Average

[0392] seq conf :(1,)←Scalar

[0393] The following is an example of the calculation of confidence assessment module 400.

[0394] L = 10, probability sequence: [0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1, 0.05].

[0395] log probability sequence:

[0396] [-0.105,-0.223,-0.357,-0.511,-0.693,-0.916,-1.204,-1.609,-2.303,-2.996]

[0397] seq conf = (1 / 10) × (-10.917) = -1.092.

[0398] The confidence assessment module 400 performs the following three-dimensional cross-modal matching score calculation process.

[0399] Similarity calculation:

[0400] sim = cosine(V,T);

[0401] cross score = sigmoid(sim).

[0402] In the above formula, sim represents cosine similarity; cross score This represents the cross-modal matching score.

[0403] The input data shape for 3D cross-modal matching is as follows.

[0404] V:(256,768)←From S15

[0405] T:(512,768)←From S15

[0406] ↓ Cosine similarity

[0407] sim:(1,)←Scalar

[0408] ↓sigmoid

[0409] cross score :(1,)←Scalar

[0410] The following is a calculation example of the confidence assessment module 400 performing three-dimensional cross-modal matching score calculation.

[0411] V=[0.8,0.6,0.9], T=[0.7,0.5,0.8];

[0412] The dot product = 0.8 × 0.7 + 0.6 × 0.5 + 0.9 × 0.8 = 0.56 + 0.3 + 0.72 = 1.58;

[0413]

[0414] sim=1.58 / (1.345×1.175)=1.58 / 1.581=0.999;

[0415] cross score =1 / (1+exp(-0.999))=1 / (1+0.368)=1 / 1.368=0.731.

[0416] The process of calculating the geometric consistency score in the confidence assessment module 400 is as follows.

[0417] Consistency verification:

[0418] pred area = (x2-x1)×(y2-y1)×mm ratio 2 ;

[0419] geom cons =1-|pred area -actual area | / actual area ;

[0420] cons score =sigmoid(geom) cons ).

[0421] In the above formula, pred area Indicates the predicted area; x i and y i These represent the bounding box coordinates; mm ratio Indicates the ratio of millimeters to pixels; actual area Indicates the actual area; geom cons Indicates geometric consistency; const score This represents the consistency score.

[0422] The input data shape for geometric consistency score calculation is as follows.

[0423] Bounding box: (4,)

[0424] mm ratio :(1,)

[0425] ↓Area Calculation

[0426] pred area :(1,)←Scalar

[0427] actual area :(1,)←Scalar

[0428] Consistency calculation

[0429] geom cons :(1,)←Scalar

[0430] ↓sigmoid

[0431] cons score :(1,)←Scalar

[0432] The following is an example of how the confidence assessment module 400 calculates the geometric consistency score.

[0433] x1=100,y1=100,x2=200,y2=200,mm ratio =0.5;

[0434] pred area = (x2-x1)×(y2-y1)×mm ratio 2 =100×100×0.25=2500mm 2 ;

[0435] actual area =2400mm 2 ;

[0436] geom cons =1 - |2500 - 2400| / 2400 = 1 - 100 / 2400=1-0.042=0.958;

[0437] cons score =1 / (1+exp(-0.958))=1 / (1+0.384)=0.723.

[0438] The confidence assessment module 400 performs confidence fusion.

[0439] The formula for weighted fusion is:

[0440] final conf =0.5×seq conf +0.3×cross score +0.2×cons score .

[0441] In the above formula, final conf This indicates the final confidence level.

[0442] The input data for the weighted fusion calculation has the following shape.

[0443] seq conf :(1,)

[0444] cross score :(1,)

[0445] cons score :(1,)

[0446] ↓ Weighted fusion

[0447] final conf :(1,)←Scalar

[0448] ↓Combined Output

[0449] confidence_output:{

[0450] "result":(M,4),

[0451] "confidence":(1,)

[0452] }

[0453] The following is an example of the calculation for weighted fusion.

[0454] final conf =0.5×0.8+0.3×0.731+0.2×0.723=0.4+0.219+0.145=0.764.

[0455] S700: Verification module 500 performs multi-scale recognition result verification.

[0456] Preferably, such as Figure 1 As shown, the processor 200 of the present invention further includes a verification module 500. The verification module 500 is connected to the confidence assessment module 400.

[0457] Tunnel surface defects are commonly distributed across tiles and across scales, and single-scale judgment is prone to boundary breaks and scale biases. This invention introduces inter-layer and neighborhood consistency verification in the pyramid, which can systematically filter isolated misjudgments and improve cross-block coherence.

[0458] The verification process of verification module 500 is as follows.

[0459] First, inter-layer consistency verification is performed on the constructed 5-level tile pyramid. Scale normalization and overlap center offset constraints are applied to the multi-level results in the same physical region. Appropriate overlap thresholds and center offset tolerances are set to determine whether the results are consistent.

[0460] Then, spatial neighborhood consistency verification is implemented. Markov smoothing of the 8-neighborhood tiles in the same layer is performed to handle the disease type. Abnormally isolated detection boxes are attenuated or removed according to the confidence level and neighborhood voting results to ensure spatial continuity.

[0461] Finally, time consistency verification is implemented by using exponential smoothing or Kalman filtering algorithms on historical inspection sequences. This requires that results at the same location maintain a consistent trend over several consecutive detection cycles, and mutations are marked as suspected anomalies, thus supporting stability verification in the time dimension.

[0462] The process of inter-layer consistency verification performed by the verification module 500 is as follows.

[0463] Overlap calculation:

[0464]

[0465] layer consistency =(overlap>0.5)&&(center) off <threshold)。

[0466] In the above formula, center off Indicates the center offset distance; cx i and cy i These represent the center coordinates of box i.

[0467] Input data shape:

[0468] Current layer result: (M,4)

[0469] Results for other layers: (N,4)

[0470] ↓Overlap Calculation

[0471] overlap:(M,N) ← matrix

[0472] ↓ Center offset calculation

[0473] center off :(M,N)←Matrix

[0474] ↓Consistency judgment

[0475] layer consistency :(M,)←Boolean vector

[0476] The process of spatial neighborhood consistency verification is as follows.

[0477] Markov smoothing:

[0478] P=(1 / Z)×exp(β×∑I(y i =y j ));

[0479] spatial consistency =P>0.7.

[0480] Where P represents the Markov probability; Z represents the normalization constant; β represents the smoothing parameter; I(y i =y j ) indicates an indicator function; y i and y j These represent neighborhood labels.

[0481] Input data shape:

[0482] Neighborhood results: (8,M,4) ← Results for 8 neighborhoods

[0483] Markov smoothing

[0484] P:(M,) ← probability vector

[0485] ↓Consistency judgment

[0486] spatial consistency :(M,)←Boolean vector

[0487] The process of time consistency verification is as follows.

[0488] Exponential smoothing:

[0489] smoothed=α×current+(1-α)×prev smoothed ;

[0490] time consistency =|smoothed-current| <threshold。

[0491] In the above formula, smoothed represents the smoothed value; α represents the smoothing factor (0.3); current represents the current value; prev smoothed This represents the previous smoothing value.

[0492] Input data shape:

[0493] Historical sequence: (T, M, 4)

[0494] Current result: (M,4)

[0495] ↓Exponential smoothing

[0496] smoothed:(M,4)

[0497] ↓Consistency judgment

[0498] time consistency :(M,)←Boolean vector

[0499] S800: Optimization module 600 processes and optimizes the recognition results.

[0500] Preferably, such asFigure 1 As shown, the processor 200 of the present invention also includes an optimization module 600. The optimization module 600 may also be connected to the verification module 500.

[0501] The original output may contain issues such as duplicate frames, rough boundaries, and inconsistent coordinate systems. By deduplicating, refining boundaries, and standardizing coordinate reprojection, the engineering usability and compatibility with 3D assets can be significantly improved.

[0502] The optimization process of optimization module 600 is as follows.

[0503] First, deduplication and merging are performed. For overlapping results on the same tile or across tiles, soft nonmaximum suppression and morphological merging algorithms are used. Cracks are merged according to skeleton connectivity, peeling is merged according to superpixel regions, and water seepage is merged according to connected regions.

[0504] Then, boundary refinement is performed. Cracks are refined using subpixel skeletonization combined with adaptive threshold backfilling technology, peeling is refined using superpixel segmentation combined with active contour algorithm, and water seepage is refined using region growth combined with boundary smoothing to improve the accuracy of boundary positioning.

[0505] Next, geometric quantization is performed, using millimeter-pixel ratio to convert crack width and length, seepage area and boundary complexity, spalling area and unevenness, and the surface unevenness features are approximated by normal variation, providing quantitative data for subsequent analysis.

[0506] The deduplication and merging process of optimization module 600 is as follows.

[0507] Soft NMS threshold:

[0508] merged output =merged duplicates (validated output ,s i ).

[0509] Input data shape:

[0510] validated output :{

[0511] "result":(M,4),

[0512] layer consistency ":(M,),

[0513] spatial consistency ":(M,),

[0514] time consistency ":(M,

[0515] }

[0516] Bounding box: (M,4)

[0517] Confidence level: (M,)

[0518] ↓NMS deduplication

[0519] merged boxes :(K,4)←K≤M

[0520] ↓ Morphological merging

[0521] merged output :(L,4)←L≤K

[0522] The optimization module 600 refines the boundaries.

[0523] Subpixel skeletonization:

[0524] sub x =x + 0.5 × (I (x+1,y) -I (x-1,y) ) / (2×I (x,y) -I (x+1,y) -I (x-1,y) );

[0525] sub y =y+0.5×(I (x,y+1) -I (x,y-1) ) / (2×I (x,y) -I (x,y+1) -I (x,y-1) );

[0526] refined output =refined boundaries (merged output ,sub x ,sub y ).

[0527] In the above formula, sub x and sub y They represent sub-pixel coordinates respectively; x and y represent pixel coordinates respectively; I( x,y) This represents the intensity value of pixel (x, y).

[0528] Input data shape:

[0529] Original boundary: (L,4)

[0530] ↓ Subpixel calculation

[0531] subpixel coords :(L,4)

[0532] ↓Boundary refinement

[0533] refined output :(L,4)

[0534] The process of geometric quantization performed by the optimization module 600 is as follows.

[0535] Actual size calculation:

[0536] real len =pixel len ×mm ratio ;

[0537] real area =pixel area ×mm ratio 2 ;

[0538] In the above formula, real len Indicates actual length; pixel len Indicates pixel length; real area Indicates the actual area.

[0539] Input data shape:

[0540] Pixel coordinates: (L, 4)

[0541] mm ratio :(1,)

[0542] ↓ Actual size calculation

[0543] real len :(L,)←Actual length

[0544] real area :(L,)←Actual area

[0545] ↓Combined Output

[0546] quantified_output:{

[0547] "result":(L,4),

[0548] real len ":(L,),

[0549] real area ":(L,

[0550] }

[0551] S900: The report generation module 700 integrates the test result data to generate an intelligent report.

[0552] Preferably, such as Figure 1 As shown, the processor 200 of the present invention also includes a report generation module 700. The report generation module 700 is connected to the optimization module 600.

[0553] Scattered identification results cannot be directly used for report generation; data integration is a necessary step in generating a complete report. Structured data facilitates subsequent processing and analysis, data validation ensures the accuracy of the report content, and handling of outliers prevents the spread of erroneous information.

[0554] The process of generating intelligent reports by the report generation module 700 is as follows.

[0555] First, based on the standardized JSON results output from the aforementioned steps, the data is grouped and aggregated according to tunnel segments and disease types to generate structured data containing information such as segment identifiers and disease summaries.

[0556] Then, data integrity verification is performed, checking the completeness of required fields, the reasonableness of numerical ranges, and the validity of spatial coordinates. Missing data is interpolated or marked, and abnormal data enters the manual review process.

[0557] Next, spatial topological relationships are constructed. Based on tile IDs and three-dimensional coordinates, spatial adjacency relationships between diseases are established to identify disease clustering areas and diffusion trends.

[0558] Finally, time series alignment is achieved by aligning historical detection results by timestamp to construct a time series of disease development.

[0559] S910: Develop report templates.

[0560] This step begins by building a standard template library, which includes standard sections such as inspection overview, disease analysis, safety risk assessment, and maintenance recommendations.

[0561] S920: Generate large-scale natural language models.

[0562] Traditional template-filling methods lack flexibility and professionalism in generating reports, while large models can produce more natural and professional report content. Based on the natural language generation capabilities of the QwenVL-Max model, professional detection reports can be automatically generated according to the detection results (see...). Figure 13 The displayed 3D structural inspection report includes a description of the defects, risk assessment, and maintenance recommendations. The prompt word engineering controls the direction and quality of the generated content, and the structured content ensures the report's logic and readability.

[0563] The process of generating natural language for large models is as follows.

[0564] First, based on the aforementioned fine-tuned QwenVL-Max model, professional prompts are constructed, including prompt templates for industry terminology, report structure, and professional requirements, to ensure the professionalism and accuracy of the generated content.

[0565] Then, multimodal information such as detection images, geometric data, and historical trends are integrated into the prompts to support the generation of professional reports with rich graphics and text. Next, content quality control parameters are set, including content length, frequency of use of professional terms, and completeness of logical structure, to ensure the professional level of the generated reports.

[0566] Finally, a Chinese terminology database was established, containing Chinese terms for disease types such as cracks, seepage, and peeling, to support accurate Chinese professional expressions.

[0567] S930: Output multi-format reports.

[0568] Different users have different requirements for report formats, and multi-format support can meet diverse use cases. Automatic format conversion improves work efficiency, and embedded charts enhance the readability and professionalism of reports.

[0569] The process of outputting multi-format reports is as follows.

[0570] First, it supports multiple output formats, including PDF, Word, and other formats, each with corresponding style templates and layout configurations.

[0571] Then, automatic format conversion is implemented, automatically adjusting the content layout, font style, and chart size based on the target format to ensure consistency and professionalism across different formats.

[0572] Next, embed charts and images, automatically embedding disease diagrams and other charts to improve the readability and professionalism of the report.

[0573] like Figure 13 The tests included concrete spalling in the tunnel, water seepage in the tunnel, and cracks in the tunnel walls.

[0574] The test results are as follows: The statistical data for the identification of tunnel defects in the section are as follows: the number of tunnels with water seepage is 473, the number of concrete spallings is 82, and the number of tunnel wall cracks is 473.

[0575] Figure 13 The report also includes a bar chart of the inspection results. The horizontal axis of the bar chart represents the tunnel type, including shield tunnels, cut-and-cover tunnels, mining tunnels, and others. The vertical axis of the bar chart represents the quantity. Blue bars represent the quantity of water seepage (tunnel water seepage), green bars represent the quantity of spalling (tunnel concrete spalling), and light blue bars represent the quantity of cracks (tunnel wall cracks).

[0576] Example 2

[0577] This embodiment is a further explanation of embodiment 1, and repeated content will not be repeated.

[0578] This embodiment describes the system of the present invention from the perspective of the data layer.

[0579] like Figure 8 As shown, the system of the present invention includes, from the bottom layer, an application layer, an intelligent recognition layer, a data processing layer, and a data acquisition layer.

[0580] The data acquisition layer includes a 3D laser scanner, a mobile platform railcar / trolley, a GNSS positioning system, and an IMU inertial navigation system.

[0581] The data processing layer includes point cloud preprocessing, noise reduction and filtering steps, data registration steps, S3M tiled spatial segmentation steps, and 3D image generation steps.

[0582] The intelligent identification layer includes the QwenVL-Max multimodal fusion step, feature extraction step, disease identification step, and risk assessment step.

[0583] The application layer includes intelligent report generation, data management, big data analysis, and anomaly detection steps. Preferably, the intelligent early warning system is triggered after anomaly detection, thus triggering the early warning. Early warning information is disseminated through multiple channels, such as email alerts, SMS alerts, and dashboard notifications.

[0584] The system architecture comprises multiple layers, with arrows indicating the interaction and support relationships between these layers. For example... Figure 9As shown, the mobile application and data analytics platform at the application layer interact with the API gateway layer via arrows. The API gateway layer, leveraging its functions of load balancing and request routing, authentication and authorization, JWT tokens, access control, request rate limiting, circuit breaker protection, and monitoring and alerting, establishes connections with the caching layer and database layer via arrows. The caching layer includes a Redis cluster (using master-slave replication and sentinel mode), hot data caching, session storage, temporary data, and cache policy TTL management and consistency guarantees, interacting with the API gateway layer and database layer via arrows. The database layer consists of a PostgreSQL master (for relational data transaction processing and data synchronization), PostgreSQL slaves (for read / write separation and data backup), a MongoDB cluster (for unstructured document data storage), and HDFS distributed storage (for point cloud data and large file storage), which exchange and manipulate data with the caching layer, monitoring layer, and data synchronization layer via arrows. Data synchronization is achieved between the PostgreSQL master and the MongoDB cluster. Large files are sent from the PostgreSQL master to the HDFS distributed storage. Master-slave replication occurs between the PostgreSQL master and PostgreSQL slaves. The monitoring layer performs database monitoring (including performance metrics and health checks) and log collection (using the ELK Stack for log analysis). It also includes an alerting system (containing anomaly detection and notification mechanisms). The monitoring layer uses arrows to monitor and provide feedback to the database layer and other components. The data synchronization layer provides data synchronization services (including CDC change capture and synchronization), ETL processing (for data transformation and cleansing), and message queues (using Kafka / RabbitMQ for asynchronous processing). The data synchronization layer uses arrows to perform data synchronization-related operations with the database layer and other components.

Claims

1. A tunnel 3D defect intelligent identification system based on a large model, comprising a point cloud data acquisition module (100) and a processor (200), characterized in that, The processor (200) includes: The data processing module (210) performs standardization, noise reduction and registration processing on the received point cloud data; The 3D tile optimization module (220) constructs a multi-level tile pyramid structure based on the registered point cloud data, establishes a mapping relationship between spatial coordinates and tile indexes, compresses tile data and adjusts texture quality based on the curvature-based point cloud simplification algorithm to form a 3D tile image, and calculates a comprehensive score of tile quality to verify the quality of the 3D tile image. The disease identification module (230) extracts the multimodal fusion features and geometric features corresponding to the disease type from the preprocessed three-dimensional tile image; fuses the multimodal fusion features with the geometric features to obtain joint fusion features; performs domain fine-tuning on the joint fusion features based on the domain fine-tuning model; and obtains the disease identification result.

2. The system according to claim 1, characterized in that, The system also includes: Confidence assessment module (400): assesses the confidence level of the identification results; Verification module (500): Verifies multi-scale recognition results; Optimization module (600): Processes and optimizes the recognition results; Report generation module (700): Integrates the test result data to generate an intelligent report.

3. The system according to claim 1 or 2, characterized in that, The data processing steps of the data processing module (210) include: Convert the received point cloud data into point cloud data in a standard format; The statistical filtering algorithm is used to denoise standard format point cloud data to remove outliers that do not conform to the statistical distribution. Point cloud registration is performed based on the ICP algorithm to generate a coordinate transformation matrix.

4. The system according to any one of claims 1 to 3, characterized in that, The steps of the three-dimensional tile optimization module (220) in constructing a multi-level tile pyramid structure based on the registered point cloud data include: The spatial range is calculated based on the point cloud bounding box. The target tile size is set to a preset size. The number of tiles is calculated and a tile mesh is generated. The mesh is adaptively adjusted based on the point cloud density to achieve adaptive mesh division of the tiles. In a multi-level tile pyramid, level 0 is the top level, and the resolution of each level of tiles increases progressively from level 0; or, the resolution of each level of tiles decreases progressively from the highest level.

5. The system according to any one of claims 1 to 4, characterized in that, The steps of establishing the mapping relationship between spatial coordinates and tile indices in the three-dimensional tile optimization module (220) include: Calculate the tile index position of a point in three-dimensional space within the multi-level tile pyramid. A unique tile ID is generated based on the tile index position; The tile ID adopts the encoding format of "level × 1000000 + row number × 1000 + column number" to realize the mapping from spatial coordinates to tile index, thereby establishing a unique correspondence between level, row, column and spatial location.

6. The system according to any one of claims 1 to 5, characterized in that, The steps of the disease identification module (230) in extracting multimodal fusion features of the three-dimensional tile image include: Receive preprocessed 3D tile images and disease description text. Extract visual feature vectors and text feature vectors. Output multimodal fusion features.

7. The system according to any one of claims 1 to 6, characterized in that, The steps of the disease identification module (230) in extracting the geometric features corresponding to the disease type from the three-dimensional tile image include: The crack direction is determined based on principal component analysis; the crack length is calculated based on the minimum spanning tree algorithm; the curvature is calculated based on local geometric changes; the seepage area is calculated based on connected component analysis; the boundary complexity is calculated based on the edge detection algorithm; and the concrete spalling area is calculated based on the morphological operation algorithm.

8. The system according to any one of claims 1 to 7, characterized in that, The step of fusing multimodal fusion features and geometric features by the disease identification module (230) includes: All geometric features are mapped to the same dimension as the multimodal fusion features through a fully connected layer, and feature fusion is performed based on a weighted average formula; The weighted average formula is: Fusion Feature = W A ×Multimodal fusion features + W B ×Geometric features; Among them, W A W represents the weights of the multimodal fusion features. B The weights represent the geometric features.

9. A method for intelligent identification of three-dimensional tunnel defects based on a large model, characterized in that, The method includes: The received point cloud data is standardized, denoised, and registered. A multi-level tile pyramid structure is constructed based on the registered point cloud data. A mapping relationship between spatial coordinates and tile indexes is established. A point cloud simplification algorithm based on curvature is used to compress the tile data and adjust the texture quality to form a three-dimensional tile image. A comprehensive score of tile quality is calculated to verify the quality of the three-dimensional tile image. Extract multimodal fusion features and geometric features corresponding to disease types from the preprocessed 3D tile image; fuse the multimodal fusion features with the geometric features to obtain joint fusion features; perform domain fine-tuning on the joint fusion features based on the domain fine-tuning model; and obtain the disease identification results.

10. The method according to claim 9, characterized in that, The method further includes: Confidence assessment of the identification results; Verify the multi-scale recognition results; The recognition results are processed and optimized; The test results data are integrated to generate an intelligent report.

Images