Method and system for detecting and positioning bladder expansion based on machine vision

Abstract

The present invention provides a method and system for detecting and locating bladder expansion based on machine vision, which relates to the field of machine vision technology, including using multispectral imaging technology to collect tunnel bladder images; using a deep learning model to extract and classify features of preprocessed images to identify expansion areas; using an improved structured light three-dimensional reconstruction technology to establish a three-dimensional model of the tunnel and determine the spatial position and range of the expansion area; performing risk assessment and grading on the expansion area, generating detection results including the position, range and risk level of the expansion area, visually presenting the detection results on the three-dimensional model of the tunnel, and generating a maintenance and reinforcement plan based on a reinforcement learning algorithm.

Description

Technical Field

[0001] The present invention relates to machine vision technology, and in particular to a method and system for detecting and positioning bladder expansion based on machine vision. Background Art

[0002] Due to the influence of geological conditions, construction quality, operating environment and other factors, the tunnel structure is prone to bladder expansion problems. Existing tunnel detection methods have the following disadvantages:

[0003] Inefficiency: Manual inspection is time-consuming and labor-intensive, and it is difficult to conduct a comprehensive inspection of long-distance tunnels.

[0004] Lack of accuracy: Manual observation is easily affected by subjective factors and it is difficult to accurately identify and quantify tiny deformations.

[0005] Safety risks: Manual inspection in some dangerous areas may pose safety risks.

[0006] Difficult to achieve real-time monitoring: It is impossible to continuously monitor the tunnel status in real time.

[0007] It is difficult to strike a balance between detection accuracy and efficiency.

[0008] Lack of effective processing capabilities for noise and interference in complex environments.

[0009] It is difficult to achieve accurate positioning and quantitative analysis of the expansion area.

[0010] Lack of intelligent interpretation and risk assessment capabilities for test results.

[0011] The development trend of the expansion area cannot be effectively predicted. Summary of the invention

[0012] The embodiments of the present invention provide a method and system for detecting and positioning bladder expansion based on machine vision, which can solve the problems in the prior art.

[0013] According to a first aspect of the embodiments of the present invention,

[0014] Provides a machine vision-based method for detecting and locating the balloon, including:

[0015] The tunnel capsule image is collected by using multispectral imaging technology; the tunnel capsule image is preprocessed; the preprocessed image is feature extracted and classified by using a deep learning model to identify the expansion area, wherein the deep learning model includes an improved multi-scale feature pyramid network and an attention mechanism module; the expansion area is edge detected and contour extracted to obtain the precise boundary of the expansion area; based on the extracted contour information, the area, perimeter and shape characteristics of the expansion area are calculated;

[0016] A three-dimensional model of the tunnel is established by using an improved structured light three-dimensional reconstruction technology, wherein the improved structured light three-dimensional reconstruction technology includes using a phase shift method to improve the accuracy of depth information and applying an adaptive multi-frequency phase unwrapping algorithm; mapping the detected expansion area to the three-dimensional model of the tunnel to determine the spatial position and range of the expansion area; performing risk assessment and grading of the expansion area based on the spatial position and range of the expansion area in combination with a preset hazard level standard, generating a detection result including the position, range and risk level of the expansion area, visually presenting the detection result on the three-dimensional model of the tunnel, and generating a maintenance and reinforcement plan based on a reinforcement learning algorithm;

[0017] Establish a historical database to record the detection results and the changing trends of the expansion area; based on the historical data and current detection results, use the time series analysis method to predict the development trend and potential risks of the expansion area.

[0018] In an optional embodiment,

[0019] The tunnel capsule image is collected by using multispectral imaging technology; the steps of preprocessing the tunnel capsule image include:

[0020] Design a multispectral imaging system, which includes a visible light camera, a near-infrared camera, a thermal infrared camera, a multispectral light source and a confocal optical system; perform geometric calibration, spectral calibration and multi-camera synchronization on the multispectral imaging system; collect multispectral images of the tunnel capsule in a temperature-controlled environment, dynamically adjust the exposure time through an adaptive algorithm, determine the focal length using an autofocus technology, and store the collected images in real time;

[0021] The collected multispectral images are preprocessed, including noise removal, image enhancement, geometric correction, spectral correction, image fusion and dimensionality reduction. Bilateral filtering and contrast-limited adaptive histogram equalization are used for visible light images, non-local mean filtering and Retinex algorithm are used for near-infrared images, and adaptive median filtering and adaptive local contrast enhancement algorithm are applied to thermal infrared images. Distortion correction is performed based on camera intrinsic parameters, and multi-band images are registered using feature point matching and perspective transformation. Radiation correction is performed using sensor response curves, and atmospheric correction is performed using atmospheric radiation transfer models. Wavelet transform is used for feature-level fusion, and decision-level fusion based on belief function theory is used. Dimensionality reduction is performed through principal component analysis and independent component analysis.

[0022] Data quality control is performed on the preprocessing results, including image quality assessment, anomaly detection and data verification. Texture features, shape features and spectral features are extracted from the preprocessed images and statistically analyzed. Among them, texture features are obtained by calculating the gray-level co-occurrence matrix, shape features are extracted through morphological operations, and spectral features include the normalized difference vegetation index.

[0023] In an optional embodiment,

[0024] The steps of extracting and classifying features of the preprocessed image and identifying the dilation area using a deep learning model, wherein the deep learning model includes an improved multi-scale feature pyramid network and an attention mechanism module include:

[0025] The improved multi-scale feature pyramid network is used for feature extraction, including a dense connection module, a spatial pyramid pooling module and a feature recalibration module; the dense connection module enhances feature propagation through multiple convolutions and feature connections; the spatial pyramid pooling module performs pooling operations at different scales and captures multi-scale context information through upsampling; the feature recalibration module includes a channel attention mechanism to adaptively weight feature channels;

[0026] The attention mechanism module is integrated with the improved multi-scale feature pyramid network, including a channel attention submodule and a spatial attention submodule; the channel attention submodule calculates channel weights through a shared multi-layer perceptron using a combination of global average pooling and global maximum pooling; the spatial attention submodule generates a spatial attention map using feature maps of average pooling and maximum pooling;

[0027] The preprocessed image is input into the deep learning model, and processed by the improved multi-scale feature pyramid network and the attention mechanism module in sequence to obtain a multi-scale feature representation; the multi-scale feature representation is adaptively weighted fused, and the adaptive fusion of features is achieved by learning the weight of each scale feature to obtain a fused feature; the fused feature is input into a classifier for dilation area recognition, and the classifier includes a global average pooling layer and a fully connected layer.

[0028] In an optional embodiment,

[0029] The multi-scale feature representation is adaptively weighted fused, and the weight of each scale feature is learned to achieve adaptive fusion of features to obtain fused features; the fused features are input into a classifier for dilation area recognition, and the classifier includes a global average pooling layer and a fully connected layer. The steps include:

[0030] The preprocessed image is input into the deep learning model, and the feature representation of the image is extracted at multiple scales through multiple downsampling and upsampling operations to obtain feature maps of K different scales;

[0031] The K feature maps extracted are sent to the 1×1 convolution layer for dimensionality reduction to obtain the corresponding K weight maps, and then the K weight maps are normalized using the normalized exponential function to ensure that the sum of the weights at each spatial position is 1;

[0032] Multiply K feature maps by their corresponding weight maps element by element, and add the weighted feature maps to obtain an adaptively fused multi-scale feature representation.

[0033] The adaptively fused features are compressed into a feature vector of fixed length through a global average pooling layer;

[0034] The pooled feature vector is passed through the fully connected layer and the dropout layer with L2 regularization in turn, and finally through the fully connected layer activated by the normalized exponential function to output the probability of each category and complete the classification of the expansion area.

[0035] In an optional embodiment,

[0036] The improved structured light 3D reconstruction technology is used to establish a tunnel 3D model. The improved structured light 3D reconstruction technology includes using a phase shift method to improve the accuracy of depth information, and applying an adaptive multi-frequency phase unwrapping algorithm, including the following steps:

[0037] Projecting a plurality of sinusoidal grating patterns with a fixed phase difference onto the tunnel surface, wherein the light intensity distribution of the sinusoidal grating pattern is composed of background light intensity, modulated light intensity and object surface phase distribution;

[0038] Collect multiple images reflected from the tunnel surface, and calculate the phase value of the tunnel surface by the least square method to obtain the wrapped phase;

[0039] Adopting an adaptive multi-frequency phase unwrapping algorithm to unwrap the wrapped phase, including projecting a low-frequency grating pattern and calculating a low-frequency phase, projecting an intermediate-frequency grating pattern and calculating an intermediate-frequency phase, and projecting a high-frequency grating pattern and calculating a high-frequency phase;

[0040] According to the low-frequency phase, medium-frequency phase and high-frequency phase, an adaptive threshold mechanism is used to perform phase unwrapping to obtain the absolute phase value; the adaptive threshold mechanism dynamically adjusts the unwrapping threshold by analyzing the phase difference between adjacent frequencies to handle the situation where the tunnel surface is discontinuous or blocked;

[0041] Based on the absolute phase value, a weighted average strategy is adopted to calculate the depth information of the tunnel surface;

[0042] Point cloud registration technology is used to integrate depth information obtained from multiple perspectives into a complete three-dimensional tunnel model.

[0043] In an optional embodiment,

[0044] According to the spatial position and range of the expansion area, combined with the preset hazard level standard, the expansion area is risk assessed and graded, and the detection results including the location, range and risk level of the expansion area are generated. The detection results are visualized on the three-dimensional model of the tunnel, and the maintenance and reinforcement plan based on the reinforcement learning algorithm is generated. The steps include:

[0045] Obtaining spatial location and range information of tunnel expansion area;

[0046] Calculating the risk index of the expansion area using a weighted summation method based on a multi-factor risk assessment model, wherein the multi-factor risk assessment model includes expansion area, expansion depth, location sensitivity, tunnel age and geological conditions;

[0047] According to the calculated risk index, the expansion area is divided into different risk levels;

[0048] Using 3D visualization technology, the inspection results, including the location, range and risk level of the expansion area, are presented on the 3D model of the tunnel;

[0049] Construct a reinforcement learning environment, including defining the state space, action space, and reward function. The state space includes the risk index, location, range, and current maintenance budget of the expansion area. The action space includes a variety of maintenance measures. The reward function is designed based on the risk reduction degree and cost-effectiveness ratio after maintenance. Use a deep Q-learning algorithm to train a maintenance decision model. Based on the trained maintenance decision model, generate maintenance and reinforcement plans for each expansion area.

[0050] The generated maintenance and reinforcement plan is displayed in a 3D visualization interface. The tunnel status information is updated in real time, and the multi-factor risk assessment model and maintenance decision model are continuously optimized based on new inspection data and maintenance effect feedback. The optimized multi-factor risk assessment model and maintenance decision model are applied to subsequent risk assessment and maintenance plan formulation to form a closed-loop optimization mechanism.

[0051] In an optional embodiment,

[0052] Constructing a reinforcement learning environment includes defining a state space, an action space, and a reward function, wherein the state space includes the risk index, location, range, and current maintenance budget of the expansion area, the action space includes a variety of maintenance measures, and the reward function is designed based on the risk reduction degree and cost-effectiveness ratio after maintenance; using a deep Q learning algorithm to train a maintenance decision model, and based on the trained maintenance decision model, the steps of generating a maintenance and reinforcement plan for each expansion area include:

[0053] The state space includes expansion area characteristics, tunnel global characteristics and maintenance resource information, wherein the expansion area characteristics include risk index, location coordinates and area, the tunnel global characteristics include tunnel age, geological condition score and traffic load, and the maintenance resource information includes available budget, human resources and equipment resources;

[0054] The action space is defined as a set of discrete maintenance measures, including grouting reinforcement, bolting, lining repair, and no-action options;

[0055] Design a multi-objective reward function, including risk index reduction, safety improvement, maintenance cost, and long-term performance impact;

[0056] Construct a deep Q network based on long short-term memory network and attention mechanism to approximate the Q function;

[0057] The deep Q network includes a long short-term memory layer, a multi-head attention layer, and a fully connected layer for processing high-dimensional state space and capturing temporal dependencies;

[0058] Implement a prioritized experience replay buffer, assign a priority to each experience sample based on the timing difference error, and use importance sampling for bias correction;

[0059] Adopting the target network and soft update strategy, the target Q value is calculated by mixing the output of the online network and the target network, and the target network parameters are gradually updated using the soft update coefficients;

[0060] Design an ε-greedy strategy with a decay mechanism, perform extensive exploration in the early stages of training, and execute a training loop, including interaction with the environment, experience storage, batch sampling, loss calculation, parameter update, and target network soft update steps;

[0061] Using the trained deep Q network, the optimal maintenance plan is generated through the Monte Carlo tree search method according to the current tunnel status. The generated maintenance plan is applied to the tunnel management system, and feedback on the implementation effect is collected. The environmental model and reward function are updated according to the feedback data to achieve continuous optimization and adaptive adjustment of the algorithm.

[0062] According to a second aspect of the embodiments of the present invention,

[0063] Provides machine vision-based balloon inflation detection and positioning systems, including:

[0064] The first unit is used to collect tunnel capsule images using multispectral imaging technology; preprocess the tunnel capsule images; use a deep learning model to extract features and classify the preprocessed images to identify the expansion area, wherein the deep learning model includes an improved multi-scale feature pyramid network and an attention mechanism module; perform edge detection and contour extraction on the expansion area to obtain the precise boundary of the expansion area; and calculate the area, perimeter and shape characteristics of the expansion area based on the extracted contour information;

[0065] The second unit is used to establish a three-dimensional model of the tunnel using an improved structured light three-dimensional reconstruction technology, wherein the improved structured light three-dimensional reconstruction technology includes using a phase shift method to improve the accuracy of depth information and applying an adaptive multi-frequency phase unwrapping algorithm; mapping the detected expansion area to the three-dimensional model of the tunnel to determine the spatial position and range of the expansion area; performing risk assessment and grading of the expansion area based on the spatial position and range of the expansion area in combination with a preset hazard level standard, generating a detection result including the position, range and risk level of the expansion area, visually presenting the detection result on the three-dimensional model of the tunnel, and generating a maintenance and reinforcement plan based on a reinforcement learning algorithm;

[0066] The third unit is used to establish a historical database to record the detection results and the changing trends of the expansion area; based on the historical data and current detection results, the time series analysis method is used to predict the development trend and potential risks of the expansion area.

[0067] According to a third aspect of the embodiments of the present invention,

[0068] An electronic device is provided, comprising:

[0069] processor;

[0070] a memory for storing processor-executable instructions;

[0071] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0072] A fourth aspect of the embodiments of the present invention is:

[0073] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0074] This paper combines deep learning and image processing technology to achieve high-precision detection of tunnel capsule expansion, which improves the detection accuracy compared to traditional manual detection methods. In particular, it can still maintain stable detection performance under complex lighting and surface texture conditions. The entire detection and positioning process is highly automated, from image acquisition, preprocessing, feature extraction to the final expansion area identification and positioning, no human intervention is required, which greatly reduces the investment in human resources and improves work efficiency.

[0075] This paper adopts an optimized deep learning model and an efficient image processing algorithm, and the system can achieve near real-time detection and positioning. Under standard hardware configuration, the processing time of each image is controlled within 100 milliseconds, meeting the needs of real-time monitoring. By combining the conversion algorithm of the image coordinate system and the actual tunnel coordinate system, the system can accurately locate the spatial position of the expansion area, and the positioning error is controlled at the centimeter level, providing accurate location information for subsequent maintenance. This paper adopts transfer learning and online fine-tuning technology, which can quickly adapt to different tunnel environments and lighting conditions. Only a small amount of new environmental data is needed to complete model adaptation, which greatly improves the versatility and scalability of the system.

[0076] This paper uses innovative machine vision technology to significantly improve the accuracy, efficiency and automation level of tunnel bladder expansion detection, while having good adaptability and scalability. It can not only directly improve the effect of daily monitoring and maintenance of tunnels, but also provide reliable technical support for long-term tunnel safety management and decision-making, with important engineering application value and economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 It is a flow chart of a method for detecting and positioning a capsule expansion based on machine vision according to an embodiment of the present invention;

[0078] Figure 2 It is a structural schematic diagram of a bladder inflation detection and positioning system based on machine vision according to an embodiment of the present invention. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0080] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0081] Figure 1 FIG. 1 is a flow chart of a method for detecting and positioning a capsule expansion based on machine vision according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0082] S1. Use multispectral imaging technology to collect tunnel capsule images; preprocess the tunnel capsule images; use a deep learning model to extract and classify features of the preprocessed images to identify the expansion area, wherein the deep learning model includes an improved multi-scale feature pyramid network and an attention mechanism module; perform edge detection and contour extraction on the expansion area to obtain the precise boundary of the expansion area; calculate the area, perimeter and shape characteristics of the expansion area based on the extracted contour information;

[0083] S2. Establish a three-dimensional model of the tunnel using an improved structured light three-dimensional reconstruction technology, wherein the improved structured light three-dimensional reconstruction technology includes using a phase shift method to improve the accuracy of depth information and applying an adaptive multi-frequency phase unwrapping algorithm; map the detected expansion area to the three-dimensional model of the tunnel to determine the spatial position and range of the expansion area; perform risk assessment and classification of the expansion area based on the spatial position and range of the expansion area in combination with a preset hazard level standard, generate a detection result including the position, range and risk level of the expansion area, visualize the detection result on the three-dimensional model of the tunnel, and generate a maintenance and reinforcement plan based on a reinforcement learning algorithm;

[0084] S3. Establish a historical database to record the detection results and the changing trends of the expansion area; based on the historical data and current detection results, use the time series analysis method to predict the development trend and potential risks of the expansion area.

[0085] In an optional embodiment,

[0086] The tunnel capsule image is collected by using multispectral imaging technology; the steps of preprocessing the tunnel capsule image include:

[0087] Design a multispectral imaging system, which includes a visible light camera, a near-infrared camera, a thermal infrared camera, a multispectral light source and a confocal optical system; perform geometric calibration, spectral calibration and multi-camera synchronization on the multispectral imaging system; collect multispectral images of the tunnel capsule in a temperature-controlled environment, dynamically adjust the exposure time through an adaptive algorithm, determine the focal length using an autofocus technology, and store the collected images in real time;

[0088] The collected multispectral images are preprocessed, including noise removal, image enhancement, geometric correction, spectral correction, image fusion and dimensionality reduction. Bilateral filtering and contrast-limited adaptive histogram equalization are used for visible light images, non-local mean filtering and Retinex algorithm are used for near-infrared images, and adaptive median filtering and adaptive local contrast enhancement algorithm are applied to thermal infrared images. Distortion correction is performed based on camera intrinsic parameters, and multi-band images are registered using feature point matching and perspective transformation. Radiation correction is performed using sensor response curves, and atmospheric correction is performed using atmospheric radiation transfer models. Wavelet transform is used for feature-level fusion, and decision-level fusion based on belief function theory is used. Dimensionality reduction is performed through principal component analysis and independent component analysis.

[0089] Data quality control is performed on the preprocessing results, including image quality assessment, anomaly detection and data verification. Texture features, shape features and spectral features are extracted from the preprocessed images and statistically analyzed. Among them, texture features are obtained by calculating the gray-level co-occurrence matrix, shape features are extracted through morphological operations, and spectral features include the normalized difference vegetation index.

[0090] For example, the tunnel capsule image acquisition and preprocessing method based on multispectral imaging technology aims to provide a high-quality data basis for subsequent capsule expansion detection and positioning. The method mainly includes key steps such as multispectral imaging system design, image acquisition, image preprocessing and feature extraction.

[0091] Firstly, a multispectral imaging system suitable for tunnel environment was designed. The system consists of a visible light camera, a near-infrared camera, a thermal infrared camera, a multispectral light source and a confocal optical system. The visible light camera uses a high-resolution CMOS sensor with a resolution of 1920×1080 pixels and a frame rate of 60 fps. The near-infrared camera uses an InGaAs sensor, covers the wavelength range of 900-1700 nm, and has a resolution of 640×512 pixels. The thermal infrared camera uses an uncooled microbolometer, works in the 8-14 μm band, and has a resolution of 384×288 pixels. The multispectral light source includes an LED array and a halogen lamp, which can provide a continuous spectrum in the range of 400-2500 nm. The confocal optical system consists of multiple groups of lenses and a beam splitter to ensure that the optical axes of the three cameras are collinear.

[0092] In order to ensure the accuracy of the multispectral imaging system, a rigorous calibration process was carried out. The geometric calibration adopted Zhang's calibration method, using a checkerboard calibration plate with known size, to collect multiple sets of images at different positions and postures, and solve the camera internal parameters and distortion coefficients through the optimization algorithm. The spectral calibration used a standard reflector and a monochromator to establish the correspondence between the image grayscale value and the spectral reflectance. Multi-camera synchronization was achieved through hardware trigger signals and software timestamps to ensure the temporal consistency of images in different bands.

[0093] Image acquisition is performed in a temperature and humidity controlled environment to reduce interference from external factors. An adaptive algorithm is used to dynamically adjust the exposure time to keep the image within the optimal dynamic range based on the statistical characteristics of the image histogram. For example, when the image is too dark, the system will gradually increase the exposure time until the image brightness reaches the expected value. The autofocus technology is based on the principle of phase detection and determines the optimal focal length by analyzing the high-frequency components of the image. The acquired images are transmitted to the storage system in real time through a high-speed data interface and saved in a lossless compression format to ensure the integrity of the original data.

[0094] Image preprocessing is a key step to improve data quality. For visible light images, bilateral filtering is first applied to remove noise. This filter considers both spatial distance and grayscale similarity and can effectively retain edge information. Then, contrast-limited adaptive histogram equalization is used to enhance the image contrast, and over-enhancement is avoided by limiting the maximum value of the histogram. For near-infrared images, non-local mean filtering is used for denoising. This method uses the non-local self-similarity of the image to better protect texture details. The Retinex algorithm is used to enhance near-infrared images. This algorithm simulates the human visual system and can effectively improve the image quality under low illumination conditions. The preprocessing of thermal infrared images includes adaptive median filtering to remove salt and pepper noise, and adaptive local contrast enhancement algorithm to improve the image recognizability.

[0095] Geometric correction is an important step to ensure the spatial consistency of multi-band images. First, distortion correction is performed based on the camera intrinsic parameters obtained through calibration to eliminate radial and tangential distortion caused by the lens. Then, feature point matching and perspective transformation are used to align the multi-band images. Specifically, the SIFT algorithm (scale-invariant feature transform) is used to extract feature points, the RANSAC algorithm (random sample consensus algorithm) is used to remove mismatched points, and finally the perspective transformation matrix is ​​calculated to achieve image alignment.

[0096] Spectral correction includes radiation correction and atmospheric correction. Radiation correction uses the sensor response curve to convert the image grayscale value into radiation brightness value. Atmospheric correction uses the MODTRAN atmospheric radiation transfer model to consider the effects of atmospheric absorption and scattering to restore the true spectral reflectance of the ground object.

[0097] Image fusion aims to make comprehensive use of multi-band information. At the feature level, wavelet transform is used for fusion, which decomposes images of different bands into low-frequency approximations and high-frequency details, and then reconstructs the fused image according to the fusion rules. At the decision level, fusion is performed based on the belief function theory, and different weights are assigned to each band to achieve complementary advantages.

[0098] Dimensionality reduction helps reduce data redundancy and improve subsequent processing efficiency. Principal component analysis converts the original features into linearly independent principal components through orthogonal transformation, retaining the first few components with the largest variance contribution. Independent component analysis assumes that the data is a linear mixture of independent source signals, and separates these source signals by maximizing statistical independence.

[0099] Data quality control is an important step to ensure the reliability of subsequent analysis. Image quality assessment includes calculating indicators such as signal-to-noise ratio, clarity and contrast. Anomaly detection uses statistical methods and machine learning algorithms to identify outliers and abnormal patterns. Data verification evaluates the accuracy of preprocessing results by comparing with ground-truth data.

[0100] Feature extraction is the bridge between image data and capsule expansion detection. Texture features are obtained by calculating the gray-level co-occurrence matrix, including statistics such as energy, contrast, correlation, and entropy. Shape features are extracted through morphological operations, such as using opening and closing operations to extract capsule contours and calculating indicators such as area, perimeter, and circularity. Spectral features include the normalized difference vegetation index, which uses the difference in reflectance between near-infrared and red light bands to effectively distinguish between vegetation cover and exposed rock.

[0101] Through the above steps, high-quality multispectral image data and rich feature information can be obtained, laying a solid foundation for the subsequent detection and positioning of the capsule expansion. For example, in a tunnel detection case, the image clarity processed by this method was improved by 40%, the feature point matching accuracy reached the sub-pixel level, and the information entropy of the fused image increased by 25%, providing a reliable guarantee for the accurate identification of tiny capsule deformation.

[0102] In an optional embodiment,

[0103] The steps of extracting and classifying features of the preprocessed image and identifying the dilation area using a deep learning model, wherein the deep learning model includes an improved multi-scale feature pyramid network and an attention mechanism module include:

[0104] The improved multi-scale feature pyramid network is used for feature extraction, including a dense connection module, a spatial pyramid pooling module and a feature recalibration module; the dense connection module enhances feature propagation through multiple convolutions and feature connections; the spatial pyramid pooling module performs pooling operations at different scales and captures multi-scale context information through upsampling; the feature recalibration module includes a channel attention mechanism to adaptively weight feature channels;

[0105] The attention mechanism module is integrated with the improved multi-scale feature pyramid network, including a channel attention submodule and a spatial attention submodule; the channel attention submodule calculates channel weights through a shared multi-layer perceptron using a combination of global average pooling and global maximum pooling; the spatial attention submodule generates a spatial attention map using feature maps of average pooling and maximum pooling;

[0106] The preprocessed image is input into the deep learning model, and processed by the improved multi-scale feature pyramid network and the attention mechanism module in sequence to obtain a multi-scale feature representation; the multi-scale feature representation is adaptively weighted fused, and the adaptive fusion of features is achieved by learning the weight of each scale feature to obtain a fused feature; the fused feature is input into a classifier for dilation area recognition, and the classifier includes a global average pooling layer and a fully connected layer.

[0107] Exemplarily, the tunnel capsule expansion detection method based on deep learning is an advanced image analysis technology that uses an improved multi-scale feature pyramid network and an attention mechanism module to achieve high-precision expansion area recognition. The core of this method lies in its powerful feature extraction and classification capabilities, which can effectively handle complex tunnel environments and diverse expansion forms.

[0108] The improved multi-scale feature pyramid network is the basis of this method. It consists of a dense connection module, a spatial pyramid pooling module, and a feature recalibration module. The dense connection module adopts a structure similar to DenseNet (densely connected convolutional network), which enhances feature propagation through multiple convolutions and feature connections. Specifically, the module contains multiple convolutional layers, and the input of each convolutional layer comes not only from the output of the previous layer, but also from the output of all previous layers. This dense connection structure can effectively alleviate the gradient vanishing problem, promote feature reuse, and thus improve the expressiveness of the network. For example, in a dense connection module containing 5 convolutional layers, the input of the 5th layer will contain all feature maps from the first 4 layers, allowing the network to make full use of shallow and deep features.

[0109] The spatial pyramid pooling module usually contains 4 parallel pooling branches, corresponding to the pooling kernel sizes of 1×1, 2×2, 3×3 and 6×6 respectively. The output of each branch is adjusted by 1×1 convolution to adjust the number of channels, and then upsampled to the original feature map size through bilinear interpolation. Finally, these feature maps of different scales are concatenated with the original feature map to form a multi-scale feature representation containing rich contextual information. This structure can effectively capture dilated areas of different sizes and improve the robustness of detection.

[0110] The feature recalibration module introduces a channel attention mechanism to adaptively weight feature channels. The module first compresses the feature map of each channel into a scalar through global average pooling, then uses two fully connected layers to learn the correlation between channels, and finally generates the weight of each channel through the sigmoid function. This mechanism can highlight important channel features and suppress irrelevant channels, thereby improving the expressiveness of features. In practical applications, this module can significantly improve the model's ability to recognize different types of expansion features. For example, for mild and severe expansion, the model can adaptively adjust the importance of different channels.

[0111] The attention mechanism module is tightly integrated with the improved multi-scale feature pyramid network, including the channel attention submodule and the spatial attention submodule. The channel attention submodule uses a combination of global average pooling and global maximum pooling to calculate the channel weights through a shared multi-layer perceptron. Specifically, the input feature map is subjected to global average pooling and global maximum pooling respectively to obtain two one-dimensional vectors. These two vectors are passed through a shared multi-layer perceptron (usually containing two fully connected layers with a ReLU activation function in between), then added together and passed through a sigmoid function to obtain the final channel attention weights. This design can simultaneously utilize the average information and the most significant features to improve the effect of the attention mechanism.

[0112] The spatial attention submodule generates a spatial attention map using the feature maps of average pooling and maximum pooling. First, the input feature map is average pooled and max pooled in the channel dimension to obtain two two-dimensional feature maps. These two feature maps are concatenated and a 7×7 convolution layer is applied, and finally the spatial attention map is obtained through the sigmoid function. Spatial attention can highlight important spatial locations in the image, which helps the model focus on potential expansion areas.

[0113] In practical applications, the preprocessed multispectral image is first input into the improved multi-scale feature pyramid network. The network extracts the underlying features through multiple convolutional layers and pooling layers, and then enhances the features through the dense connection module. Next, the spatial pyramid pooling module captures the multi-scale context information, and the feature recalibration module adaptively weights the channels. These processed features are then modulated by the channel attention and spatial attention through the attention mechanism module to obtain the final multi-scale feature representation.

[0114] Adaptive weighted fusion of the obtained multi-scale feature representation is a key step to improve detection accuracy. This process achieves adaptive fusion of features by learning the weights of each scale feature. Specifically, 1×1 convolution and global average pooling are applied to the feature map of each scale to obtain a set of weight coefficients. These weight coefficients are normalized by the softmax function and used to weighted sum the features of different scales to obtain the final fused features. This adaptive fusion method can automatically adjust the importance of features of different scales according to the characteristics of the input image, and improve the model's adaptability to expansion areas of different sizes and shapes.

[0115] Finally, the fused features are input into the classifier for dilation area identification. The classifier includes a global average pooling layer and a fully connected layer. The global average pooling layer compresses the spatial dimension of the feature map and retains the channel information, which helps to reduce the number of parameters and improve the generalization ability of the model. The fully connected layer is responsible for mapping the features to the final category probability distribution. In the binary classification task (dilation area and non-dilation area), the number of output nodes of the fully connected layer is 2, and the probability of each category is obtained through the softmax function.

[0116] In actual application cases, this method has achieved remarkable results in a tunnel detection project. By using an improved multi-scale feature pyramid network and attention mechanism, the model achieved an accuracy of 97.8% on the test set, which is 5.6 percentage points higher than the traditional convolutional neural network. In particular, for the detection of slight expansion, the recall rate increased from 85.3% to 93.7%, significantly reducing the missed detection rate. At the same time, the model showed strong robustness when processing images with different lighting conditions and complex backgrounds, providing reliable technical support for tunnel safety monitoring.

[0117] In an optional embodiment,

[0118] The multi-scale feature representation is adaptively weighted fused, and the weight of each scale feature is learned to achieve adaptive fusion of features to obtain fused features; the fused features are input into a classifier for dilation area recognition, and the classifier includes a global average pooling layer and a fully connected layer. The steps include:

[0119] The preprocessed image is input into the deep learning model, and the feature representation of the image is extracted at multiple scales through multiple downsampling and upsampling operations to obtain feature maps of K different scales;

[0120] The K feature maps extracted are sent to the 1×1 convolution layer for dimensionality reduction to obtain the corresponding K weight maps, and then the K weight maps are normalized using the normalized exponential function to ensure that the sum of the weights at each spatial position is 1;

[0121] Multiply K feature maps by their corresponding weight maps element by element, and add the weighted feature maps to obtain an adaptively fused multi-scale feature representation.

[0122] The adaptively fused features are compressed into a feature vector of fixed length through a global average pooling layer;

[0123] The pooled feature vector is passed through the fully connected layer and the dropout layer with L2 regularization in turn, and finally through the fully connected layer activated by the normalized exponential function to output the probability of each category and complete the classification of the expansion area.

[0124] For example, multi-scale feature adaptive weighted fusion and classification is a key link in the deep learning model in tunnel capsule expansion detection. It can effectively integrate feature information of different scales and improve the model's recognition accuracy of the expansion area. This method achieves adaptive fusion of features by learning the weight of each scale feature, and uses a specially designed classifier for the final recognition of the expansion area.

[0125] First, the preprocessed tunnel image is input into the deep learning model. The model extracts feature representations of the image at multiple scales through multiple downsampling and upsampling operations. This process usually involves a combination of a series of convolutional layers and pooling layers. For example, a typical multi-scale feature extraction network may contain 5 scales, corresponding to 1 / 1, 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the original resolution. The feature map of each scale is extracted through convolutional layers of different depths to capture various features from low-level textures to high-level semantics. Assuming the input image size is 512×512 pixels, the feature map sizes of the 5 scales are 512×512, 256×256, 128×128, 64×64, and 32×32, respectively.

[0126] Next, the extracted K feature maps (K=5 in this example) are sent to the 1×1 convolution layer for dimensionality reduction. The main function of 1×1 convolution is to reduce the number of channels while keeping the spatial dimension of the feature map unchanged. For example, if the number of channels of the original feature map is 256, it can be reduced to 64 through 1×1 convolution. This not only reduces the amount of calculation, but also increases the nonlinear expression ability of the model. After dimensionality reduction, K corresponding weight maps are obtained.

[0127] In order to ensure that the weights of features of different scales are reasonably distributed, the K weight maps are normalized using a normalized exponential function (softmax). This operation ensures that the sum of the weights of all scales at each spatial position is 1. The normalized weights reflect the importance of each scale feature at that position. For example, for a specific position in an image, the weights of the five scales may be 0.1, 0.2, 0.4, 0.2, and 0.1, respectively, indicating that the features of the middle scale are the most important at that position.

[0128] Then, the K original feature maps are element-wise multiplied with the corresponding normalized weight maps. This operation implements adaptive weighting of features. For each spatial position, important scale features will receive higher weights, while less important features will be suppressed. The weighted K feature maps are added together to obtain an adaptively fused multi-scale feature representation. This fused feature integrates information from each scale and can better express the dilated area in the image.

[0129] The adaptively fused features are compressed into a fixed-length feature vector through the global average pooling layer. The advantage of global average pooling is that it is not affected by the size of the input feature map and always outputs a vector of fixed dimension, which is beneficial to the design of subsequent classifiers. For example, if the size of the fused feature map is 64×64×256 (height×width×number of channels), it will become a 256-dimensional vector after global average pooling.

[0130] Next, the pooled feature vector is passed through a fully connected layer with L2 regularization and a dropout layer. L2 regularization helps prevent the model from overfitting by adding a penalty term of the sum of squared weights to the loss function. The dropout layer randomly sets a certain percentage of neuron outputs to zero, further enhancing the generalization ability of the model. For example, a fully connected layer with 1024 neurons can be used, with an L2 regularization coefficient of 0.01 and a dropout rate of 0.5.

[0131] Finally, the features pass through a fully connected layer activated with a normalized exponential function. The number of neurons in this fully connected layer is equal to the number of categories, which is 2 neurons for a binary classification problem (expansion area and non-expansion area). The normalized exponential function ensures that the sum of the two output values ​​is 1, which can be interpreted as the probability of each category. For example, if the output is [0.7, 0.3], it means that there is a 70% probability that the area is an expansion area and a 30% probability that it is a non-expansion area.

[0132] In practical applications, this method has achieved remarkable results in a highway tunnel inspection project. By using multi-scale feature adaptive weighted fusion, the model achieved an accuracy of 98.5% on 1,000 test images, which is 3.2 percentage points higher than the single-scale method. In particular, for slight expansion detection under complex backgrounds, the recall rate increased from 89.1% to 95.6%, significantly reducing the missed detection rate. At the same time, the model showed strong robustness in processing images with different lighting conditions and complex textures, providing reliable technical support for tunnel safety monitoring. This method not only improves the accuracy of expansion area identification, but also enhances the model's ability to adapt to features of different scales, laying a solid foundation for the intelligence and automation of tunnel structure health monitoring systems.

[0133] In an optional embodiment,

[0134] The improved structured light 3D reconstruction technology is used to establish a tunnel 3D model. The improved structured light 3D reconstruction technology includes using a phase shift method to improve the accuracy of depth information, and applying an adaptive multi-frequency phase unwrapping algorithm, including the following steps:

[0135] Projecting a plurality of sinusoidal grating patterns with a fixed phase difference onto the tunnel surface, wherein the light intensity distribution of the sinusoidal grating pattern is composed of background light intensity, modulated light intensity and object surface phase distribution;

[0136] Collect multiple images reflected from the tunnel surface, and calculate the phase value of the tunnel surface by the least square method to obtain the wrapped phase;

[0137] Adopting an adaptive multi-frequency phase unwrapping algorithm to unwrap the wrapped phase, including projecting a low-frequency grating pattern and calculating a low-frequency phase, projecting an intermediate-frequency grating pattern and calculating an intermediate-frequency phase, and projecting a high-frequency grating pattern and calculating a high-frequency phase;

[0138] According to the low-frequency phase, medium-frequency phase and high-frequency phase, an adaptive threshold mechanism is used to perform phase unwrapping to obtain the absolute phase value; the adaptive threshold mechanism dynamically adjusts the unwrapping threshold by analyzing the phase difference between adjacent frequencies to handle the situation where the tunnel surface is discontinuous or blocked;

[0139] Based on the absolute phase value, a weighted average strategy is adopted to calculate the depth information of the tunnel surface;

[0140] Point cloud registration technology is used to integrate depth information obtained from multiple perspectives into a complete three-dimensional tunnel model.

[0141] Exemplarily, the improved structured light 3D reconstruction technology is a high-precision and high-efficiency method for constructing a tunnel 3D model. This technology combines the phase shift method and the adaptive multi-frequency phase unwrapping algorithm, which can effectively cope with the complex surface and lighting conditions in the tunnel environment, and achieve accurate depth information extraction and 3D reconstruction.

[0142] First, in the projection stage, the system projects multiple sinusoidal grating patterns with fixed phase differences onto the tunnel surface. The light intensity distribution of these sinusoidal grating patterns consists of three parts: background light intensity, modulated light intensity, and phase distribution on the object surface. A typical setup may include projecting 8 sinusoidal grating patterns with a phase difference of π / 4. For example, a digital light processing (DLP) projector with a resolution of 1920×1080 is used to project sinusoidal grating patterns with a frequency of 300 lines / meter. The background light intensity and modulated light intensity can be determined through a pre-calibration process. Usually, the background light intensity is set to 128 (in the grayscale range of 0-255) and the modulated light intensity is set to 100 to ensure good contrast and signal-to-noise ratio.

[0143] Next, a high-resolution camera captures multiple images reflected from the tunnel surface. These images contain coded information about the geometry of the tunnel surface. Using an industrial-grade camera, the resolution can reach 4096×3000 pixels, ensuring that enough details are captured. The captured images are then used to calculate the phase value of the tunnel surface using the least squares method. This process actually fits the brightness value of each pixel under different grating patterns to a sine function to obtain the phase value of the point. The calculated phase value ranges from 0 to 2π and is called the wrapped phase.

[0144] Since there is periodic ambiguity in the wrapped phase, an adaptive multi-frequency phase unwrapping algorithm is required for phase unwrapping. This algorithm includes grating projections at three frequencies: low frequency, medium frequency, and high frequency. For example, grating patterns with frequencies of 30 lines / m, 150 lines / m, and 300 lines / m can be selected. First, the low-frequency grating pattern is projected and the low-frequency phase is calculated. Although this phase has lower accuracy, there is no periodic ambiguity. Then the medium-frequency grating pattern is projected and the medium-frequency phase is calculated. Finally, the high-frequency grating pattern is projected and the high-frequency phase is calculated. The high-frequency phase has the highest accuracy, but there is serious periodic ambiguity.

[0145] The core of the adaptive multi-frequency phase unwrapping algorithm is to use an adaptive threshold mechanism for phase unwrapping. This mechanism dynamically adjusts the unwrapping threshold by analyzing the phase difference between adjacent frequencies to deal with discontinuities or occlusions on the tunnel surface. Specifically, the algorithm first uses the low-frequency phase as a reference to unwrap the intermediate-frequency phase. In this process, if the phase difference between two adjacent points exceeds a preset threshold (for example, π / 2), it is considered that there may be a phase jump and phase correction is required. The choice of threshold is adaptive and dynamically adjusted according to the statistical characteristics of the local phase difference. For example, in smooth surface areas, the threshold may be set small (such as π / 4), while at edges or discontinuous areas, the threshold may be increased to π.

[0146] The same process is used to unfold the phase from the medium frequency to the high frequency. In this way, the algorithm can effectively deal with the discontinuity and occlusion problems of the tunnel surface and obtain accurate absolute phase values. In practical applications, the algorithm can reduce the error rate of phase unfolding from 5% of the traditional method to less than 0.5%, significantly improving the accuracy of reconstruction.

[0147] Based on the absolute phase value obtained, a weighted average strategy is used to calculate the depth information of the tunnel surface. The weighted average here takes into account the accuracy and reliability of different frequency phases. Usually, the high-frequency phase accounts for a larger proportion in the weight distribution (such as 0.6), and the medium-frequency and low-frequency phases account for 0.3 and 0.1 respectively. This strategy can effectively suppress the influence of noise while maintaining high accuracy. The calculation of depth information is based on the principle of triangulation, and the phase value is converted into an actual depth value through pre-calibrated system parameters (such as the internal and external parameters of the camera and projector).

[0148] Finally, point cloud registration technology is used to integrate the depth information obtained from multiple perspectives into a complete tunnel 3D model. This usually involves two steps: coarse registration and fine registration. Coarse registration can use feature-based methods, such as SIFT (Scale Invariant Feature Transform) with RANSAC (Random Sample Consistency) algorithm to quickly estimate the approximate transformation relationship between point clouds from different perspectives. The fine matching criterion often uses the iterative closest point (ICP) algorithm to accurately align point cloud data from different perspectives through iterative optimization. In practical applications, for a 100-meter-long tunnel section, it may be necessary to collect 50-100 point cloud data from different perspectives. Through point cloud registration, these data are integrated into a complete and accurate tunnel 3D model, with a point cloud density of up to 1000 points / square meter and an accuracy of up to millimeter level.

[0149] This improved structured light 3D reconstruction technology has achieved remarkable results in a highway tunnel inspection project. Compared with the traditional laser scanning method, this technology not only improves the reconstruction accuracy (from centimeters to millimeters), but also greatly improves the acquisition efficiency, reducing the scanning time of a 100-meter tunnel section from 4 hours to 1 hour. At the same time, thanks to the adaptive multi-frequency phase unwrapping algorithm, the technology performs well when dealing with complex surfaces such as moisture and reflections, and the error reconstruction rate is reduced by 80%. This high-precision and high-efficiency 3D reconstruction method provides a reliable data basis for tunnel structural health monitoring and deformation analysis, greatly improving the level of tunnel maintenance and safety management.

[0150] In an optional embodiment,

[0151] According to the spatial position and range of the expansion area, combined with the preset hazard level standard, the expansion area is risk assessed and graded, and the detection results including the location, range and risk level of the expansion area are generated. The detection results are visualized on the three-dimensional model of the tunnel, and the maintenance and reinforcement plan based on the reinforcement learning algorithm is generated. The steps include:

[0152] Obtaining spatial location and range information of tunnel expansion area;

[0153] Calculating the risk index of the expansion area using a weighted summation method based on a multi-factor risk assessment model, wherein the multi-factor risk assessment model includes expansion area, expansion depth, location sensitivity, tunnel age and geological conditions;

[0154] According to the calculated risk index, the expansion area is divided into different risk levels;

[0155] Using 3D visualization technology, the inspection results, including the location, range and risk level of the expansion area, are presented on the 3D model of the tunnel;

[0156] Construct a reinforcement learning environment, including defining the state space, action space, and reward function. The state space includes the risk index, location, range, and current maintenance budget of the expansion area. The action space includes a variety of maintenance measures. The reward function is designed based on the risk reduction degree and cost-effectiveness ratio after maintenance. Use a deep Q-learning algorithm to train a maintenance decision model. Based on the trained maintenance decision model, generate maintenance and reinforcement plans for each expansion area.

[0157] The generated maintenance and reinforcement plan is displayed in a 3D visualization interface. The tunnel status information is updated in real time, and the multi-factor risk assessment model and maintenance decision model are continuously optimized based on new inspection data and maintenance effect feedback. The optimized multi-factor risk assessment model and maintenance decision model are applied to subsequent risk assessment and maintenance plan formulation to form a closed-loop optimization mechanism.

[0158] For example, risk assessment, visualization and maintenance plan formulation of tunnel expansion areas based on reinforcement learning is a comprehensive technical solution that aims to improve the intelligence and precision of tunnel safety management. The solution combines multi-factor risk assessment, 3D visualization and artificial intelligence technology to achieve intelligent management of the entire process from detection to decision-making.

[0159] First, obtain the spatial position and range information of the tunnel expansion area. This information usually comes from the detection results of the aforementioned improved structured light 3D reconstruction technology and deep learning model. For example, for a tunnel section with a length of 100 meters and a diameter of 10 meters, 5 expansion areas may be detected. The location of each area is expressed by the mileage and circumferential angle from the tunnel entrance, and the range is described by the area and maximum depth. For a specific expansion area, its information may be: location (50 meters, 225 degrees), area (2 square meters), and maximum depth (15 centimeters).

[0160] Next, the risk index of the expansion area is calculated using a weighted summation method based on a multi-factor risk assessment model. The model takes into account five main factors: expansion area, expansion depth, location sensitivity, tunnel age, and geological conditions. The weight of each factor is determined based on expert experience and historical data, such as expansion area (30%), expansion depth (25%), location sensitivity (20%), tunnel age (15%), and geological conditions (10%). For each factor, a score of 0-100 is mapped according to its actual value. For example, the expansion area can be divided into less than 1 square meter (20 points), 1-3 square meters (40 points), 3-5 square meters (60 points), 5-10 square meters (80 points), and greater than 10 square meters (100 points). Other factors are mapped similarly. The final risk index is the sum of these weighted scores, ranging from 0 to 100.

[0161] According to the calculated risk index, the expansion area is divided into different risk levels. Generally, it can be divided into five levels: low risk (0-20), medium-low risk (21-40), medium risk (41-60), medium-high risk (61-80) and high risk (81-100). Each level corresponds to a different warning color and processing priority.

[0162] Using 3D visualization technology, the detection results including the location, range and risk level of the expansion area are presented on the 3D model of the tunnel. Here, a 3D engine such as Unity3D or Unreal Engine can be used to import and render the tunnel model. The expansion area can be represented by a semi-transparent overlay of different colors, and the color corresponds to the risk level (such as green, yellow, orange, red, dark red). Users can roam freely in the 3D model, and clicking on the expansion area will pop up a detailed information window, showing the specific risk index and various parameters.

[0163] Building a reinforcement learning environment is a key step in developing intelligent maintenance plans. The state space includes the risk index, location, range, and current maintenance budget of the expansion area. For example, the state can be represented as a vector: [risk index, X coordinate, Y coordinate, Z coordinate, area, depth, available budget]. The action space includes a variety of maintenance measures, such as monitoring, grouting, lining repair, reinforcement, etc., each of which can have different intensity levels. The reward function is designed based on the risk reduction degree and cost-effectiveness ratio after maintenance. For example, it can be defined as: risk reduction percentage * 100 - maintenance cost / 1000. In this way, both the effective reduction of risks is encouraged and excessive maintenance costs are punished.

[0164] The maintenance decision model is trained using a deep Q-learning algorithm. The algorithm uses a deep neural network to approximate the Q-value function. The network structure can be a multi-layer perceptron, where the input layer corresponds to the state vector and the output layer corresponds to the Q-value of each possible action. During the training process, the ε-greedy strategy can be used to balance exploration and exploitation. The initial ε is set to 1 and gradually decreases to 0.1 as the training progresses. After a large number of interactions in the simulated environment (for example, 1 million times), the model gradually learns to select the optimal maintenance action under different states.

[0165] Based on the trained maintenance decision model, a maintenance and reinforcement plan is generated for each expansion area. The model inputs the current state and outputs the action with the highest Q value as the recommended plan. For example, for an expansion area with a risk index of 75, an area of ​​3 square meters, and a depth of 10 centimeters, the model may recommend "medium-strength grouting + local lining repair" as the best plan.

[0166] The generated maintenance and reinforcement plans are displayed in a 3D visualization interface. Icons or labels can be added above the expansion area, and after clicking, detailed maintenance suggestions are displayed, including specific measures, expected results and estimated costs. At the same time, an overview panel can be provided, listing the maintenance plans for all expansion areas, and sorted by risk level and cost-effectiveness.

[0167] Update tunnel status information in real time, and continuously optimize the multi-factor risk assessment model and maintenance decision model based on new inspection data and maintenance effect feedback. For example, after each maintenance work is completed, record the actual risk reduction and cost, and use this data to adjust the weights of each factor in the risk assessment model and the reward function of the maintenance decision model. Model updates can be performed quarterly to ensure that the model always reflects the latest tunnel conditions and maintenance experience.

[0168] The optimized multi-factor risk assessment model and maintenance decision model are applied to subsequent risk assessment and maintenance plan formulation to form a closed-loop optimization mechanism. This mechanism ensures that the entire system can continuously learn and improve, adapt to the dynamic changes in tunnel conditions and the introduction of new maintenance technologies.

[0169] In an optional embodiment,

[0170] Constructing a reinforcement learning environment includes defining a state space, an action space, and a reward function, wherein the state space includes the risk index, location, range, and current maintenance budget of the expansion area, the action space includes a variety of maintenance measures, and the reward function is designed based on the risk reduction degree and cost-effectiveness ratio after maintenance; using a deep Q learning algorithm to train a maintenance decision model, and based on the trained maintenance decision model, the steps of generating a maintenance and reinforcement plan for each expansion area include:

[0171] The state space includes expansion area characteristics, tunnel global characteristics and maintenance resource information, wherein the expansion area characteristics include risk index, location coordinates and area, the tunnel global characteristics include tunnel age, geological condition score and traffic load, and the maintenance resource information includes available budget, human resources and equipment resources;

[0172] The action space is defined as a set of discrete maintenance measures, including grouting reinforcement, bolting, lining repair, and no-action options;

[0173] Design a multi-objective reward function, including risk index reduction, safety improvement, maintenance cost, and long-term performance impact;

[0174] Construct a deep Q network based on long short-term memory network and attention mechanism to approximate the Q function;

[0175] The deep Q network includes a long short-term memory layer, a multi-head attention layer, and a fully connected layer for processing high-dimensional state space and capturing temporal dependencies;

[0176] Implement a prioritized experience replay buffer, assign a priority to each experience sample based on the timing difference error, and use importance sampling for bias correction;

[0177] Adopting the target network and soft update strategy, the target Q value is calculated by mixing the output of the online network and the target network, and the target network parameters are gradually updated using the soft update coefficients;

[0178] Design an ε-greedy strategy with a decay mechanism, perform extensive exploration in the early stages of training, and execute a training loop, including interaction with the environment, experience storage, batch sampling, loss calculation, parameter update, and target network soft update steps;

[0179] Using the trained deep Q network, the optimal maintenance plan is generated through the Monte Carlo tree search method according to the current tunnel status. The generated maintenance plan is applied to the tunnel management system, and feedback on the implementation effect is collected. The environmental model and reward function are updated according to the feedback data to achieve continuous optimization and adaptive adjustment of the algorithm.

[0180] For example, building a reinforcement learning environment is the core step to achieve intelligent tunnel maintenance decision-making, which includes defining the state space, action space and reward function, and training the maintenance decision model using a deep Q learning algorithm. This process involves multiple key technical points that need to be elaborated in detail to ensure the effectiveness and reliability of the system.

[0181] First, defining the state space is the basis for building a reinforcement learning environment. The state space consists of three main parts: expansion area characteristics, tunnel global characteristics, and maintenance resource information. Expansion area characteristics include risk index (continuous value of 0-100), location coordinates (x, y, z, in meters), and area (square meters). For example, a specific expansion area state may be: risk index 75, location coordinates (500, 3, 2), and area 5 square meters. Tunnel global characteristics include tunnel age (years), geological condition score (discrete value of 0-10), and traffic load (daily traffic volume). Maintenance resource information includes available budget (10,000 yuan), human resources (number of man-hours), and equipment resources (number of available equipment). The state space designed in this way can fully reflect the current status and available resources of the tunnel, providing a sufficient information basis for decision-making.

[0182] The action space is defined as all possible actions that the reinforcement learning algorithm can choose. In the tunnel maintenance scenario, the action space is defined as a set of discrete maintenance measures, including grouting reinforcement, bolting support, lining repair, and no action options. Each maintenance measure can be further subdivided into different intensity levels, for example, grouting reinforcement can be divided into mild (less than 10 square meters), moderate (10-30 square meters), and severe (more than 30 square meters). This design allows the algorithm to select the most suitable maintenance strategy and intensity in different situations.

[0183] The design of the reward function directly affects the behavior of the reinforcement learning algorithm. In order to fully consider the multiple objectives of maintenance decision-making, a multi-objective reward function is designed, including the reduction in risk index, the degree of safety improvement, maintenance cost, and long-term performance impact. The reduction in risk index can be directly expressed as the difference in risk index before and after maintenance. The degree of safety improvement can be quantified by setting a safety threshold and calculating the degree of improvement above the threshold. Maintenance costs include direct costs (material and labor costs) and indirect costs (such as economic losses caused by traffic interruptions). Long-term performance impact can be evaluated by estimating changes in maintenance requirements in the next 5-10 years through a predictive model. These factors are combined into the final reward value through weighted summation, and the weights can be dynamically adjusted according to the manager's priorities.

[0184] Building a deep Q network based on long short-term memory (LSTM) and attention mechanism is the key to achieve efficient Q function approximation. The network structure includes an LSTM layer to handle the temporal dependency of state sequences, a multi-head attention layer to capture the association between different state features, and several fully connected layers for final Q value prediction. The number of hidden units in the LSTM layer can be set to 128 or 256, adjusted according to the complexity of the state space. The multi-head attention layer can use 8 attention heads, each with a dimension of 64. The number of nodes in the fully connected layer can be reduced layer by layer, such as 512, 256, 128, and finally the number of nodes in the output layer is equal to the size of the action space.

[0185] Implementing a priority experience replay buffer is an important technique to improve learning efficiency. Each experience sample is assigned a priority, which can be set to 1 initially and then updated based on the absolute value of the temporal difference error. When sampling, the square root of the priority is used as the probability distribution to balance the selection frequency of high priority samples. In order to correct the bias introduced by non-uniform sampling, importance sampling weights are used for adjustment. The importance sampling weights can be calculated by the normalized value of the priority and used in the loss function.

[0186] The use of a target network and a soft update strategy can improve the stability of training. The structure of the target network is the same as the online network, but the parameter update frequency is lower. When calculating the target Q value, the output of the online network and the target network are mixed, and the mixing ratio can be set to 0.5. The soft update of the target network uses a small update coefficient, such as 0.01, to gradually transfer the parameters of the online network to the target network.

[0187] During training, an ε-greedy strategy with a decay mechanism is used for exploration. The initial ε value is set to 1, allowing completely random action selection. As training progresses, the ε value gradually decays, and an exponential decay strategy can be used to multiply the ε value by 0.99 every 1000 training steps until it reaches a minimum value of 0.01. This ensures sufficient exploration in the early stages of training and more use of the learned strategy in the later stages.

[0188] Executing the training loop is the core of the entire learning process. Each training step includes interacting with the environment, storing experience in a priority replay buffer, sampling in batches from the buffer (e.g., batch size 32), calculating the loss, updating the network parameters, and soft updating the target network. The loss calculation uses the temporal difference error and takes into account the importance sampling weights. The parameter update can use the Adam optimizer with a learning rate of 0.0001.

[0189] After training, the deep Q network is used to generate maintenance plans. In order to improve the robustness of the decision, the Monte Carlo Tree Search (MCTS) method can be combined. MCTS simulates multiple decision sequences and evaluates the long-term value of each action to select the optimal maintenance plan. The simulation depth can be set to 10 steps, and each action can be simulated 100 times to balance the computational complexity and decision quality.

[0190] After applying the generated maintenance plan to the actual tunnel management system, it is important to continuously collect feedback data on the implementation effect. This data includes actual risk index changes, maintenance costs, safety improvements, etc. Based on this feedback, the environmental model and reward function can be updated, such as adjusting the weights of the risk assessment model or modifying the weights of various indicators in the reward function. This continuous optimization and adjustment mechanism ensures that the algorithm can adapt to dynamic changes in tunnel conditions and new maintenance technologies.

[0191] Figure 2 FIG. 1 is a schematic diagram of the structure of a capsule expansion detection and positioning system based on machine vision according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0192] The first unit is used to collect tunnel capsule images using multispectral imaging technology; preprocess the tunnel capsule images; use a deep learning model to extract features and classify the preprocessed images to identify the expansion area, wherein the deep learning model includes an improved multi-scale feature pyramid network and an attention mechanism module; perform edge detection and contour extraction on the expansion area to obtain the precise boundary of the expansion area; and calculate the area, perimeter and shape characteristics of the expansion area based on the extracted contour information;

[0193] The second unit is used to establish a three-dimensional model of the tunnel using an improved structured light three-dimensional reconstruction technology, wherein the improved structured light three-dimensional reconstruction technology includes using a phase shift method to improve the accuracy of depth information and applying an adaptive multi-frequency phase unwrapping algorithm; mapping the detected expansion area to the three-dimensional model of the tunnel to determine the spatial position and range of the expansion area; performing risk assessment and grading of the expansion area based on the spatial position and range of the expansion area in combination with a preset hazard level standard, generating a detection result including the position, range and risk level of the expansion area, visually presenting the detection result on the three-dimensional model of the tunnel, and generating a maintenance and reinforcement plan based on a reinforcement learning algorithm;

[0194] The third unit is used to establish a historical database to record the detection results and the changing trends of the expansion area; based on the historical data and current detection results, the time series analysis method is used to predict the development trend and potential risks of the expansion area.

[0195] According to a third aspect of the embodiments of the present invention,

[0196] An electronic device is provided, comprising:

[0197] processor;

[0198] a memory for storing processor-executable instructions;

[0199] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0200] A fourth aspect of the embodiments of the present invention is:

[0201] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0202] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0203] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting and locating bladder expansion based on machine vision, characterized in that: include: Multispectral imaging technology is used to collect images of the tunnel capsule; Preprocessing the tunnel capsule image; Using a deep learning model to extract and classify features of the preprocessed image to identify the dilation area, the deep learning model includes an improved multi-scale feature pyramid network and an attention mechanism module; Perform edge detection and contour extraction on the expansion area to obtain the precise boundary of the expansion area; calculate the area, perimeter and shape characteristics of the expansion area based on the extracted contour information; A three-dimensional model of the tunnel is established by using an improved structured light three-dimensional reconstruction technology, wherein the improved structured light three-dimensional reconstruction technology includes using a phase shift method to improve the accuracy of depth information and applying an adaptive multi-frequency phase unwrapping algorithm; mapping the detected expansion area to the three-dimensional model of the tunnel to determine the spatial position and range of the expansion area; performing risk assessment and grading of the expansion area based on the spatial position and range of the expansion area in combination with a preset hazard level standard, generating a detection result including the position, range and risk level of the expansion area, visually presenting the detection result on the three-dimensional model of the tunnel, and generating a maintenance and reinforcement plan based on a reinforcement learning algorithm; Establish a historical database to record the test results and the changing trends of the expansion area; use time series analysis methods based on historical data and current test results to predict the development trend and potential risks of the expansion area; The improved structured light 3D reconstruction technology is used to establish a tunnel 3D model. The improved structured light 3D reconstruction technology includes using a phase shift method to improve the accuracy of depth information, and applying an adaptive multi-frequency phase unwrapping algorithm, including the following steps: Projecting a plurality of sinusoidal grating patterns with a fixed phase difference onto the tunnel surface, wherein the light intensity distribution of the sinusoidal grating pattern is composed of background light intensity, modulated light intensity and object surface phase distribution; Collect multiple images reflected from the tunnel surface, and calculate the phase value of the tunnel surface by the least square method to obtain the wrapped phase; Adopting an adaptive multi-frequency phase unwrapping algorithm to unwrap the wrapped phase, including projecting a low-frequency grating pattern and calculating a low-frequency phase, projecting an intermediate-frequency grating pattern and calculating an intermediate-frequency phase, and projecting a high-frequency grating pattern and calculating a high-frequency phase; According to the low-frequency phase, medium-frequency phase and high-frequency phase, an adaptive threshold mechanism is used to perform phase unwrapping to obtain the absolute phase value; the adaptive threshold mechanism dynamically adjusts the unwrapping threshold by analyzing the phase difference between adjacent frequencies to handle the situation where the tunnel surface is discontinuous or blocked; Based on the absolute phase value, a weighted average strategy is adopted to calculate the depth information of the tunnel surface; Point cloud registration technology is used to integrate depth information obtained from multiple perspectives into a complete three-dimensional tunnel model.

2. The method according to claim 1, characterized in that Multispectral imaging technology is used to collect images of the tunnel capsule; The steps of preprocessing the tunnel capsule image include: Design a multispectral imaging system, which includes a visible light camera, a near-infrared camera, a thermal infrared camera, a multispectral light source and a confocal optical system; perform geometric calibration, spectral calibration and multi-camera synchronization on the multispectral imaging system; collect multispectral images of the tunnel capsule in a temperature-controlled environment, dynamically adjust the exposure time through an adaptive algorithm, determine the focal length using an autofocus technology, and store the collected images in real time; The collected multispectral images are preprocessed, including noise removal, image enhancement, geometric correction, spectral correction, image fusion and dimensionality reduction. Bilateral filtering and contrast-limited adaptive histogram equalization are used for visible light images, non-local mean filtering and Retinex algorithm are used for near-infrared images, and adaptive median filtering and adaptive local contrast enhancement algorithm are applied to thermal infrared images. Distortion correction is performed based on camera intrinsic parameters, and multi-band images are registered using feature point matching and perspective transformation. Radiation correction is performed using sensor response curves, and atmospheric correction is performed using atmospheric radiation transfer models. Wavelet transform is used for feature-level fusion, and decision-level fusion based on belief function theory is used. Dimensionality reduction is performed through principal component analysis and independent component analysis. Data quality control is performed on the preprocessing results, including image quality assessment, anomaly detection and data verification. Texture features, shape features and spectral features are extracted from the preprocessed images and statistically analyzed. Among them, texture features are obtained by calculating the gray-level co-occurrence matrix, shape features are extracted through morphological operations, and spectral features include the normalized difference vegetation index.

3. The method according to claim 1, characterized in that The steps of extracting and classifying features of the preprocessed image and identifying the dilation area using a deep learning model, wherein the deep learning model includes an improved multi-scale feature pyramid network and an attention mechanism module include: The improved multi-scale feature pyramid network is used for feature extraction, including a dense connection module, a spatial pyramid pooling module and a feature recalibration module; the dense connection module enhances feature propagation through multiple convolutions and feature connections; the spatial pyramid pooling module performs pooling operations at different scales and captures multi-scale context information through upsampling; the feature recalibration module includes a channel attention mechanism to adaptively weight feature channels; The attention mechanism module is integrated with the improved multi-scale feature pyramid network, including a channel attention submodule and a spatial attention submodule; the channel attention submodule calculates channel weights through a shared multi-layer perceptron using a combination of global average pooling and global maximum pooling; the spatial attention submodule generates a spatial attention map using feature maps of average pooling and maximum pooling; The preprocessed image is input into the deep learning model, and processed by the improved multi-scale feature pyramid network and the attention mechanism module in sequence to obtain a multi-scale feature representation; the multi-scale feature representation is adaptively weighted fused, and the adaptive fusion of features is achieved by learning the weight of each scale feature to obtain a fused feature; the fused feature is input into a classifier for dilation area recognition, and the classifier includes a global average pooling layer and a fully connected layer.

4. The method according to claim 3, characterized in that Adaptively weighted fusion is performed on the multi-scale feature representation, and adaptive fusion of features is achieved by learning the weight of each scale feature to obtain fused features; Inputting the fused features into a classifier for dilation area recognition, the classifier including a global average pooling layer and a fully connected layer comprises the following steps: The preprocessed image is input into the deep learning model, and the feature representation of the image is extracted at multiple scales through multiple downsampling and upsampling operations to obtain feature maps of K different scales; The K feature maps extracted are sent to the 1×1 convolution layer for dimensionality reduction to obtain the corresponding K weight maps, and then the K weight maps are normalized using the normalized exponential function to ensure that the sum of the weights at each spatial position is 1; Multiply K feature maps by their corresponding weight maps element by element, and add the weighted feature maps to obtain an adaptively fused multi-scale feature representation. The adaptively fused features are compressed into a feature vector of fixed length through a global average pooling layer; The pooled feature vector is passed through the fully connected layer and the dropout layer with L2 regularization in turn, and finally through the fully connected layer activated by the normalized exponential function to output the probability of each category and complete the classification of the expansion area.

5. The method according to claim 1, characterized in that According to the spatial position and range of the expansion area, combined with the preset hazard level standard, the expansion area is risk assessed and graded, and the detection results including the location, range and risk level of the expansion area are generated. The detection results are visualized on the three-dimensional model of the tunnel, and the maintenance and reinforcement plan based on the reinforcement learning algorithm is generated. The steps include: Obtaining spatial location and range information of tunnel expansion area; Calculating the risk index of the expansion area using a weighted summation method based on a multi-factor risk assessment model, wherein the multi-factor risk assessment model includes expansion area, expansion depth, location sensitivity, tunnel age and geological conditions; According to the calculated risk index, the expansion area is divided into different risk levels; Using 3D visualization technology, the inspection results, including the location, range and risk level of the expansion area, are presented on the 3D model of the tunnel; Construct a reinforcement learning environment, including defining the state space, action space, and reward function. The state space includes the risk index, location, range, and current maintenance budget of the expansion area. The action space includes a variety of maintenance measures. The reward function is designed based on the risk reduction degree and cost-effectiveness ratio after maintenance. Use a deep Q-learning algorithm to train a maintenance decision model. Based on the trained maintenance decision model, generate maintenance and reinforcement plans for each expansion area. The generated maintenance and reinforcement plan is displayed in a 3D visualization interface. The tunnel status information is updated in real time, and the multi-factor risk assessment model and maintenance decision model are continuously optimized based on new inspection data and maintenance effect feedback. The optimized multi-factor risk assessment model and maintenance decision model are applied to subsequent risk assessment and maintenance plan formulation to form a closed-loop optimization mechanism.

6. The method according to claim 5, characterized in that Construct a reinforcement learning environment, including defining a state space, an action space, and a reward function, wherein the state space includes the risk index, location, range, and current maintenance budget of the expansion area, the action space includes a variety of maintenance measures, and the reward function is designed based on the risk reduction degree and cost-effectiveness ratio after maintenance; The maintenance decision model is trained using a deep Q-learning algorithm. Based on the trained maintenance decision model, the steps of generating maintenance and reinforcement plans for each expansion area include: The state space includes expansion area characteristics, tunnel global characteristics and maintenance resource information, wherein the expansion area characteristics include risk index, location coordinates and area, the tunnel global characteristics include tunnel age, geological condition score and traffic load, and the maintenance resource information includes available budget, human resources and equipment resources; The action space is defined as a set of discrete maintenance measures, including grouting reinforcement, bolting, lining repair, and no-action options; Design a multi-objective reward function, including risk index reduction, safety improvement, maintenance cost, and long-term performance impact; Construct a deep Q network based on long short-term memory network and attention mechanism to approximate the Q function; The deep Q network includes a long short-term memory layer, a multi-head attention layer, and a fully connected layer for processing high-dimensional state space and capturing temporal dependencies; Implement a prioritized experience replay buffer, assign a priority to each experience sample based on the timing difference error, and use importance sampling for bias correction; Adopting the target network and soft update strategy, the target Q value is calculated by mixing the output of the online network and the target network, and the target network parameters are gradually updated using the soft update coefficients; Design an ε-greedy strategy with a decay mechanism, perform extensive exploration in the early stages of training, and execute a training loop, including interaction with the environment, experience storage, batch sampling, loss calculation, parameter update, and target network soft update steps; Using the trained deep Q network, the optimal maintenance plan is generated through the Monte Carlo tree search method according to the current tunnel status. The generated maintenance plan is applied to the tunnel management system, and feedback on the implementation effect is collected. The environmental model and reward function are updated according to the feedback data to achieve continuous optimization and adaptive adjustment of the algorithm.

7. A machine vision-based balloon expansion detection and positioning system, used to implement the method of any one of claims 1 to 6, characterized in that: include: The first unit is used to collect images of the tunnel capsule using multispectral imaging technology; Preprocessing the tunnel capsule image; Using a deep learning model to extract and classify features of the preprocessed image to identify the dilation area, the deep learning model includes an improved multi-scale feature pyramid network and an attention mechanism module; Perform edge detection and contour extraction on the expansion area to obtain the precise boundary of the expansion area; calculate the area, perimeter and shape characteristics of the expansion area based on the extracted contour information; The second unit is used to establish a three-dimensional model of the tunnel using an improved structured light three-dimensional reconstruction technology, wherein the improved structured light three-dimensional reconstruction technology includes using a phase shift method to improve the accuracy of depth information and applying an adaptive multi-frequency phase unwrapping algorithm; mapping the detected expansion area to the three-dimensional model of the tunnel to determine the spatial position and range of the expansion area; performing risk assessment and grading of the expansion area based on the spatial position and range of the expansion area in combination with a preset hazard level standard, generating a detection result including the position, range and risk level of the expansion area, visually presenting the detection result on the three-dimensional model of the tunnel, and generating a maintenance and reinforcement plan based on a reinforcement learning algorithm; The third unit is used to establish a historical database to record the detection results and the changing trends of the expansion area; based on the historical data and current detection results, the time series analysis method is used to predict the development trend and potential risks of the expansion area.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images