Subway tunnel crack monitoring method and equipment based on multi-data fusion

Abstract

The invention relates to a subway tunnel crack monitoring method and device based on multi-data fusion, and the method comprises the steps: fixing an image collection device and a meteorological sensor in front of a subway tunnel crack, and obtaining the image, temperature and humidity information of the crack in real time; a crack image is collected in advance and marked, and a crack segmentation data set is constructed; a crack real-time segmentation model is constructed, and depth separable convolution and multi-scale attention fusion are adopted; training the crack real-time segmentation model by using the constructed crack segmentation data set, and detecting an image acquired by the image acquisition equipment at the subway tunnel crack in real time based on the trained crack real-time segmentation model to obtain a binary image of the crack; obtaining the boundary of the crack, and calculating the width of the crack; a crack-meteorological model is constructed by adopting crack width information and meteorological data collected by a meteorological sensor at a subway tunnel crack, and the influence of temperature and humidity on the crack width is fitted to predict the change of the subway tunnel crack width.

Description

Technical Field

[0001] The present invention belongs to the field of subway tunnel disease monitoring, and particularly relates to a subway tunnel crack monitoring technical solution based on multi-data fusion, which can realize real-time detection and precise measurement of cracks under complex environments and is applicable to the health monitoring and maintenance of subway tunnels. Background Technique

[0002] Subway tunnels are important infrastructure for urban rail transit. During their long-term operation, due to the combined effects of factors such as train loads, groundwater seepage, temperature and humidity changes, and environmental vibrations, defects such as cracks are likely to appear on the surface of the tunnel structure. Cracks are key factors affecting the safety and durability of the tunnel structure. They not only weaken the bearing capacity of the tunnel structure but may also lead to serious consequences such as water leakage, corrosion, and even collapse. Therefore, crack detection and width quantification analysis are important technical means to ensure the safe operation and maintenance efficiency of the tunnel.

[0003] Currently, crack detection methods mainly include manual visual inspection, traditional image detection, and point cloud detection, etc. Manual visual inspection is the most common detection method in current projects. It is simple to operate, but has low efficiency and strong dependence on the experience of detection personnel, making it difficult to ensure the consistency and comprehensiveness of detection results. Especially in tunnel environments with insufficient light or high humidity, it is often difficult to accurately identify tiny cracks manually, and there are large subjective errors. Traditional image detection methods use cameras to collect images of the tunnel surface and extract and analyze crack features through traditional image processing techniques such as threshold segmentation, edge detection, and morphological processing. This method has a certain degree of automation, but its effect is easily affected by changes in lighting conditions and image noise interference, and it is difficult to guarantee the detection accuracy and stability in complex tunnel environments.

[0004] With the rapid development of computer vision and deep learning technologies, these technologies have gradually been introduced into the field of crack detection. By training convolutional neural network (CNN) models, deep learning methods can automatically extract multi-scale features of cracks, significantly improving the accuracy and efficiency of crack detection. For example, object detection algorithms such as YOLO and Faster R-CNN can quickly locate crack areas, while segmentation models such as U-Net and DeepLab can perform pixel-level segmentation analysis on crack areas. However, these methods still have deficiencies in practical applications. On the one hand, deep learning models usually require a large amount of labeled data for training, but due to the complexity of the subway tunnel crack data and the limitations of data collection difficulty, it is relatively difficult to construct a dataset. On the other hand, the recognition effect of existing models on tiny cracks or complex backgrounds is not ideal, and at the same time, they lack the ability to measure quantification parameters such as crack width. In addition, the real-time performance and resource consumption problems of deep learning models also pose challenges in actual engineering environments.

[0005] Affected by the environment and external forces, cracks may further deteriorate. Therefore, real-time monitoring of crack width is required after cracks are detected. Traditional single image data detection methods often ignore the influence of environmental conditions on crack width measurement. The present invention takes into account that factors such as temperature and humidity in the tunnel environment may directly or indirectly affect the actual width and measurement accuracy of cracks. For example, water stains under high humidity conditions may cause the crack edges to be blurred, and temperature changes may cause thermal expansion and contraction of the tunnel structure, thus interfering with the measurement of crack width. Therefore, the present invention proposes a multi-data fusion method that combines environmental parameters with crack feature data, providing a new technical means for accurately measuring crack width. Summary of the Invention

[0006] Aiming at the above deficiencies existing in the prior art, the purpose of the present invention is to realize real-time monitoring of subway tunnel cracks.

[0007] The technical solution proposed by the present invention includes a subway tunnel crack monitoring method based on multi-data fusion. A subway tunnel crack monitoring method based on multi-data fusion includes the following processes.

[0008] Fix an image acquisition device and a meteorological sensor in front of the subway tunnel crack to obtain real-time images, temperature, and humidity information of the crack.

[0009] Pre-collect crack images, annotate them, and construct a crack segmentation data set.

[0010] Construct a crack real-time segmentation model. The crack real-time segmentation model uses depthwise separable convolution and multi-scale attention fusion, takes the collected image as the input, and outputs a crack binary image. Use the constructed crack segmentation data set to train the crack real-time segmentation model, and detect the images collected in real time by the image acquisition device at the subway tunnel crack based on the trained crack real-time segmentation model to obtain a binary image of the crack.

[0011] According to the binary image of the crack obtained by the detection, obtain the boundary of the crack and calculate the width of the crack.

[0012] Adopt the calculated crack width information and the meteorological data collected by the meteorological sensor at the subway tunnel crack to construct a crack-meteorological model. The crack-meteorological model fits the influence of temperature and humidity on the crack width and is used to predict the change of the subway tunnel crack width.

[0013] Moreover, the real-time crack segmentation model adopts an encoder-decoder structure. The encoding part includes four encoding modules. Depthwise separable convolutions are used in the encoding modules to enhance the ability to extract crack features. The decoding part correspondingly sets four decoding modules. A triple attention module is set. The outputs of the last three encoding modules respectively enter the large-scale, medium-scale, and small-scale feature branches of the triple attention module for processing. The processed results are merged and then transmitted to the bottommost decoding module to support high-precision subway tunnel crack segmentation.

[0014] Moreover, when detecting the images collected in real time by the image acquisition device at the subway tunnel crack, a sliding window method is used for detection.

[0015] Moreover, when calculating the width of the crack, the central normal method is adopted to solve the maximum width, average width, and width at a specified position of the subway tunnel crack.

[0016] Moreover, the crack meteorological model is constructed by using polynomial fitting.

[0017] Moreover, the crack image acquisition device includes a camera, a bracket, and an LED fill light. The camera, LED fill light, and meteorological sensor are fixed on the bracket and then the whole is placed beside the crack. The data of the camera and meteorological sensor are transmitted to the background through network cables.

[0018] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the above-mentioned subway tunnel crack monitoring method based on multi-data fusion is implemented.

[0019] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned subway tunnel crack monitoring method based on multi-data fusion is implemented.

[0020] On the other hand, the present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the above-mentioned subway tunnel crack monitoring method based on multi-data fusion is implemented.

[0021] On the other hand, the present invention also provides a subway tunnel crack monitoring system based on multi-data fusion, including an image acquisition device and a meteorological sensor fixed in front of the subway tunnel crack, and a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the above-mentioned subway tunnel crack monitoring method based on multi-data fusion is implemented.

[0022] The above subway tunnel crack monitoring solution based on multi - data fusion provided by the present invention collects the images of cracks in real time, segments the cracks based on the images and completes the width measurement, realizes the long - term monitoring of the crack width, analyzes the correlation between the crack width and meteorology, and predicts the normal change range of the crack width. It can be considered that the purpose of remote real - time monitoring of tunnel cracks can be achieved. The deep - learning model is used to automatically detect cracks in image data and further accurately measure the crack width. In order to achieve real - time detection, the algorithm structure is optimized to improve the detection efficiency. At the same time, the deep - learning method is used to fit the relationship between the crack width and environmental variables (such as temperature, humidity), and the environmental parameters are incorporated into the analysis framework. This model can effectively identify and learn the influence of temperature and humidity changes on the crack width measurement, so as to predict the maximum width that the crack may reach under extreme environments, providing an efficient and reliable technical means for the health monitoring of subway tunnels.

[0023] The technical solution provided by the present invention can significantly improve the automation level of crack detection, reduce manual intervention, and at the same time improve the accuracy and reliability of crack detection through multi - data fusion. This method is applicable to the long - term monitoring and evaluation of cracks in structures such as subway tunnels and underground pipelines, and has broad application prospects.

[0024] The implementation of the solution of the present invention is simple and convenient, with strong practicability. It solves the problems of low practicability and inconvenience in actual application existing in the related technologies, can improve the user experience, and has important market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is the overall flowchart of the subway tunnel crack monitoring with multi - data fusion according to the embodiment of the present invention;

[0026] Figure 2 It is the structural diagram of the crack real - time segmentation model T_Cracknet according to the embodiment of the present invention;

[0027] Figure 3 It is the structural diagram of the ED Block according to the embodiment of the present invention.

[0028] Figure 4 It is the structural diagram of the DC Block according to the embodiment of the present invention.

[0029] Figure 5 It is the structural diagram of the Triple attention according to the embodiment of the present invention.

[0030] Figure 6 It is the schematic diagram of the application of the crack real - time monitoring according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The following specifically describes the technical solution of the present invention in conjunction with the drawings and embodiments.

[0032] An embodiment of the present invention proposes a subway tunnel crack monitoring method based on deep learning. By adopting a multi-data fusion technology, it combines a camera and a meteorological sensor to monitor the image information of cracks in the tunnel and temperature and humidity data in real time, so as to realize the automatic detection and analysis of tunnel cracks. This method first designs a crack segmentation network model to accurately segment the cracks on the tunnel wall, then uses the central normal method to measure the width of the cracks, and conducts correlation analysis with the meteorological data (such as temperature, humidity, etc.) collected by the sensors to further evaluate the dynamic changes of the cracks and their relationship with environmental conditions, thereby providing accurate data support for tunnel maintenance and safety detection.

[0033] In an embodiment of the present invention, the set image acquisition device includes a camera, a bracket, and an LED fill light. First, install the bracket on one side of the crack, and fix the camera, the LED fill light device, and the meteorological sensor. Use a rangefinder to measure the distance between the camera center and the crack and record it. Then, collect the tunnel crack images and annotate them to establish a crack segmentation data set. Next, construct a crack segmentation model and use the data set to train the model. Deploy the trained model to the server to process the images collected by the camera and segment the binary image of the crack. Calculate the crack width according to the binary image, and further establish a transformation relationship model between the crack changes and environmental factors to fit the change trend of the crack and predict its development.

[0034] In an embodiment of the present invention, an overall process of subway tunnel crack monitoring based on multi-data fusion is as Figure 1 shown, including the following steps:

[0035] S1. Install a crack image acquisition device and a meteorological sensor on one side of the subway tunnel crack.

[0036] In this step, install an image acquisition device and a meteorological sensor beside the subway tunnel crack. In the embodiment, it is implemented as follows:

[0037] (1) Place the image acquisition device beside the crack so that the crack to be monitored is located in the center area of the image, and use a rangefinder to measure the distance from the camera center to the crack surface. Most preferably, the camera can be installed directly above the crack and vertically directed towards the plane where the crack is located.

[0038] (2) Place a meteorological sensor beside the image acquisition device to obtain the temperature and humidity at the time of image acquisition at the same frequency.

[0039] In specific implementation, a camera, an LED fill light (used to illuminate the crack to obtain clearer crack image information), and a meteorological sensor can be fixed on a bracket, and then the bracket is fixed beside the crack; in addition, after the device is installed, the device needs to be calibrated, and the actual size represented by a single pixel is calculated according to the distance from the camera center to the crack surface. The data of the camera and the meteorological sensor can be transmitted to the background through a network cable or wirelessly.

[0040] S2. Pre-collect crack images and construct a crack segmentation dataset.

[0041] After the device is deployed, crack images are collected, and the cracks in the collected images are labeled to construct a crack segmentation dataset for subsequent training of the crack segmentation model. In specific implementation, it is better to be able to collect images of various cracks in different environments. To expand the sample data, data collection can be simulated in a tunnel environment. If conditions are limited, the crack objects collected may not be limited to the cracks in the tunnel.

[0042] Further, it is preferably recommended that when data is labeled, the crack is labeled as a binary image with white for the crack and black for the background, and data augmentation is used to expand the crack segmentation dataset.

[0043] The implementation in the embodiment is as follows:

[0044] (1) Timed shooting data: Control the camera to take pictures at regular intervals and collect crack image data at different time periods within a day.

[0045] (2) Multi-device collection: To improve the generalization ability of the model, in addition to using a fixed camera to capture crack images, devices such as cameras and mobile phones are also used to collect crack images.

[0046] (3) Image cropping and annotation: Crop the collected crack images to a size of 512×512, and use the Labelme annotation software to annotate the crack area to generate a binary map label of the crack.

[0047] (4) Data augmentation: Perform data augmentation on the annotated dataset, and use methods such as rotation, scaling, and brightness change to expand the samples to improve the robustness of the model.

[0048] S3. Construct a real-time crack segmentation model and train it using the crack segmentation dataset constructed in step S2; based on the trained real-time crack segmentation model, detect the images collected in real time by the imaging device at the crack of the subway tunnel to obtain a binary image of the crack.

[0049] This step constructs a real-time crack segmentation model T_CrackNet, which is specifically designed for the accurate segmentation of subway tunnel cracks. The real-time crack segmentation model takes the collected images as input and outputs binary crack images. That is, after training, the crack images can be segmented in real time through T_Cracknet.

[0050] T_CrackNet proposed by the present invention is a convolutional neural network (CNN) based on deep learning, which combines image segmentation technology and attention mechanism to address the challenges in tunnel crack detection, such as complex backgrounds, diverse crack morphologies, and crack segmentation under different lighting conditions. The model adopts an encoder-decoder structure, uses depthwise separable convolution and multi-scale attention fusion, better learns the crack features, suppresses the interference of the background, and achieves high-precision crack segmentation.

[0051] Specifically, in T_CrackNet of the present invention, a decoding module is constructed to improve the feature extraction ability of the model, and at the decoding layer, a CARAFE lightweight upsampling operator is added; a Triple attention module is also designed to capture more context information and details.

[0052] The encoding part of T_CrackNet in the embodiment contains four encoding modules, and depthwise separable convolution (DSConv) is used to improve the model's ability to extract crack features. The four decoding modules (DC Block) in the decoder restore the spatial resolution of the feature map through the Crarfe upsampling layer and convolution operations, enhance the image details, and improve the segmentation accuracy of the crack edges. The triple attention module (Triple Attention) combines branches of large-scale, medium-scale, and small-scale features, fully combines the semantic information of the context, effectively improves the model's ability to detect cracks of different scales, and enhances the robustness of crack segmentation. The outputs of the last three encoding modules respectively enter the large-scale, medium-scale, and small-scale feature branches of the triple attention module for processing and merging, and then are transmitted to the bottom decoding module. Finally, the fusion module (Fusion Block) fuses the results of each layer of decoding modules, enhances the multi-scale learning ability, and improves the accuracy and robustness of crack segmentation.

[0053] To improve the performance of the crack segmentation model, the present invention further proposes to design a new loss function, which combines the traditional cross-entropy loss and the structural loss specifically for crack segmentation. The model can efficiently extract and restore crack details and accurately segment the crack area.

[0054] The preferred implementation method in the embodiment is as follows:

[0055] (1) In the embodiment, the resolution of the original image collected by the camera is 1920×1080. During the detection process, to ensure the detection efficiency, the image input size is 512×512. Before detection, the image captured by the camera needs to be cropped into an image of 1920×1024 size with the center point as the center. Therefore, when installing the device, the preferred method is to make the crack as much as possible in the middle of the image.

[0056] (2) Build a crack segmentation model. The structure diagram of the detection model is as Figure 2 shown. T_CrackNet adopts an encoder-decoder structure. The encoder extracts the features of the input image through a series of convolutional layers, and the decoder restores the spatial resolution of the image through upsampling operations, so as to accurately segment the crack area.

[0057] The encoder contains four encoding modules (ED Block). The structure of each ED Block module is as Figure 3 shown, including a convolutional module (Conv) and a depthwise separable convolutional module (DSConv). The input signal enters the convolutional module Conv and the depthwise separable convolutional module DSConv in sequence. Another path is added and combined with the output of the depthwise separable convolutional module DSConv. Each Conv module consists of a convolution, a Relu function, and a batch normalization connected in sequence. Each DSConv module contains a distribution shift convolution, a Relu function, and a batch normalization connected in sequence.

[0058] The decoder contains four DC Block. The structure of each DC Block module is as Figure 4 shown, including a Crarfe upsampling layer and two Conv modules connected in sequence. First, the input feature map is passed into the Crarfe upsampling layer. This layer performs an upsampling operation on the spatial resolution of the feature map. After passing through the Crarfe upsampling layer, the spatial resolution of the feature map is increased, and the output feature map is used as a signal to be passed to the first convolutional module. In this module, the feature map output by the Crarfe upsampling layer is input into the convolutional layer for processing. The convolution operation extracts local features through filters and performs a non-linear transformation (such as ReLU activation). The output feature map continues to be passed to the next convolutional module for further feature extraction. This module also contains a convolutional layer and an activation function. After two convolution operations, the size and number of channels of the output feature map can be adjusted continuously, and finally the required output feature map is generated. That is, the input signal of each DC Block module starts from the Crarfe upsampling layer, and the spatial resolution is increased through the upsampling operation; after being processed by the first convolutional module and the second convolutional module, features are gradually extracted and the information is enhanced through the convolution operation.

[0059] In specific implementation, the Crarfe upsampling layer, convolutional module, depthwise separable convolutional module, etc. can be implemented using existing technologies, which will not be elaborated in the present invention.

[0060] The bottom layers of the encoder and decoder (ED Block4, DC Block4) are connected through the Triple attention module. The structure of the Triple attention is as Figure 5 shown, including three branches: 1) The input of the large-scale branch passes through a Conv module, a max pooling layer (Maxpool), a Conv module in sequence, and finally passes through an average pooling layer (Avgpool). 2) The medium-scale branch first passes through a Conv module and then inputs an average pooling layer and a max pooling layer respectively, and finally adds and combines (Add) the two. 3) The small-scale branch only has a Conv module. Finally, the results of the three branches are concatenated and combined (Concat).

[0061] In the basic structure of the crack segmentation model, the input of each encoding module is the output of the previous layer, that is, the encoder input passes through ED Block1, ED Block2, ED Block3, and ED Block4 in sequence. The inputs of the Triple attention module include the output results of ED Block2, ED Block3, and ED Block4, which are respectively used as the inputs of the large-scale branch, medium-scale branch, and small-scale branch. In the encoder, the input of each decoding module is the output of the previous layer, that is, the output of the Triple attention module passes through DC Block4, DC Block3, DC Block2, and DC Block1 in sequence.

[0062] Furthermore, in the embodiment, other layers except the bottom layers of the encoder and decoder are connected through skip connections to improve the robustness of the model, such as Figure 2As shown, the outputs of ED Block1, ED Block2, and ED Block3 are directly passed to the inputs of DC Block1, DC Block2, and DC Block3 respectively, and the feature maps of the encoder and decoder are concatenated in the channel dimension. Specifically, the concatenation method is to concatenate the output feature map of the corresponding layer of the encoder with the feature map after upsampling by Crarfe in the corresponding layer of the decoder, and the concatenated feature map will be passed to the first convolution module inside the corresponding layer of the decoder for further processing. To achieve multi-scale feature fusion for better crack segmentation. For example, the output feature map of ED Block1 is concatenated with the feature map obtained from the Crarfe upsampling layer inside DC Block1 in the decoder, and the concatenated feature map will be passed to the first convolution module inside DC Block1, and then through the second convolution module inside DC Block1 to obtain the output of DCBlock1. The skip connections between ED Block2 and DC Block2, and between ED Block3 and DC Block3 have the same implementation principle.

[0063] Furthermore, a fusion module (Fusion Block) is set at the end of the decoder (DC Block1) to fuse the results of each decoding module, that is, the outputs of DC Block4, DC Block3, and DC Block2 are also connected to the fusion module and concatenated with the output of DC Block1 in the fusion module to achieve fusion.

[0064] The innovative loss function (SP_Loss) that combines structural information and pixel-level loss combines two main parts: Pixel-wise Loss: Measures the segmentation accuracy of each pixel point, using the common cross-entropy loss.

[0065] Structure Loss: Measures the shape, boundary, and detail retention of the crack region, encouraging the model to better retain the geometric structure features of the crack during segmentation.

[0066] L SP_Loss =L CE +λ·L Structure

[0067] Where:

[0068] L CE is the pixel-level loss, used to measure the accuracy at the pixel level.

[0069]

[0070] Among them, y i is the ground truth label, pi is the predicted probability, N is the number of samples, and i is the sample number.

[0071] L Structure is the structural loss, which measures the geometric structure difference between the model segmentation result and the true crack region. The present invention preferably designs the structural loss by comparing the boundaries and shapes of the cracks.

[0072]

[0073] Among them, is the gradient operator, I pred and I gt are the predicted image and the true label image respectively, x j is the position in the image, M is the number of all positions in the image, and j is the position serial number. This term calculates the difference between the predicted image and the true label image at the boundary, promoting the accurate retention of the crack structure by the model.

[0074] λ is a balancing factor used to control the weight balance between the pixel-level loss and the structural loss.

[0075] (3) Train the model. After constructing the crack segmentation model, use the crack segmentation dataset to train the model.

[0076] (4) Deploy and detect the segmentation model. After the model is trained, deploy it on the background server, process the images transmitted back by the image acquisition device, segment and identify the cracks in the images, and obtain the binary image of the cracks. During detection, since the image is larger than 512×512, the sliding window method is used to detect the image, the window size is 512×512, and the sliding step is 512.

[0077] S4. Calculate the width of the crack.

[0078] In the previous step S3, the crack segmentation model has output the binary image of the crack. The crack region in this image is white (255), and the background is black (0). This binary image will be used as the basis for subsequent processing, and image processing techniques are used to extract the boundary of the crack. The present invention further proposes that before calculating the crack width, the binary image obtained by segmentation is preprocessed, and dilation and erosion algorithms are used to eliminate holes and noise. Moreover, during the crack width measurement process, the central normal method is used to calculate the average width, the maximum width of the overall crack in the image, and the crack width at a specified position.

[0079] In the embodiment, the specific implementation of this step is as follows:

[0080] Extract the boundary of the crack: Use the contour extraction method (such as the findContours function in OpenCV) to extract the boundary contour of the crack region.

[0081] Calculating the center line of the crack: To calculate the width of the crack using the central normal method, it is necessary to first calculate the center line of the crack. The center line is the central axis between the two boundaries of the crack. During specific implementation, after binarization, the Skeletonize function can be used to perform a skeletonization operation on the crack area to achieve extraction. The purpose of skeletonization is to reduce the crack area to a thin line, that is, the center line of the crack, ensuring that the topological structure of the crack does not change and facilitating subsequent crack width calculation.

[0082] Width calculation: The crack width is the sum of the normal distances from the center line to the two side boundaries. For each point on the center line, by extending along the normal direction to both sides, the two boundaries of the crack are found. Calculate the perpendicular distance from the crack center line point to the boundary to obtain the crack width at that point. Finally, the width of the crack is the average of all normal measurement results. During specific implementation, specific statistics such as the maximum value or minimum value can also be selected as the final result. Therefore, for each center line point C(x,y), the crack width can be calculated by the following formula:

[0083] Normal direction: For each point C(x,y) on the crack center line, calculate the normal direction starting from this point. Among them, (x,y) are the coordinates of the center line point, and the normal direction θ is the perpendicular direction intersecting with the crack boundary.

[0084] θ = atan2(dy / dx)

[0085] Among them, (dx,dy) is the direction vector of the crack boundary, and the atan2 function calculates the normal direction perpendicular to this direction.

[0086] Normal distance: Calculate the shortest distance d(x,y) from the center line point C(x,y) along the normal direction to the boundary, which can be obtained by calculating the pixel values in the image or the distance transform image.

[0087] Width of each point: According to the above formula, for each center line point C(x,y) in the image, calculate the crack width W(x,y), that is:

[0088] W(x,y) = d1(x,y) + d2(x,y)

[0089] Among them, d1(x,y) and d2(x,y) respectively represent the normal distances from the center line point to the left and right boundaries.

[0090] Overall crack width: If it is necessary to calculate the width statistics of the entire crack, the average value or maximum value of all points can be taken as the overall width:

[0091]

[0092] Among them, K is the total number of points on the center line of the crack.

[0093] Alternatively, if the maximum width of the crack needs to be considered:

[0094] W max = max(W(x 1 , y 1 ), W(x 2 , y 2 ), …, W(x K , y K ))

[0095] All the widths obtained above are pixel values, and the true width value can be calculated according to the actual size represented by a single pixel in the image obtained in Step 1.

[0096] S5. Construct a crack - meteorological model to fit the influence of temperature and humidity on the crack width through this model, so as to obtain the change interval of the crack width.

[0097] In the embodiment, the specific implementation method of this step is as follows:

[0098] By installing meteorological sensors in monitoring environments such as subway tunnels, environmental data related to cracks are collected in real time, mainly including temperature and humidity. The relationship between these meteorological data and the change of crack width is very close. Use the crack width calculated in Step S4 and collect the corresponding time - series data including temperature and humidity. The data can be collected hourly or minutely to form a multi - dimensional time - series data set, and a crack - meteorological model is established. Polynomial regression is used to construct the crack - meteorological model. The goal of the model is to obtain a mathematical model that can predict the crack width by fitting the relationship between meteorological data such as temperature and humidity and the crack width.

[0099] Polynomial regression: If it is considered that the influence of temperature and humidity on the crack width is non - linear, a polynomial regression model can be used:

[0100] W = β 0 + β 1 T + β 2 H + β 3 T 2 + β 4 H 2 + TH

[0101] Among them, W is the crack width, T is the temperature, H is the humidity, and β 0 is the intercept term of the regression model. β 1 is the linear regression coefficient of temperature (T). β 2 is the linear regression coefficient of humidity (H). β 3 is the square term of temperature (T 2)'s regression coefficient is used to capture the non - linear effect of temperature on W. β 4 is the regression coefficient of the square term of humidity (H 2 ). By the least - squares method or other optimization methods, the parameters in the model are fitted to predict the crack width according to the changes in temperature and humidity.

[0102] In specific implementation, the collected training data (the corresponding data of temperature, humidity and crack width) are used to train the crack - meteorological model. By fitting the relationship between temperature, humidity and crack width in the data, a set of optimal regression coefficients are obtained. Algorithms such as the least - squares method and gradient descent can be used to optimize the model to obtain a set of optimal parameters. By iteratively adjusting the parameters of the model, the error between the predicted crack width and the actually calculated crack width is minimized. Through the constructed and trained crack - meteorological model, by inputting the real - time collected temperature and humidity data into the crack - meteorological model, the change of the crack width can be predicted. At the same time, by using the model to calculate the crack width under extreme meteorological conditions, the change range of the crack width can be obtained, which is also a prediction result. The prediction by the model provides accurate data support for later maintenance.

[0103] In specific implementation, the cracks to be monitored can be pre - placed in the central area of the camera shooting. The LED fill light is located below the camera, and the brightness of the LED fill light is adjustable and can be set according to the on - site environment during the deployment process to ensure that relatively clear images can be obtained. When calculating the crack width, the maximum width and average width of the crack are processed, and the position to be calculated can also be specified. The background for real - time monitoring of cracks used in the experiment is as Figure 6 shown. This interface real - time displays the current temperature and humidity, as well as the crack width in the current image, and also provides information such as device name, status, power, image resolution, data update time, installation location, etc.

[0104] In specific implementation, the method proposed by the technical solution of the present invention can be automatically run by those skilled in the art using computer software technology. The system device for implementing the method, such as a computer - readable storage medium storing the corresponding computer program of the technical solution of the present invention and a computer device including running the corresponding computer program, should also be within the protection scope of the present invention.

[0105] Next, the subway tunnel crack monitoring device based on multi - data fusion provided by the present invention will be described. The subway tunnel crack monitoring device based on multi - data fusion described below can be mutually corresponding and referred to the subway tunnel crack monitoring method based on multi - data fusion described above.

[0106] The electronic device provided by the present invention may include: a processor, a communications interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus. The processor can call the logical instructions in the memory to execute the subway tunnel crack monitoring method based on multi-data fusion provided by each of the above methods.

[0107] In addition, when the logical instructions in the above-mentioned memory are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0108] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the subway tunnel crack monitoring method based on multi-data fusion provided by each of the above methods.

[0109] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the subway tunnel crack monitoring method based on multi-data fusion provided by each of the above methods.

[0110] On yet another aspect, the present invention also provides a subway tunnel crack monitoring system based on multi-data fusion, including an image acquisition device and a meteorological sensor fixed in front of the subway tunnel crack, and a memory, a processor, and a computer program stored on the memory and executable on the processor are provided. When the processor executes the program, the subway tunnel crack monitoring method based on multi-data fusion as described above is implemented.

[0111] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0112] To verify the superior performance of the model proposed by the present invention in the task of real-time crack monitoring, Unet, DeepCrack, and Deeplabv3+ models were also trained and tested on the tunnel crack segmentation dataset, and the test results of these models were compared with the model proposed by the present invention. The three index parameters of accuracy, recall rate, and mIoU of the crack segmentation model proposed by the present invention are better than those of other detection methods.

[0113] Table 1. Comparison of detection results of different models.

[0114]

[0115] To verify the accuracy of the crack width calculated by the proposed method, the true width of the crack was measured and compared with the result calculated by this method. The comparison results are shown in Table 2. It can be seen that the relative error of each width is basically about 13%. Since most of the crack widths are about 1 mm, the maximum error of the calculated crack width is only a few tenths of a millimeter, which has good accuracy.

[0116] Relative error = [(measured value - true value) / true value] × 100%

[0117] Table 2. Positioning error information

[0118]

[0119] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solutions, in essence, or the parts that contribute to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0121] The above embodiments are only illustrative examples of the technical solutions of the present invention. The mobile edge computing task scheduling and offloading method, device, and storage medium based on multi-agent collaborative deep reinforcement learning involved in the present invention are not limited to the content described in the above embodiments, but are subject to the scope defined by the claims. Any modifications, supplements, or equivalent replacements made by those skilled in the art to which the present invention pertains on the basis of this embodiment are within the scope protected by the claims of the present invention.

Claims

1. A subway tunnel crack monitoring method based on multi-data fusion, characterized by: Including the following process, Fix image acquisition equipment and meteorological sensors in front of cracks in subway tunnels to obtain images, temperature and humidity information of cracks in real time; Collect crack images in advance, annotate them, and build a crack segmentation dataset; A real-time crack segmentation model is constructed, wherein the real-time crack segmentation model adopts deep separable convolution and multi-scale attention fusion, takes the collected image as input, and takes the crack binary image as output; the real-time crack segmentation model is trained using the constructed crack segmentation dataset, and the images collected in real time by the image acquisition device at the crack of the subway tunnel are detected based on the trained real-time crack segmentation model to obtain a binary image of the crack; According to the binary image of the crack detected, the boundary of the crack is obtained and the width of the crack is calculated; A crack-meteorological model is constructed using the calculated crack width information and the meteorological data collected by meteorological sensors at the crack locations in subway tunnels. The crack-meteorological model fits the effects of temperature and humidity on crack width and is used to predict changes in crack width in subway tunnels.

2. The subway tunnel crack monitoring method based on multi-data fusion according to claim 1 is characterized by: The real-time crack segmentation model adopts an encoder-decoder structure, the encoding part includes four layers of encoding modules, the encoding module uses deep separable convolution to improve the feature extraction capability of cracks, and the decoding part correspondingly sets four layers of decoding modules; A triple attention module is set up, and the outputs of the last three encoding modules enter the large-scale, medium-scale and small-scale feature branches of the triple attention module for processing respectively. The processing results are merged and transmitted to the bottom-level decoding module to support high-precision subway tunnel crack segmentation.

3. The subway tunnel crack monitoring method based on multi-data fusion according to claim 1 is characterized by: When detecting the images collected in real time by the image acquisition equipment at the cracks in the subway tunnel, a sliding window method is used for detection.

4. The subway tunnel crack monitoring method based on multi-data fusion according to claim 1 is characterized in that: When calculating the width of the crack, the center normal method is used to solve the maximum width, average width and width of the crack at the specified position in the subway tunnel.

5. The subway tunnel crack monitoring method based on multi-data fusion according to claim 1 is characterized by: The construction of the crack meteorological model is achieved by using polynomial fitting method.

6. The subway tunnel crack monitoring method based on multi-data fusion according to claim 1, 2, 3, 4 or 5, characterized in that: The crack image acquisition device includes a camera, a bracket, and an LED fill light. The camera, the LED fill light, and a meteorological sensor are fixed on the bracket and then placed next to the crack as a whole. The data of the camera and the meteorological sensor are transmitted to the background through a network cable.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the subway tunnel crack monitoring method based on multi-data fusion as described in any one of claims 1 to 6 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for monitoring cracks in a subway tunnel based on multi-data fusion as claimed in any one of claims 1 to 6 is implemented.

9. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the method for monitoring cracks in a subway tunnel based on multi-data fusion as claimed in any one of claims 1 to 6 is implemented.

10. A subway tunnel crack monitoring system based on multi-data fusion, characterized by: The invention comprises an image acquisition device and a meteorological sensor fixed in front of a crack in a subway tunnel, and is provided with a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, a subway tunnel crack monitoring method based on multi-data fusion as claimed in any one of claims 1 to 6 is implemented.

Images