An AI-based subway automatic monitoring and early warning system

Abstract

The present invention relates to the technical field of subway monitoring, and specifically relates to a subway automatic monitoring and early warning system based on artificial intelligence, including a temperature and humidity meter, an ultrasonic radar, an infrared thermal imager, and a central processing module. The central processing module corrects the tunnel ultrasonic point cloud according to the temperature and relative humidity in the tunnel, registers the tunnel infrared image with the ultrasonic point cloud, and generates a temperature-position point cloud. The difference between the temperature-position point cloud and the laser point cloud in the prior art is that each point in the temperature-position point cloud has a corresponding temperature. The temperature-position point cloud characterizes not only the tunnel structure but also the temperature distribution in the tunnel. This technical feature of the temperature-position point cloud helps the subway monitoring neural network model more easily diagnose tunnel diseases caused by temperature changes. The present invention realizes multi-modal, moisture-resistant, and highly reliable subway automatic monitoring and early warning by combining multi-modal fusion technology, ultrasonic point cloud scanning technology, and infrared imaging technology.

Description

Technical Field

[0001] The present invention relates to the technical field of subway monitoring, and particularly to an artificial intelligence-based subway automatic monitoring and early warning system. Background Art

[0002] A subway automatic monitoring and early warning system is a system for monitoring the status of subway tunnels. It collects, analyzes, and processes data through automated means to promptly detect potential problems or abnormal conditions. The existing subway automatic monitoring mainly includes two technical routes: multi-source sensors and laser point cloud scanning. The multi-source sensors monitor the status of subway tunnels by comprehensively analyzing the readings of static level gauges, crack meters, inclinometers, and vibration meters. The problem with this solution is that there are too many potential failure points, resulting in high maintenance costs and low reliability. Moreover, the readings of the sensors cannot fully describe the structural status of the subway tunnel. Laser point cloud scanning can fully describe the structural status of the subway tunnel, but the problem is that laser equipment is difficult to cope with extreme humidity changes. The high humidity state during subway tunnel leakage will cause the optical components of the laser equipment to condense. Therefore, laser point cloud scanning is still not suitable for long-term subway tunnel monitoring tasks, and laser point cloud scanning cannot collect the temperature distribution of the subway tunnel, and thus cannot determine the causes of tunnel diseases. Summary of the Invention

[0003] The present invention overcomes the deficiencies of the prior art and provides an artificial intelligence-based subway automatic monitoring and early warning system, including a temperature and humidity meter, an ultrasonic radar, a thermal imager, and a central processing module. The ultrasonic radar has a much higher tolerance for extreme humidity changes than a laser point cloud scanner. The central processing module corrects the tunnel ultrasonic point cloud based on the temperature and relative humidity inside the tunnel, registers the tunnel infrared image with the tunnel ultrasonic point cloud, and generates a temperature-position point cloud. The difference between the temperature-position point cloud and the laser point cloud in the prior art is that each point in the temperature-position point cloud has a corresponding temperature. The temperature-position point cloud characterizes both the tunnel structure and the temperature distribution inside the tunnel. This technical feature of the temperature-position point cloud helps the subway monitoring neural network model more easily diagnose tunnel diseases caused by temperature changes such as freeze cracks and expansion cracks. The present invention realizes multi-modal, moisture-resistant, and highly reliable subway automatic monitoring and early warning by combining multi-modal fusion technology, tunnel ultrasonic point cloud scanning technology, and infrared imaging technology.

[0004] An artificial intelligence-based subway automatic monitoring and early warning system includes: a temperature and humidity meter, an ultrasonic radar, a thermal imager, and a central processing module;

[0005] The temperature and humidity meter, the thermal imager, and the ultrasonic radar are respectively electrically connected to the central processing module;

[0006] The ultrasonic radar collects the tunnel ultrasonic point cloud and uploads it to the central processing module;

[0007] The thermometer and hygrometer collect the temperature and relative humidity inside the tunnel and upload them to the central processing module;

[0008] The thermal imager collects the infrared image of the tunnel and uploads it to the central processing module;

[0009] The central processing module corrects the tunnel ultrasonic point cloud according to the temperature and relative humidity inside the tunnel, then registers the tunnel infrared image and the tunnel ultrasonic point cloud, and assigns a value to each point in the tunnel ultrasonic point cloud according to the temperature in the tunnel infrared image to generate a temperature-position point cloud, and constructs a three-dimensional model of the tunnel temperature-position point cloud according to the temperature-position point cloud;

[0010] The central processing module constructs a subway monitoring neural network model and uses the subway monitoring neural network model for monitoring.

[0011] Further, the process of the central processing module correcting the tunnel ultrasonic point cloud according to the temperature and relative humidity inside the tunnel includes the following steps:

[0012] Step P1: Extract the horizontal angle, vertical angle and pre-correction distance corresponding to each point from the tunnel ultrasonic point cloud, and divide the pre-correction distance corresponding to each point by 343 to obtain the round-trip time of sound waves corresponding to each point;

[0013] Step P2: Apply the ultrasonic distance correction formula to each point in the tunnel ultrasonic point cloud to determine the actual distance between the point and the ultrasonic radar. The ultrasonic distance correction formula is as follows:

[0014] ;

[0015] ;

[0016] Wherein, represents the actual ultrasonic wave velocity, the unit of the actual ultrasonic wave velocity is meters per second, C represents the temperature inside the tunnel, and the unit of the temperature inside the tunnel is degrees Celsius, represents the relative humidity inside the tunnel, represents the round-trip time of sound waves, and the unit is seconds, represents the actual distance between the point and the ultrasonic radar;

[0017] Step P3: Regenerate the tunnel ultrasonic point cloud according to the horizontal angle, vertical angle and the actual distance between the point and the ultrasonic radar corresponding to each point.

[0018] Further, the process of the central processing module constructing the subway monitoring neural network model specifically includes the following steps:

[0019] Step S1: Construct a tunnel disease dataset;

[0020] Step S2: construct a point cloud feature extraction branch;

[0021] Step S3: construct a crack feature analysis branch, and connect the point cloud feature extraction branch to the crack feature analysis branch;

[0022] Step S4: construct a structural feature analysis branch, and connect the point cloud feature extraction branch to the structural feature analysis branch;

[0023] Step S5: construct a disease diagnosis and early warning branch, and connect the crack feature analysis branch and the structure feature analysis branch to the disease diagnosis and early warning branch;

[0024] Step S6: Construct a subway monitoring neural network model and use the tunnel disease dataset to train the subway monitoring neural network model.

[0025] Furthermore, the step S1 specifically includes the following steps:

[0026] Step S11: collecting the tunnel temperature position point cloud 3D models of other subway tunnels when they are completed as the reference point cloud 3D models, collecting the tunnel temperature position point cloud 3D models of other subway tunnels when tunnel diseases occur as the disease point cloud 3D models, and collecting the relative humidity in the tunnel while collecting the disease point cloud 3D models;

[0027] Step S12: marking the length of the widest crack, the width of the widest crack and the depth of the widest crack in each of the three-dimensional models of the defect point cloud;

[0028] Step S13: marking the segment convergence deformation level, interval tunnel health and tunnel disease cause of each defect point cloud 3D model, and encoding the tunnel disease cause in alphabetical order to generate a tunnel disease cause code;

[0029] Step S14: The three-dimensional model of each defect point cloud and its corresponding three-dimensional model of the reference point cloud, the relative humidity in the tunnel, the length of the widest crack, the width of the widest crack, the depth of the widest crack, the convergence deformation level of the segment, the health of the interval tunnel and the tunnel disease cause code are summarized into a tunnel disease data set.

[0030] Furthermore, the step S2 specifically includes the following steps:

[0031] Step S21: construct a custom displacement extraction layer;

[0032] Step S22: constructing a first custom three-dimensional extreme value pooling layer and a second custom three-dimensional extreme value pooling layer, wherein the first custom three-dimensional extreme value pooling layer and the second custom three-dimensional extreme value pooling layer are constructed by calling the same custom three-dimensional extreme value pooling layer instance;

[0033] Step S23: Construct a first 3D convolutional layer, a second 3D convolutional layer, and a first splicing and fusion layer, where the first 3D convolutional layer and the second 3D convolutional layer share weights;

[0034] Step S24: Connect the custom displacement extraction layer to the first custom 3D extreme pooling layer, connect the first custom 3D extreme pooling layer to the first 3D convolutional layer, connect the second custom 3D extreme pooling layer to the second 3D convolutional layer, and connect the first 3D convolutional layer and the second 3D convolutional layer to the first splicing and fusion layer.

[0035] Further, the hyperparameters of the custom displacement extraction layer include a displacement threshold. The custom displacement extraction layer receives two point cloud 3D models as inputs. The custom displacement extraction layer traverses each point in the first point cloud 3D model of the two point cloud 3D models, and determines whether at least one point can be found in the second point cloud 3D model within a region with a radius of the displacement threshold centered on the traversed point in the first point cloud 3D model. If not, this traversed point is marked. After the traversal of the first point cloud 3D model is completed, a new point cloud 3D model is generated based on all the marked points as the output.

[0036] Further, the custom 3D extreme pooling layer instance has two 3D pooling windows. The first 3D pooling window in the two 3D pooling windows takes the minimum value within the window to generate a pooling result, and the second 3D pooling window in the two 3D pooling windows takes the maximum value within the window to generate a pooling result. Finally, the pooling result of the first 3D pooling window and the pooling result of the second 3D pooling window are superimposed to generate a dual-channel 3D feature map and then output.

[0037] Further, step S3 specifically includes the following steps:

[0038] Step S31: Construct a first separable 3D convolutional layer, a first flattening layer, a second flattening layer, a first fully connected layer, a second fully connected layer, and a 3D Xception model. The first separable 3D convolutional layer extracts the crack depth features in its input, and the 3D Xception model extracts the crack width features and crack length features in its input;

[0039] Step S32: Connect the first splicing and fusion layer to the first separable 3D convolutional layer and the 3D Xception model, connect the first separable 3D convolutional layer, the first flattening layer, and the first fully connected layer in sequence, and connect the 3D Xception model, the second flattening layer, and the second fully connected layer in sequence.

[0040] Further, step S4 specifically includes the following steps:

[0041] Step S41: Construct a three-dimensional local connection layer, a third flattening layer, and a third fully-connected layer. The three-dimensional local connection layer applies independent convolutional kernels to different parts of its input to extract features related to specific positions;

[0042] Step S42: Connect the first splicing and fusion layer to the three-dimensional local connection layer, and connect the three-dimensional local connection layer, the third flattening layer, and the third fully-connected layer in sequence.

[0043] Further, step S5 specifically includes the following steps:

[0044] Step S51: Construct a second splicing and fusion layer, a third splicing and fusion layer, a second separable three-dimensional convolutional layer, a fourth flattening layer, a fourth fully-connected layer, a fifth fully-connected layer, and a fourth splicing and fusion layer;

[0045] Step S52: Connect the first separable three-dimensional convolutional layer, the three-dimensional Xception model, and the three-dimensional local connection layer to the second splicing and fusion layer, connect the first fully-connected layer, the second fully-connected layer, and the third fully-connected layer to the third splicing and fusion layer, connect the second splicing and fusion layer, the second separable three-dimensional convolutional layer, the fourth flattening layer, and the third splicing and fusion layer in sequence, connect the third splicing and fusion layer to the fourth fully-connected layer, connect the fourth fully-connected layer and the third splicing and fusion layer to the fourth splicing and fusion layer, and connect the fourth splicing and fusion layer to the fifth fully-connected layer.

[0046] Further, step S6 specifically includes the following steps:

[0047] Step S61: Designate the custom displacement extraction layer, the second custom three-dimensional extreme pooling layer, and the fourth splicing and fusion layer as the input layer of the subway monitoring neural network model, and designate the first fully-connected layer, the second fully-connected layer, the third fully-connected layer, the fourth fully-connected layer, and the fifth fully-connected layer as the output layer of the subway monitoring neural network model;

[0048] Step S62: Divide the tunnel disease dataset into a training set and a test set at a ratio of 8:2. Designate the disease point cloud three-dimensional model and the reference point cloud three-dimensional model in the tunnel disease dataset as the inputs of the custom displacement extraction layer, designate the reference point cloud three-dimensional model in the tunnel disease dataset as the input of the second custom three-dimensional extreme value pooling layer, designate the relative humidity inside the tunnel in the tunnel disease dataset as the input of the fourth splicing and fusion layer, designate the depth of the widest crack in the tunnel disease dataset as the target output of the first fully connected layer, designate the length and width of the widest crack in the tunnel disease dataset as the target outputs of the second fully connected layer, designate the segment convergence deformation level in the tunnel disease dataset as the target output of the third fully connected layer, designate the tunnel disease cause code as the target output of the fifth fully connected layer, designate the interval tunnel health degree as the target output of the fourth fully connected layer, train the subway monitoring neural network model, and use Bayesian optimization to adjust the value of the displacement threshold with the goal of minimizing the loss value of the depth of the widest crack.

[0049] Further, the process of using the subway monitoring neural network model for monitoring specifically includes the following steps:

[0050] Step M1: Collect the tunnel temperature position point cloud three-dimensional model at the completion of the target subway tunnel as the target reference point cloud three-dimensional model, collect the tunnel temperature position point cloud three-dimensional model of the current target subway tunnel as the target current point cloud three-dimensional model, and simultaneously collect the relative humidity inside the tunnel of the current target subway tunnel.

[0051] Step M2: Input the target current point cloud three-dimensional model and the target reference point cloud three-dimensional model into the custom displacement extraction layer, input the target reference point cloud three-dimensional model into the second custom three-dimensional extreme value pooling layer, input the relative humidity inside the tunnel of the current target subway tunnel into the fourth splicing and fusion layer, and the subway monitoring neural network model outputs the interval tunnel health degree, tunnel disease cause code, length of the widest crack, width of the widest crack, depth of the widest crack, and segment convergence deformation level of the target subway tunnel.

[0052] Step M3: Adjust the value of the displacement threshold for the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack: If the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack exceeds 5%, then use Bayesian optimization to re-determine the value of the displacement threshold with the goal of minimizing the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack, and then return to Step M2; otherwise, proceed to Step M4.

[0053] Step M4: If the interval tunnel health degree of the target subway tunnel reaches 4 or above, issue a tunnel disease warning; otherwise, do not issue a tunnel disease warning.

[0054] The beneficial effects of the present invention are as follows:

[0055] (1) The present invention provides a subway automatic monitoring and early warning system based on artificial intelligence, including a temperature and humidity meter, an ultrasonic radar, a thermal imager, and a central processing module. The ultrasonic radar has a much higher tolerance to extreme humidity changes than a laser point cloud scanner. The central processing module corrects the tunnel ultrasonic point cloud according to the temperature and relative humidity in the tunnel, and registers the tunnel infrared image with the tunnel ultrasonic point cloud to generate a temperature position point cloud. The difference between the temperature position point cloud and the laser point cloud in the prior art is that each point in the temperature position point cloud has a corresponding temperature. The temperature position point cloud characterizes the temperature distribution in the tunnel while characterizing the tunnel structure. This technical feature of the temperature position point cloud helps the subway monitoring neural network model more easily diagnose tunnel diseases caused by temperature changes. The present invention realizes multi-modal, moisture-resistant, and highly reliable subway automatic monitoring and early warning by combining multi-modal fusion technology, tunnel ultrasonic point cloud scanning technology, and infrared imaging technology.

[0056] (2) The default value of the sound speed of the ultrasonic radar is generally taken as the sound speed under the conditions of 1 and 20 °C, that is, 343 m / s. Although this is sufficient for most application scenarios, the present invention enables it to maintain high accuracy under extreme temperature and humidity changes through an ultrasonic distance correction formula, which cannot be achieved by existing laser point cloud scanning solutions.

[0057] (3) The overall design concept of the subway monitoring neural network model proposed by the present invention is to adopt the divide-and-conquer method. First, the features in the 3D point cloud model are extracted through the point cloud feature extraction branch. Then, through the hard sharing mechanism, the crack feature analysis branch and the structural feature analysis branch respectively analyze the local and overall features and output the relevant feature maps and indicators. Finally, the disease diagnosis and early warning branch obtains the health degree of the interval tunnel and the tunnel disease cause code. The feature of the point cloud feature extraction branch is that it extracts the crack and deformation features of the tunnel through the custom displacement extraction layer and extracts the extreme temperature distribution while retaining the structural features through the custom 3D extreme value pooling layer. This can help the subway monitoring and early warning system more easily diagnose the tunnel diseases caused by temperature changes. The weight sharing mechanism between the first 3D convolutional layer and the second 3D convolutional layer can maintain the consistency of features while further extracting features, facilitating the further comparison and analysis of the crack feature analysis branch. The first separable 3D convolutional layer in the crack feature analysis branch first convolves different input channels separately and then mixes all the input channels together for point-by-point convolution, which can effectively extract the crack depth features in its input. The 3D Xception model can effectively extract the crack width features and crack length features through multi-scale feature extraction. The structural feature analysis branch adopts a 3D local connection layer, which extracts the features related to specific positions by applying independent convolutional kernels at each position. This is particularly suitable for extracting the features related to the tunnel structure deformation because different positions of the tunnel structure obviously have different importance for the structural stability. Due to the combination of information such as temperature distribution, relative humidity in the tunnel, tunnel structure, and cracks in the tunnel, the disease diagnosis and early warning branch can obtain the health degree of the interval tunnel and the tunnel disease cause code, which cannot be achieved by simple laser scanning in the prior art. The subway monitoring neural network model organically combines the multi-task learning mechanism and the multi-modal learning mechanism, adopts the ideas of the divide-and-conquer method and syllogism, first extracts the point cloud features, then analyzes the local and overall features respectively, and finally summarizes the local and overall features for realizing the early warning and diagnosis of tunnel diseases. The difference from the technical effects of the existing subway monitoring and early warning systems is that the present invention can not only directly obtain the length, width, and depth of the segment convergence deformation and grade cracks, but also obtain the health degree of the interval tunnel and the tunnel disease cause. The relative humidity in the tunnel also plays a key role in the diagnosis of the tunnel disease cause. Without the relative humidity in the tunnel, it is impossible to diagnose the tunnel diseases caused by water leakage. The diagnosis of the tunnel disease cause is impossible for the existing subway monitoring and early warning systems that do not refer to the temperature distribution and relative humidity in the tunnel.

[0058] (4) The custom displacement extraction layer proposed by the present invention traverses each point of the first 3D point cloud model in the two 3D point cloud models, and determines whether at least one point can be found in the second 3D point cloud model within a region with a displacement threshold as the radius centered on the traversed point in the first 3D point cloud model. If no point can be found, the traversed point is marked. Here, the first 3D point cloud model represents the current structure of the target subway tunnel, and the second 3D point cloud model represents the structure of the target subway tunnel when it was just completed. The principle is that the points within the region with a displacement threshold as the radius centered on the traversed point are regarded as the same point as the traversed point. If no corresponding point can be found within this radius range, the traversed point is marked as having undergone displacement. The 3D model composed of all the marked points can effectively represent the cracks and deformations of the tunnel. The advantage of the custom displacement extraction layer compared with the prior art is that it has high interpretability, and can adjust the sensitivity of the custom displacement extraction layer to displacement by adjusting the hyperparameter of the displacement threshold to adapt to the working conditions of the ultrasonic radar and the non-uniformity of point cloud scanning, thereby improving the accuracy of subway automated monitoring and early warning. Description of the Drawings

[0059] Figure 1 It is a module diagram of a subway automated monitoring and early warning system based on artificial intelligence proposed by the present invention;

[0060] Figure 2 It is a model sketch of the subway monitoring neural network model proposed by the present invention;

[0061] Figure 3 It is a schematic diagram of the point cloud feature extraction branch proposed by the present invention;

[0062] Figure 4 It is a schematic diagram of the crack feature analysis branch proposed by the present invention;

[0063] Figure 5 It is a schematic diagram of the structure feature analysis branch proposed by the present invention;

[0064] Figure 6 It is a schematic diagram of the disease diagnosis and early warning branch proposed by the present invention.

[0065] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, and do not constitute a limitation to the present invention. Detailed Embodiments

[0066] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the examples given are only used to explain the present invention, and are not used to limit the scope of the present invention.

[0067] Example 1. Refer to Figure 1, an artificial intelligence-based subway automatic monitoring and early warning system, comprising: a temperature and humidity meter, an ultrasonic radar, a thermal imager, and a central processing module;

[0068] The temperature and humidity meter, the thermal imager, and the ultrasonic radar are respectively electrically connected to the central processing module;

[0069] The ultrasonic radar collects tunnel ultrasonic point clouds and uploads them to the central processing module;

[0070] The temperature and humidity meter collects the tunnel temperature and the relative humidity inside the tunnel and uploads them to the central processing module;

[0071] The thermal imager collects tunnel infrared images and uploads them to the central processing module;

[0072] The central processing module corrects the tunnel ultrasonic point clouds according to the tunnel temperature and the relative humidity inside the tunnel, then registers the tunnel infrared images with the tunnel ultrasonic point clouds, and assigns values to each point in the tunnel ultrasonic point clouds according to the temperature in the tunnel infrared images to generate temperature position point clouds, and constructs a three-dimensional model of the tunnel temperature position point clouds according to the temperature position point clouds;

[0073] The central processing module constructs a subway monitoring neural network model and uses the subway monitoring neural network model for monitoring.

[0074] Embodiment 2, based on the above embodiment, the process of the central processing module correcting the tunnel ultrasonic point clouds according to the tunnel temperature and the relative humidity inside the tunnel includes the following steps:

[0075] Step P1: Extract the horizontal angle, vertical angle, and pre-correction distance corresponding to each point from the tunnel ultrasonic point clouds, and divide the pre-correction distance corresponding to each point by 343 to obtain the acoustic wave round-trip time corresponding to each point;

[0076] Step P2: Apply the ultrasonic distance correction formula to each point in the tunnel ultrasonic point clouds to determine the actual distance between the point and the ultrasonic radar. The ultrasonic distance correction formula is as follows:

[0077] ;

[0078] ;

[0079] Wherein, represents the actual ultrasonic wave speed, the unit of the actual ultrasonic wave speed is meters per second, C represents the tunnel temperature, the unit of the tunnel temperature is degrees Celsius, represents the relative humidity inside the tunnel, represents the acoustic wave round-trip time, the unit is seconds, represents the actual distance between the point and the ultrasonic radar;

[0080] Step P3: Regenerate the tunnel ultrasonic point cloud according to the horizontal angle, vertical angle corresponding to each point, and the actual distance between the point and the ultrasonic radar.

[0081] Embodiment 3. Refer to Figure 2 , based on the above embodiment, the process of the central processing module constructing the subway monitoring neural network model specifically includes the following steps:

[0082] Step S1: Construct a tunnel disease dataset;

[0083] Step S2: Construct a point cloud feature extraction branch;

[0084] Step S3: Construct a crack feature analysis branch and connect the point cloud feature extraction branch to the crack feature analysis branch;

[0085] Step S4: Construct a structural feature analysis branch and connect the point cloud feature extraction branch to the structural feature analysis branch;

[0086] Step S5: Construct a disease diagnosis and early warning branch and connect the crack feature analysis branch and the structural feature analysis branch to the disease diagnosis and early warning branch;

[0087] Step S6: Construct a subway monitoring neural network model and use the tunnel disease dataset to train the subway monitoring neural network model.

[0088] Embodiment 4. Based on the above embodiment, the specific steps of step S1 are as follows:

[0089] Step S11: Collect the three-dimensional model of the tunnel temperature position point cloud at the completion of other subway tunnels as the reference point cloud three-dimensional model, collect the three-dimensional model of the tunnel temperature position point cloud when tunnel diseases occur in other subway tunnels as the disease point cloud three-dimensional model, and collect the relative humidity in the tunnel while collecting the disease point cloud three-dimensional model;

[0090] Step S12: Mark the length of the widest crack, the width of the widest crack, and the depth of the widest crack in each disease point cloud three-dimensional model;

[0091] Step S13: Mark the segment convergence deformation level, the health degree of the interval tunnel, and the causes of tunnel diseases in each disease point cloud three-dimensional model, and encode the causes of tunnel diseases in alphabetical order to generate tunnel disease cause codes;

[0092] Step S14: Summarize the 3D models of each disease point cloud, their corresponding reference point cloud 3D models, the relative humidity inside the tunnel, the length of the widest crack, the width of the widest crack, the depth of the widest crack, the segment convergence deformation level, the health status of the interval tunnel, and the tunnel disease cause code into a tunnel disease dataset. In this embodiment, all annotation work is carried out according to the standards of T / CECS 788-2020 and JTG H12-2015.

[0093] Embodiment 5, refer to Figure 3 , based on the above embodiment, the specific steps of step S2 are as follows:

[0094] Step S21: Construct a custom displacement extraction layer;

[0095] Step S22: Construct a first custom 3D extreme value pooling layer and a second custom 3D extreme value pooling layer. The first custom 3D extreme value pooling layer and the second custom 3D extreme value pooling layer are constructed by calling the same custom 3D extreme value pooling layer instance. In this embodiment, the custom 3D extreme value pooling layer instance is implemented using layers.core.Lambda in Keras;

[0096] Step S23: Construct a first 3D convolutional layer, a second 3D convolutional layer, and a first splicing and fusion layer. The weights of the first 3D convolutional layer and the second 3D convolutional layer are shared. In this embodiment, the weight sharing between the first 3D convolutional layer and the second 3D convolutional layer is implemented by calling the same instance, and the first splicing and fusion layer is implemented using keras.layers.Concatenate;

[0097] Step S24: Connect the custom displacement extraction layer to the first custom 3D extreme value pooling layer, connect the first custom 3D extreme value pooling layer to the first 3D convolutional layer, connect the second custom 3D extreme value pooling layer to the second 3D convolutional layer, and connect the first 3D convolutional layer and the second 3D convolutional layer to the first splicing and fusion layer.

[0098] Example 6. This example is based on the above example. The hyperparameters of the custom displacement extraction layer include a displacement threshold. The custom displacement extraction layer receives two 3D point cloud models as inputs. The custom displacement extraction layer traverses each point in the first 3D point cloud model among the two 3D point cloud models, and determines whether at least one point can be found in the second 3D point cloud model within the area with the displacement threshold as the radius for the traversed point in the first 3D point cloud model. If not, this traversed point is marked. After the traversal of the first 3D point cloud model is completed, a new 3D point cloud model is generated based on all the marked points as the output. In this example, the custom displacement extraction layer is implemented by using layers.core.Lambda in Keras in combination with search_radius_vector_3d in Open3D.

[0099] Example 7. This example is based on the above example. The instance of the custom 3D extreme value pooling layer has two 3D pooling windows. The first 3D pooling window among the two 3D pooling windows takes the minimum value within the window to generate a pooling result, and the second 3D pooling window among the two 3D pooling windows takes the maximum value within the window to generate a pooling result. Finally, the pooling result of the first 3D pooling window and the pooling result of the second 3D pooling window are superimposed to generate a dual-channel 3D feature map and then output.

[0100] Example 8. Refer to Figure 4 , this example is based on the above example. Step S3 specifically includes the following steps:

[0101] Step S31: Construct a first separable 3D convolutional layer, a first flattening layer, a second flattening layer, a first fully connected layer, a second fully connected layer, and a 3D Xception model. The first separable 3D convolutional layer extracts the crack depth features in its input, and the 3D Xception model extracts the crack width features and crack length features in its input. In this example, the 3D Xception model is implemented by replacing the convolutional kernels of each layer in keras.applications.xception with 3D convolutional kernels. The first separable 3D convolutional layer is implemented by replacing the 2D convolutional kernels of the separable 2D convolutional layer with 3D convolutional kernels. The separable 3D convolutional layer performs 3D convolutions on the inputs of different channels respectively and then mixes them together for pointwise convolution. The formula of the separable 3D convolutional layer is as follows:

[0102] ;

[0103] Among them, represents the output of the separable 3D convolutional layer, represents pointwise convolution, represents the number of channels of the separable 3D convolutional input. In this example, A = 2, Represents a three-dimensional convolution, represents the input of the th channel of the separable three-dimensional convolutional layer, represents the convolutional kernel corresponding to the th channel of the separable three-dimensional convolutional layer, represents the convolutional kernel of the pointwise convolution;

[0104] Step S32: Connect the first splicing and fusion layer to the first separable three-dimensional convolutional layer and the three-dimensional Xception model, connect the first separable three-dimensional convolutional layer, the first flattening layer, and the first fully connected layer in sequence, and connect the three-dimensional Xception model, the second flattening layer, and the second fully connected layer in sequence.

[0105] Example 9, refer to Figure 5 , based on the above example, the specific steps of step S4 are as follows:

[0106] Step S41: Construct a three-dimensional local connection layer, a third flattening layer, and a third fully connected layer. The three-dimensional local connection layer applies independent convolutional kernels to different parts of its input to extract features related to specific positions. In this example, the three-dimensional local connection layer is implemented by replacing the two-dimensional convolutional kernel of the two-dimensional local connection layer with a three-dimensional convolutional kernel;

[0107] Step S42: Connect the first splicing and fusion layer to the three-dimensional local connection layer, and connect the three-dimensional local connection layer, the third flattening layer, and the third fully connected layer in sequence.

[0108] Example 10, refer to Figure 6 , based on the above example, the specific steps of step S5 are as follows:

[0109] Step S51: Construct a second splicing and fusion layer, a third splicing and fusion layer, a second separable three-dimensional convolutional layer, a fourth flattening layer, a fourth fully connected layer, a fifth fully connected layer, and a fourth splicing and fusion layer;

[0110] Step S52: Connect the first separable three-dimensional convolutional layer, the three-dimensional Xception model, and the three-dimensional local connection layer to the second splicing and fusion layer, connect the first fully connected layer, the second fully connected layer, and the third fully connected layer to the third splicing and fusion layer, connect the second splicing and fusion layer, the second separable three-dimensional convolutional layer, the fourth flattening layer, and the third splicing and fusion layer in sequence, connect the third splicing and fusion layer to the fourth fully connected layer, connect the fourth fully connected layer and the third splicing and fusion layer to the fourth splicing and fusion layer, and connect the fourth splicing and fusion layer to the fifth fully connected layer.

[0111] Example 11, based on the above example, the specific steps of step S6 are as follows:

[0112] Step S61: Designate the custom displacement extraction layer, the second custom three-dimensional extreme pooling layer, and the fourth splicing and fusion layer as the input layer of the subway monitoring neural network model, and designate the first fully connected layer, the second fully connected layer, the third fully connected layer, the fourth fully connected layer, and the fifth fully connected layer as the output layer of the subway monitoring neural network model;

[0113] Step S62: Divide the tunnel disease dataset into a training set and a test set at a ratio of 8:2. Designate the disease point cloud three-dimensional model and the reference point cloud three-dimensional model in the tunnel disease dataset as the input of the custom displacement extraction layer, designate the reference point cloud three-dimensional model in the tunnel disease dataset as the input of the second custom three-dimensional extreme pooling layer, designate the relative humidity inside the tunnel in the tunnel disease dataset as the input of the fourth splicing and fusion layer, designate the depth of the widest crack in the tunnel disease dataset as the target output of the first fully connected layer, designate the length and width of the widest crack in the tunnel disease dataset as the target output of the second fully connected layer, designate the segment convergence deformation level in the tunnel disease dataset as the target output of the third fully connected layer, designate the tunnel disease cause code as the target output of the fifth fully connected layer, and designate the interval tunnel health in the tunnel disease dataset as the target output of the fourth fully connected layer. Train the subway monitoring neural network model, and use Bayesian optimization to adjust the value of the displacement threshold with the goal of minimizing the depth loss value of the widest crack. In this embodiment, Bayesian optimization is implemented using the bayesian-optimization library.

[0114] Embodiment 12. Based on the above embodiment, the process of using the subway monitoring neural network model for monitoring specifically includes the following steps:

[0115] Step M1: Collect the tunnel temperature position point cloud three-dimensional model at the time of completion of the target subway tunnel as the target reference point cloud three-dimensional model, collect the tunnel temperature position point cloud three-dimensional model of the target subway tunnel currently as the target current point cloud three-dimensional model, and simultaneously collect the relative humidity inside the tunnel of the target subway tunnel currently;

[0116] Step M2: Input the target current point cloud three-dimensional model and the target reference point cloud three-dimensional model into the custom displacement extraction layer, input the target reference point cloud three-dimensional model into the second custom three-dimensional extreme pooling layer, and input the relative humidity inside the tunnel of the target subway tunnel currently into the fourth splicing and fusion layer. The subway monitoring neural network model outputs the interval tunnel health, tunnel disease cause code, length of the widest crack, width of the widest crack, depth of the widest crack, and segment convergence deformation level of the target subway tunnel;

[0117] Step M3: Adjust the value of the displacement threshold according to the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack: If the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack exceeds 5%, then use Bayesian optimization to re-determine the value of the displacement threshold with the goal of minimizing the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack, and then return to Step M2; otherwise, proceed to Step M4. In this embodiment, the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack does not exceed 5%, so proceed to Step M4;

[0118] Step M4: If the health degree of the interval tunnel of the target subway tunnel reaches 4 or above, issue a tunnel disease warning; otherwise, do not issue a tunnel disease warning.

[0119] Embodiment Thirteen. This embodiment is based on the above embodiment. This embodiment runs in the Windows operating system environment, relies on the Anacond3 environment, and uses the Open3D library to complete point cloud related processing. Keras is used as the algorithm framework of the subway monitoring neural network model. The thermal imager is implemented, the ultrasonic radar is implemented by AK2 USS, the central processing module is implemented by Dell Precision T5820, and the thermometer and hygrometer are implemented by KENTA temperature and humidity detector.

[0120] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

[0121] The above describes the present invention and its implementation manners. This description is not restrictive. What is shown in the drawings is only one of the implementation manners of the present invention. The actual structure is not limited thereto. In short, if those of ordinary skill in the art are inspired by the present invention and, without departing from the purpose of the present invention's creation, creatively design a structural manner and an embodiment similar to this technical solution, they shall fall within the protection scope of the present invention.

Claims

1. An artificial intelligence-based subway automatic monitoring and early warning system, characterized by: The subway automatic monitoring and early warning system based on artificial intelligence comprises: a thermometer and hygrometer, an ultrasonic radar, a thermal imager and a central processing module; The thermometer, the thermal imager and the ultrasonic radar are electrically connected to the central processing module respectively; The ultrasonic radar collects the ultrasonic point cloud of the tunnel and uploads it to the central processing module; The thermometer and hygrometer collect the temperature and relative humidity in the tunnel and upload them to the central processing module; The thermal imager collects tunnel infrared images and uploads them to the central processing module; The central processing module corrects the tunnel ultrasonic point cloud according to the temperature in the tunnel and the relative humidity in the tunnel to generate a corrected tunnel ultrasonic point cloud; aligns the tunnel infrared image with the corrected tunnel ultrasonic point cloud, assigns values ​​to each point in the corrected tunnel ultrasonic point cloud to generate a temperature position point cloud; and constructs a three-dimensional model of the tunnel temperature position point cloud according to the temperature position point cloud; The central processing module constructs a subway monitoring neural network model based on the tunnel temperature position point cloud three-dimensional model and uses the subway monitoring neural network model for monitoring; The process of the central processing module using the subway monitoring neural network model to perform monitoring specifically includes the following steps: Step S1: construct tunnel disease dataset; Step S2: Using the tunnel temperature position point cloud 3D model as the target reference point cloud 3D model to construct a point cloud feature extraction branch, using the tunnel disease dataset as input to extract point cloud features; Step S3: construct a crack feature analysis branch, and based on the point cloud features, connect the output end of the point cloud feature extraction branch to the crack feature analysis branch to extract the crack features; Step S4: construct a structural feature analysis branch, and based on the point cloud features, connect the output end of the point cloud feature extraction branch to the structural feature analysis branch to generate structural features; Step S5: construct a disease diagnosis and early warning branch, and connect the crack feature analysis branch and the structural feature analysis branch to the disease diagnosis and early warning branch, and obtain a tunnel disease early warning based on the crack features and structural features extracted in steps S3 and S4.

2. According to claim 1, an artificial intelligence-based subway automatic monitoring and early warning system is characterized in that: The process of the central processing module correcting the tunnel ultrasonic point cloud according to the temperature in the tunnel and the relative humidity in the tunnel includes the following steps: Step P1: extract the horizontal angle, vertical angle and pre-correction distance corresponding to each point from the tunnel ultrasonic point cloud, and divide the pre-correction distance corresponding to each point by 343 to obtain the sound wave round-trip time corresponding to each point; Step P2: Apply the ultrasonic distance correction formula to each point in the tunnel ultrasonic point cloud to determine the actual distance between the point and the ultrasonic radar; Step P3: Regenerate the tunnel ultrasonic point cloud according to the horizontal angle, vertical angle and actual distance between each point and the ultrasonic radar.

3. The subway automatic monitoring and early warning system based on artificial intelligence according to claim 1 is characterized by: In step S2, constructing a point cloud feature extraction branch specifically includes the following steps: Step S21: construct a custom displacement extraction layer; Step S22: constructing a first custom three-dimensional extreme value pooling layer and a second custom three-dimensional extreme value pooling layer, wherein the first custom three-dimensional extreme value pooling layer and the second custom three-dimensional extreme value pooling layer are constructed by calling the same custom three-dimensional extreme value pooling layer instance; Step S23: constructing a first three-dimensional convolutional layer, a second three-dimensional convolutional layer and a first splicing fusion layer, wherein the first three-dimensional convolutional layer and the second three-dimensional convolutional layer share weights; Step S24: connect the custom displacement extraction layer to the first custom three-dimensional extreme value pooling layer, connect the first custom three-dimensional extreme value pooling layer to the first three-dimensional convolutional layer, connect the second custom three-dimensional extreme value pooling layer to the second three-dimensional convolutional layer, and connect the first three-dimensional convolutional layer and the second three-dimensional convolutional layer to the first splicing fusion layer.

4. The subway automatic monitoring and early warning system based on artificial intelligence according to claim 3 is characterized by: The hyperparameters of the custom displacement extraction layer include a displacement threshold. The custom displacement extraction layer receives two point cloud 3D models as inputs. The custom displacement extraction layer traverses each point of the first point cloud 3D model of the two point cloud 3D models, and determines whether at least one point can be found in the second point cloud 3D model within an area with the displacement threshold as a radius of the traversed point in the first point cloud 3D model. If not found, the traversed point is marked. After the traversal of the first point cloud 3D model is completed, a new point cloud 3D model is generated as output based on all the marked points.

5. The subway automatic monitoring and early warning system based on artificial intelligence according to claim 4 is characterized by: The custom three-dimensional extreme value pooling layer instance has two three-dimensional pooling windows. The first three-dimensional pooling window of the two three-dimensional pooling windows takes the minimum value within the window to generate a pooling result, and the second three-dimensional pooling window of the two three-dimensional pooling windows takes the maximum value within the window to generate a pooling result. Finally, the pooling result of the first three-dimensional pooling window is superimposed with the pooling result of the second three-dimensional pooling window to generate a dual-channel three-dimensional feature map and then output.

Images