Subway automatic monitoring and early warning system based on artificial intelligence

Abstract

The invention relates to the technical field of subway monitoring, in particular to an automatic subway monitoring and early warning system based on artificial intelligence, which comprises a hygrothermograph, an ultrasonic radar, a thermal imager and a central processing module, the central processing module corrects the tunnel ultrasonic point cloud according to the temperature in the tunnel and the relative humidity in the tunnel and generates a temperature position point cloud after registering the tunnel infrared image with the ultrasonic point cloud, and the difference between the temperature position point cloud and a laser point cloud in the prior art is that each point in the temperature position point cloud has a corresponding temperature. The temperature position point cloud represents the tunnel structure and also represents the temperature distribution in the tunnel, and the technical feature of the temperature position point cloud helps a subway monitoring neural network model to more easily diagnose tunnel diseases caused by temperature changes. According to the invention, the multi-modal fusion technology, the ultrasonic point cloud scanning technology and the infrared imaging technology are combined to realize multi-modal, moisture-resistant and high-reliability metro automatic monitoring and early warning.

Description

Technical Field

[0001] The present invention relates to the technical field of subway monitoring, and in particular to an artificial intelligence-based subway automatic monitoring and early warning system. Background Art

[0002] The subway automatic monitoring and early warning system is a system used to monitor the status of subway tunnels. It collects, analyzes and processes data through automated means to timely discover potential problems or abnormal situations. The existing subway automatic monitoring mainly includes two technical routes: multi-source sensors and laser point cloud scanning. Multi-source sensors monitor the status of subway tunnels by comprehensively analyzing the readings of static levels, crack meters, inclinometers and vibrometers. The problem is that this solution has too many potential failure points, which leads to high maintenance costs and low reliability. In addition, the sensor readings cannot fully describe the structural status of the subway tunnel. Laser point cloud scanning can fully describe the structural status of the subway tunnel, but the problem is that laser equipment is difficult to cope with extreme humidity changes. The high humidity state when the subway tunnel leaks will cause condensation on the optical components of the laser equipment. Therefore, laser point cloud scanning is still not suitable for long-term subway tunnel monitoring tasks. In addition, laser point cloud scanning cannot collect the temperature distribution of the subway tunnel, and thus cannot determine the cause of tunnel diseases. Summary of the invention

[0003] The present invention overcomes the shortcomings of the prior art and provides an artificial intelligence-based subway automatic monitoring and early warning system, including a thermometer and hygrometer, an ultrasonic radar, a thermal imager and a central processing module. The ultrasonic radar has a much higher tolerance to extreme humidity changes than a laser point cloud scanner. The central processing module corrects the tunnel ultrasonic point cloud according to the temperature in the tunnel and the relative humidity in the tunnel, and generates a temperature position point cloud after aligning the tunnel infrared image with the tunnel ultrasonic point cloud. The difference between the temperature position point cloud and the laser point cloud in the prior art is that each point in the temperature position point cloud has a corresponding temperature. The temperature position point cloud not only characterizes the tunnel structure, but also characterizes the temperature distribution in the tunnel. The technical feature of the temperature position point cloud helps the subway monitoring neural network model to more easily diagnose tunnel diseases caused by temperature changes such as freezing cracks and expansion cracks. The present invention realizes multimodal, moisture-resistant and highly reliable subway automatic monitoring and early warning by combining multimodal fusion technology, tunnel ultrasonic point cloud scanning technology and infrared imaging technology.

[0004] An artificial intelligence-based subway automatic monitoring and early warning system, comprising: a thermometer and hygrometer, an ultrasonic radar, a thermal imager and a central processing module;

[0005] The thermometer, the thermal imager and the ultrasonic radar are electrically connected to the central processing module respectively;

[0006] The ultrasonic radar collects the ultrasonic point cloud of the tunnel and uploads it to the central processing module;

[0007] The thermometer and hygrometer collect the temperature and relative humidity in the tunnel and upload them to the central processing module;

[0008] The thermal imager collects tunnel infrared images and uploads them to the central processing module;

[0009] The central processing module corrects the tunnel ultrasonic point cloud according to the temperature in the tunnel and the relative humidity in the tunnel, aligns the tunnel infrared image with the tunnel ultrasonic point cloud, assigns a value to each point in the tunnel ultrasonic point cloud according to the temperature in the tunnel infrared image to generate a temperature position point cloud, and constructs a three-dimensional model of the tunnel temperature position point cloud according to the temperature position point cloud;

[0010] The central processing module constructs a subway monitoring neural network model and uses the subway monitoring neural network model for monitoring.

[0011] Furthermore, the process of the central processing module correcting the tunnel ultrasonic point cloud according to the temperature in the tunnel and the relative humidity in the tunnel includes the following steps:

[0012] Step P1: extract the horizontal angle, vertical angle and pre-correction distance corresponding to each point from the tunnel ultrasonic point cloud, and divide the pre-correction distance corresponding to each point by 343 to obtain the sound wave round-trip time corresponding to each point;

[0013] Step P2: Apply the ultrasonic distance correction formula to each point in the tunnel ultrasonic point cloud to determine the actual distance between the point and the ultrasonic radar. The ultrasonic distance correction formula is as follows:

[0014] ;

[0015] ;

[0016] in, represents the actual ultrasonic velocity, the unit of which is meters per second, C represents the temperature in the tunnel, the unit of which is degrees Celsius, represents the relative humidity in the tunnel, Represents the round trip time of the sound wave in seconds. The actual distance between the representative point and the ultrasonic radar;

[0017] Step P3: Regenerate the tunnel ultrasonic point cloud according to the horizontal angle, vertical angle and actual distance between each point and the ultrasonic radar.

[0018] Furthermore, the process of constructing the subway monitoring neural network model by the central processing module specifically includes the following steps:

[0019] Step S1: construct tunnel disease dataset;

[0020] Step S2: construct a point cloud feature extraction branch;

[0021] Step S3: construct a crack feature analysis branch, and connect the point cloud feature extraction branch to the crack feature analysis branch;

[0022] Step S4: construct a structural feature analysis branch, and connect the point cloud feature extraction branch to the structural feature analysis branch;

[0023] Step S5: construct a disease diagnosis and early warning branch, and connect the crack feature analysis branch and the structure feature analysis branch to the disease diagnosis and early warning branch;

[0024] Step S6: Construct a subway monitoring neural network model and use the tunnel disease dataset to train the subway monitoring neural network model.

[0025] Furthermore, the step S1 specifically includes the following steps:

[0026] Step S11: collecting the tunnel temperature position point cloud 3D models of other subway tunnels when they are completed as the reference point cloud 3D models, collecting the tunnel temperature position point cloud 3D models of other subway tunnels when tunnel diseases occur as the disease point cloud 3D models, and collecting the relative humidity in the tunnel while collecting the disease point cloud 3D models;

[0027] Step S12: marking the length of the widest crack, the width of the widest crack and the depth of the widest crack in each of the three-dimensional models of the defect point cloud;

[0028] Step S13: marking the segment convergence deformation level, interval tunnel health and tunnel disease cause of each defect point cloud 3D model, and encoding the tunnel disease cause in alphabetical order to generate a tunnel disease cause code;

[0029] Step S14: The three-dimensional model of each defect point cloud and its corresponding three-dimensional model of the reference point cloud, the relative humidity in the tunnel, the length of the widest crack, the width of the widest crack, the depth of the widest crack, the convergence deformation level of the segment, the health of the interval tunnel and the tunnel disease cause code are summarized into a tunnel disease data set.

[0030] Furthermore, the step S2 specifically includes the following steps:

[0031] Step S21: construct a custom displacement extraction layer;

[0032] Step S22: constructing a first custom three-dimensional extreme value pooling layer and a second custom three-dimensional extreme value pooling layer, wherein the first custom three-dimensional extreme value pooling layer and the second custom three-dimensional extreme value pooling layer are constructed by calling the same custom three-dimensional extreme value pooling layer instance;

[0033] Step S23: constructing a first three-dimensional convolutional layer, a second three-dimensional convolutional layer and a first splicing fusion layer, wherein the first three-dimensional convolutional layer and the second three-dimensional convolutional layer share weights;

[0034] Step S24: connect the custom displacement extraction layer to the first custom three-dimensional extreme value pooling layer, connect the first custom three-dimensional extreme value pooling layer to the first three-dimensional convolutional layer, connect the second custom three-dimensional extreme value pooling layer to the second three-dimensional convolutional layer, and connect the first three-dimensional convolutional layer and the second three-dimensional convolutional layer to the first splicing fusion layer.

[0035] Furthermore, the hyperparameters of the custom displacement extraction layer include a displacement threshold. The custom displacement extraction layer receives two point cloud 3D models as inputs. The custom displacement extraction layer traverses each point of the first point cloud 3D model of the two point cloud 3D models, and determines whether at least one point can be found in the second point cloud 3D model within an area with the displacement threshold as a radius of the traversed point in the first point cloud 3D model. If not found, the traversed point is marked. After the traversal of the first point cloud 3D model is completed, a new point cloud 3D model is generated as output based on all the marked points.

[0036] Furthermore, the custom three-dimensional extreme pooling layer instance has two three-dimensional pooling windows. The first three-dimensional pooling window of the two three-dimensional pooling windows takes the minimum value within the window to generate a pooling result, and the second three-dimensional pooling window of the two three-dimensional pooling windows takes the maximum value within the window to generate a pooling result. Finally, the pooling result of the first three-dimensional pooling window is superimposed with the pooling result of the second three-dimensional pooling window to generate a dual-channel three-dimensional feature map and then output.

[0037] Furthermore, the step S3 specifically includes the following steps:

[0038] Step S31: constructing a first separable three-dimensional convolutional layer, a first flattening layer, a second flattening layer, a first fully connected layer, a second fully connected layer and a three-dimensional Xception model, wherein the first separable three-dimensional convolutional layer extracts crack depth features in its input, and the three-dimensional Xception model extracts crack width features and crack length features in its input;

[0039] Step S32: Connect the first concatenated fusion layer to the first separable 3D convolutional layer and the 3D Xception model, connect the first separable 3D convolutional layer, the first flattened layer and the first fully connected layer in sequence, and connect the 3D Xception model, the second flattened layer and the second fully connected layer in sequence.

[0040] Furthermore, the step S4 specifically includes the following steps:

[0041] Step S41: constructing a three-dimensional local connection layer, a third flattening layer, and a third fully connected layer, wherein the three-dimensional local connection layer applies independent convolution kernels to different parts of its input to extract features related to a specific position;

[0042] Step S42: Connect the first splicing fusion layer to the three-dimensional local connection layer, and connect the three-dimensional local connection layer, the third flattening layer, and the third fully connected layer in sequence.

[0043] Furthermore, the step S5 specifically includes the following steps:

[0044] Step S51: construct a second splicing and fusion layer, a third splicing and fusion layer, a second separable three-dimensional convolution layer, a fourth flattening layer, a fourth fully connected layer, a fifth fully connected layer and a fourth splicing and fusion layer;

[0045] Step S52: connect the first separable 3D convolutional layer, the 3D Xception model and the 3D local connection layer to the second splicing fusion layer, connect the first fully connected layer, the second fully connected layer and the third fully connected layer to the third splicing fusion layer, connect the second splicing fusion layer, the second separable 3D convolutional layer, the fourth flattening layer and the third splicing fusion layer in sequence, connect the third splicing fusion layer to the fourth fully connected layer, connect the fourth fully connected layer and the third splicing fusion layer to the fourth splicing fusion layer, and connect the fourth splicing fusion layer to the fifth fully connected layer.

[0046] Furthermore, the step S6 specifically includes the following steps:

[0047] Step S61: designate the custom displacement extraction layer, the second custom three-dimensional extreme value pooling layer, and the fourth splicing fusion layer as the input layer of the subway monitoring neural network model, and designate the first fully connected layer, the second fully connected layer, the third fully connected layer, the fourth fully connected layer, and the fifth fully connected layer as the output layer of the subway monitoring neural network model;

[0048] Step S62: Divide the tunnel disease dataset into a training set and a test set in a ratio of 8:2, designate the disease point cloud 3D model and the reference point cloud 3D model in the tunnel disease dataset as the input of the custom displacement extraction layer, designate the reference point cloud 3D model in the tunnel disease dataset as the input of the second custom 3D extreme value pooling layer, designate the relative humidity in the tunnel in the tunnel disease dataset as the input of the fourth splicing and fusion layer, designate the depth of the widest crack in the tunnel disease dataset as the target output of the first fully connected layer, designate the length of the widest crack and the width of the widest crack in the tunnel disease dataset as the target output of the second fully connected layer, designate the segment convergence deformation level in the tunnel disease dataset as the target output of the third fully connected layer, designate the tunnel disease cause code as the target output of the fifth fully connected layer, designate the interval tunnel health as the target output of the fourth fully connected layer, train the subway monitoring neural network model, and use Bayesian optimization to adjust the value of the displacement threshold with the goal of minimizing the depth loss value of the widest crack.

[0049] Furthermore, the process of using the subway monitoring neural network model for monitoring specifically includes the following steps:

[0050] Step M1: collecting the tunnel temperature position point cloud 3D model of the target subway tunnel when it is completed as the target reference point cloud 3D model, collecting the current tunnel temperature position point cloud 3D model of the target subway tunnel as the target current point cloud 3D model, and collecting the current relative humidity in the target subway tunnel;

[0051] Step M2: input the target current point cloud 3D model and the target benchmark point cloud 3D model into the custom displacement extraction layer, input the target benchmark point cloud 3D model into the second custom 3D extreme value pooling layer, input the current relative humidity in the target subway tunnel into the fourth splicing and fusion layer, and the subway monitoring neural network model outputs the interval tunnel health degree, tunnel disease cause code, the length of the widest crack, the width of the widest crack, the depth of the widest crack and the segment convergence deformation level of the target subway tunnel;

[0052] Step M3: adjusting the value of the displacement threshold according to the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack: if the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack exceeds 5%, then using Bayesian optimization to minimize the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack as the goal to redetermine the value of the displacement threshold and then return to step M2, otherwise proceed to step M4;

[0053] Step M4: If the health level of the interval tunnel of the target subway tunnel reaches 4 or above, a tunnel disease warning is issued, otherwise no tunnel disease warning is issued.

[0054] The beneficial effects of the present invention are as follows:

[0055] (1) The present invention provides an artificial intelligence-based subway automatic monitoring and early warning system, comprising a thermometer, an ultrasonic radar, a thermal imager and a central processing module. The ultrasonic radar has a much higher tolerance to extreme humidity changes than a laser point cloud scanner. The central processing module corrects the tunnel ultrasonic point cloud according to the temperature and relative humidity in the tunnel and aligns the tunnel infrared image with the tunnel ultrasonic point cloud to generate a temperature position point cloud. The difference between the temperature position point cloud and the laser point cloud in the prior art is that each point in the temperature position point cloud has a corresponding temperature. The temperature position point cloud not only characterizes the tunnel structure but also characterizes the temperature distribution in the tunnel. The technical feature of the temperature position point cloud helps the subway monitoring neural network model to more easily diagnose tunnel diseases caused by temperature changes. The present invention realizes multimodal, moisture-resistant and highly reliable subway automatic monitoring and early warning by combining multimodal fusion technology, tunnel ultrasonic point cloud scanning technology and infrared imaging technology.

[0056] (2) The default value of the sound speed of ultrasonic radar is generally 1 and the speed of sound at 20°C, which is 343m / s. Although this is sufficient for most application scenarios, the ultrasonic distance correction formula is used in the present invention to enable it to maintain a high accuracy under extreme temperature and humidity changes, which is impossible for existing laser point cloud scanning solutions.

[0057] (3) The subway monitoring neural network model proposed in the present invention generally adopts the divide-and-conquer design idea. First, the features in the point cloud three-dimensional model are extracted through the point cloud feature extraction branch. Then, the crack feature analysis branch and the structural feature analysis branch are used to analyze the local and overall features respectively through the hard sharing mechanism and output relevant feature maps and indicators. Finally, the disease diagnosis and early warning branch obtains the health of the interval tunnel and the tunnel disease cause code. The point cloud feature extraction branch is characterized in that the crack and deformation features of the tunnel are extracted through the custom displacement extraction layer and the extreme temperature distribution is extracted while retaining the structural features through the custom three-dimensional extreme value pooling layer. This can help subway monitoring and early warning. The system can more easily diagnose tunnel diseases caused by temperature changes. The weight sharing mechanism between the first 3D convolutional layer and the second 3D convolutional layer can maintain the consistency of features while further extracting features, which is convenient for further comparison and analysis of the crack feature analysis branch. The first separable 3D convolutional layer in the crack feature analysis branch first convolves different input channels separately and then mixes all input channels together for point-by-point convolution, which can effectively extract the crack depth features in its input. The 3D Xception model can effectively extract crack width features and crack length features through multi-scale feature extraction, and the structural feature analysis branch uses a 3D local connection layer. The local connection layer extracts features related to specific locations by applying independent convolution kernels at each location. This is particularly suitable for extracting features related to tunnel structure deformation, because different locations of the tunnel structure obviously have different importance for structural stability. By combining information such as temperature distribution, relative humidity in the tunnel, tunnel structure and cracks in the tunnel, the disease diagnosis and early warning branch can derive the health of the interval tunnel and the cause code of tunnel disease, which is impossible with simple laser scanning in the existing technology. The subway monitoring neural network model organically combines the multi-task learning mechanism with the multi-modal learning mechanism, adopts the divide-and-conquer method and the syllogism approach, and first extracts point cloud features. , then analyze the local and overall characteristics respectively, and finally summarize the local and overall characteristics to realize the early warning and diagnosis of tunnel diseases. The difference from the technical effect of the existing subway monitoring and early warning system is that the present invention can not only directly obtain the length, width and depth of the segment convergence deformation and grade cracks, but also obtain the health of the interval tunnel and the cause of tunnel diseases. The relative humidity in the tunnel also plays a key role in the diagnosis of the cause of tunnel diseases. Without the relative humidity in the tunnel, it is impossible to diagnose the tunnel diseases caused by leakage. The diagnosis of the cause of tunnel diseases is impossible for the existing subway monitoring and early warning system that does not refer to the temperature distribution and the relative humidity in the tunnel.

[0058] (4) The custom displacement extraction layer proposed in the present invention traverses each point of the first point cloud 3D model of the two point cloud 3D models, and determines whether at least one point can be found in the second point cloud 3D model within the area with the displacement threshold as the radius of the traversed point in the first point cloud 3D model. If not, the traversed point is marked. The first point cloud 3D model represents the current structure of the target subway tunnel, and the second point cloud 3D model represents the structure of the target subway tunnel when it is just completed. The principle is that the points within the area with the displacement threshold as the radius of the traversed point are regarded as the same point as the traversed point. If no corresponding point is found within this radius, the traversed point is marked as having been displaced. The 3D model composed of all the marked points can effectively represent the cracks and deformation of the tunnel. The advantage of the custom displacement extraction layer over the existing technology is that it has a high degree of interpretability, and the sensitivity of the custom displacement extraction layer to displacement can be adjusted by adjusting the hyperparameter of the displacement threshold to adapt to the working condition changes of the ultrasonic radar and the unevenness of the point cloud scanning, thereby improving the accuracy of the subway automatic monitoring and early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a module diagram of an artificial intelligence-based subway automatic monitoring and early warning system proposed by the present invention;

[0060] Figure 2 This is a schematic diagram of the subway monitoring neural network model proposed by the present invention;

[0061] Figure 3 A schematic diagram of the point cloud feature extraction branch proposed in the present invention;

[0062] Figure 4 A schematic diagram of the fracture characteristic analysis branch proposed by the present invention;

[0063] Figure 5 A schematic diagram of the structural feature analysis branch proposed by the present invention;

[0064] Figure 6 This is a schematic diagram of the disease diagnosis and early warning branch proposed by the present invention.

[0065] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the examples given are only used to explain the present invention, and are not used to limit the scope of the present invention.

[0067] Example 1, see Figure 1, an artificial intelligence-based subway automated monitoring and early warning system, including: a thermometer and hygrometer, an ultrasonic radar, a thermal imager, and a central processing module;

[0068] The thermometer, the thermal imager and the ultrasonic radar are electrically connected to the central processing module respectively;

[0069] The ultrasonic radar collects the ultrasonic point cloud of the tunnel and uploads it to the central processing module;

[0070] The thermometer and hygrometer collect the temperature and relative humidity in the tunnel and upload them to the central processing module;

[0071] The thermal imager collects tunnel infrared images and uploads them to the central processing module;

[0072] The central processing module corrects the tunnel ultrasonic point cloud according to the temperature in the tunnel and the relative humidity in the tunnel, aligns the tunnel infrared image with the tunnel ultrasonic point cloud, assigns a value to each point in the tunnel ultrasonic point cloud according to the temperature in the tunnel infrared image to generate a temperature position point cloud, and constructs a three-dimensional model of the tunnel temperature position point cloud according to the temperature position point cloud;

[0073] The central processing module constructs a subway monitoring neural network model and uses the subway monitoring neural network model for monitoring.

[0074] Embodiment 2: This embodiment is based on the above embodiment. The process of the central processing module correcting the tunnel ultrasonic point cloud according to the temperature in the tunnel and the relative humidity in the tunnel includes the following steps:

[0075] Step P1: extract the horizontal angle, vertical angle and pre-correction distance corresponding to each point from the tunnel ultrasonic point cloud, and divide the pre-correction distance corresponding to each point by 343 to obtain the sound wave round-trip time corresponding to each point;

[0076] Step P2: Apply the ultrasonic distance correction formula to each point in the tunnel ultrasonic point cloud to determine the actual distance between the point and the ultrasonic radar. The ultrasonic distance correction formula is as follows:

[0077] ;

[0078] ;

[0079] in, represents the actual ultrasonic velocity, the unit of which is meters per second, C represents the temperature in the tunnel, the unit of which is degrees Celsius, represents the relative humidity in the tunnel, Represents the round trip time of the sound wave in seconds. The actual distance between the representative point and the ultrasonic radar;

[0080] Step P3: Regenerate the tunnel ultrasonic point cloud according to the horizontal angle, vertical angle and actual distance between each point and the ultrasonic radar.

[0081] Example 3, see Figure 2 This embodiment is based on the above embodiment, and the process of the central processing module constructing the subway monitoring neural network model specifically includes the following steps:

[0082] Step S1: construct tunnel disease dataset;

[0083] Step S2: construct a point cloud feature extraction branch;

[0084] Step S3: construct a crack feature analysis branch, and connect the point cloud feature extraction branch to the crack feature analysis branch;

[0085] Step S4: construct a structural feature analysis branch, and connect the point cloud feature extraction branch to the structural feature analysis branch;

[0086] Step S5: construct a disease diagnosis and early warning branch, and connect the crack feature analysis branch and the structure feature analysis branch to the disease diagnosis and early warning branch;

[0087] Step S6: Construct a subway monitoring neural network model and use the tunnel disease dataset to train the subway monitoring neural network model.

[0088] Embodiment 4: This embodiment is based on the above embodiment, and step S1 specifically includes the following steps:

[0089] Step S11: collecting the tunnel temperature position point cloud 3D models of other subway tunnels when they are completed as the reference point cloud 3D models, collecting the tunnel temperature position point cloud 3D models of other subway tunnels when tunnel diseases occur as the disease point cloud 3D models, and collecting the relative humidity in the tunnel while collecting the disease point cloud 3D models;

[0090] Step S12: marking the length of the widest crack, the width of the widest crack and the depth of the widest crack in each of the three-dimensional models of the defect point cloud;

[0091] Step S13: marking the segment convergence deformation level, interval tunnel health and tunnel disease cause of each defect point cloud 3D model, and encoding the tunnel disease cause in alphabetical order to generate a tunnel disease cause code;

[0092] Step S14: The three-dimensional model of each defect point cloud and its corresponding three-dimensional model of the reference point cloud, the relative humidity in the tunnel, the length of the widest crack, the width of the widest crack, the depth of the widest crack, the convergence deformation level of the segment, the health of the interval tunnel and the tunnel defect cause code are summarized into a tunnel defect data set. In this embodiment, all annotation work is performed in accordance with the standards of T / CECS 788-2020 and JTG H12-2015.

[0093] Example 5, see Figure 3 This embodiment is based on the above embodiment, and step S2 specifically includes the following steps:

[0094] Step S21: construct a custom displacement extraction layer;

[0095] Step S22: constructing a first custom three-dimensional extreme value pooling layer and a second custom three-dimensional extreme value pooling layer, wherein the first custom three-dimensional extreme value pooling layer and the second custom three-dimensional extreme value pooling layer are constructed by calling the same custom three-dimensional extreme value pooling layer instance. In this embodiment, the custom three-dimensional extreme value pooling layer instance is implemented using Keras layers.core.Lambda;

[0096] Step S23: construct a first three-dimensional convolutional layer, a second three-dimensional convolutional layer and a first concatenation and fusion layer, wherein the first three-dimensional convolutional layer and the second three-dimensional convolutional layer share weights. In this embodiment, the weight sharing between the first three-dimensional convolutional layer and the second three-dimensional convolutional layer is implemented by calling the same instance, and the first concatenation and fusion layer is implemented by keras.layers.Concatenate.

[0097] Step S24: connect the custom displacement extraction layer to the first custom three-dimensional extreme value pooling layer, connect the first custom three-dimensional extreme value pooling layer to the first three-dimensional convolutional layer, connect the second custom three-dimensional extreme value pooling layer to the second three-dimensional convolutional layer, and connect the first three-dimensional convolutional layer and the second three-dimensional convolutional layer to the first splicing fusion layer.

[0098] Embodiment 6. This embodiment is based on the above embodiment. The hyperparameters of the custom displacement extraction layer include a displacement threshold. The custom displacement extraction layer receives two point cloud 3D models as input. The custom displacement extraction layer traverses each point of the first point cloud 3D model of the two point cloud 3D models, and determines whether the traversed point in the first point cloud 3D model can find at least one point in the second point cloud 3D model within an area with the displacement threshold as a radius. If not, the traversed point is marked. After the traversal of the first point cloud 3D model is completed, a new point cloud 3D model is generated as an output based on all the marked points. In this embodiment, the custom displacement extraction layer is implemented using Keras' layers.core.Lambda combined with Open3D's search_radius_vector_3d.

[0099] Embodiment 7. This embodiment is based on the above embodiment. The custom three-dimensional extreme value pooling layer instance has two three-dimensional pooling windows. The first three-dimensional pooling window of the two three-dimensional pooling windows takes the minimum value in the window to generate a pooling result. The second three-dimensional pooling window of the two three-dimensional pooling windows takes the maximum value in the window to generate a pooling result. Finally, the pooling result of the first three-dimensional pooling window is superimposed with the pooling result of the second three-dimensional pooling window to generate a dual-channel three-dimensional feature map and then output.

[0100] Embodiment 8, see Figure 4 This embodiment is based on the above embodiment, and step S3 specifically includes the following steps:

[0101] Step S31: construct a first separable three-dimensional convolutional layer, a first flattening layer, a second flattening layer, a first fully connected layer, a second fully connected layer and a three-dimensional Xception model, wherein the first separable three-dimensional convolutional layer extracts crack depth features in its input, and the three-dimensional Xception model extracts crack width features and crack length features in its input. In this embodiment, the three-dimensional Xception model is implemented by replacing the convolution kernels of each layer in keras.applications.xception with three-dimensional convolution kernels, and the first separable three-dimensional convolutional layer is implemented by replacing the two-dimensional convolution kernel of the separable two-dimensional convolutional layer with a three-dimensional convolution kernel. The separable three-dimensional convolutional layer performs three-dimensional convolution on the inputs of different channels respectively and then mixes them together for point-by-point convolution. The formula of the separable three-dimensional convolutional layer is as follows:

[0102] ;

[0103] in, represents the output of a separable 3D convolutional layer, represents point-wise convolution, Represents the number of channels of the separable 3D convolution input. In this embodiment, A=2. represents a three-dimensional convolution, Represents the separable 3D convolutional layer Channel input, Represents the separable 3D convolutional layer The convolution kernel corresponding to the channel, The convolution kernel representing the point-wise convolution;

[0104] Step S32: Connect the first concatenated fusion layer to the first separable 3D convolutional layer and the 3D Xception model, connect the first separable 3D convolutional layer, the first flattened layer and the first fully connected layer in sequence, and connect the 3D Xception model, the second flattened layer and the second fully connected layer in sequence.

[0105] Embodiment 9, see Figure 5 This embodiment is based on the above embodiment, and step S4 specifically includes the following steps:

[0106] Step S41: constructing a three-dimensional local connection layer, a third flattening layer, and a third fully connected layer. The three-dimensional local connection layer applies independent convolution kernels to different parts of its input to extract features related to a specific position. In this embodiment, the three-dimensional local connection layer is implemented by replacing the two-dimensional convolution kernel of the two-dimensional local connection layer with a three-dimensional convolution kernel.

[0107] Step S42: Connect the first splicing fusion layer to the three-dimensional local connection layer, and connect the three-dimensional local connection layer, the third flattening layer, and the third fully connected layer in sequence.

[0108] Embodiment 10, see Figure 6 This embodiment is based on the above embodiment, and step S5 specifically includes the following steps:

[0109] Step S51: construct a second splicing and fusion layer, a third splicing and fusion layer, a second separable three-dimensional convolution layer, a fourth flattening layer, a fourth fully connected layer, a fifth fully connected layer and a fourth splicing and fusion layer;

[0110] Step S52: connect the first separable 3D convolutional layer, the 3D Xception model and the 3D local connection layer to the second splicing fusion layer, connect the first fully connected layer, the second fully connected layer and the third fully connected layer to the third splicing fusion layer, connect the second splicing fusion layer, the second separable 3D convolutional layer, the fourth flattening layer and the third splicing fusion layer in sequence, connect the third splicing fusion layer to the fourth fully connected layer, connect the fourth fully connected layer and the third splicing fusion layer to the fourth splicing fusion layer, and connect the fourth splicing fusion layer to the fifth fully connected layer.

[0111] Embodiment 11: This embodiment is based on the above embodiment, and step S6 specifically includes the following steps:

[0112] Step S61: designate the custom displacement extraction layer, the second custom three-dimensional extreme value pooling layer, and the fourth splicing fusion layer as the input layer of the subway monitoring neural network model, and designate the first fully connected layer, the second fully connected layer, the third fully connected layer, the fourth fully connected layer, and the fifth fully connected layer as the output layer of the subway monitoring neural network model;

[0113] Step S62: Divide the tunnel disease data set into a training set and a test set in a ratio of 8:2, designate the disease point cloud 3D model and the reference point cloud 3D model in the tunnel disease data set as the input of the custom displacement extraction layer, designate the reference point cloud 3D model in the tunnel disease data set as the input of the second custom 3D extreme value pooling layer, designate the relative humidity in the tunnel in the tunnel disease data set as the input of the fourth splicing and fusion layer, designate the depth of the widest crack in the tunnel disease data set as the target output of the first fully connected layer, designate the length of the widest crack and the width of the widest crack in the tunnel disease data set as the target output of the second fully connected layer, designate the segment convergence deformation level in the tunnel disease data set as the target output of the third fully connected layer, designate the tunnel disease cause code as the target output of the fifth fully connected layer, designate the interval tunnel health as the target output of the fourth fully connected layer, train the subway monitoring neural network model, and use Bayesian optimization to adjust the value of the displacement threshold with the goal of minimizing the depth loss value of the widest crack. In this embodiment, Bayesian optimization is implemented using the bayesian-optimization library.

[0114] Embodiment 12: This embodiment is based on the above embodiment, and the process of using the subway monitoring neural network model for monitoring specifically includes the following steps:

[0115] Step M1: collecting the tunnel temperature position point cloud 3D model of the target subway tunnel when it is completed as the target reference point cloud 3D model, collecting the current tunnel temperature position point cloud 3D model of the target subway tunnel as the target current point cloud 3D model, and collecting the current relative humidity in the target subway tunnel;

[0116] Step M2: input the target current point cloud 3D model and the target benchmark point cloud 3D model into the custom displacement extraction layer, input the target benchmark point cloud 3D model into the second custom 3D extreme value pooling layer, input the current relative humidity in the target subway tunnel into the fourth splicing and fusion layer, and the subway monitoring neural network model outputs the interval tunnel health degree, tunnel disease cause code, the length of the widest crack, the width of the widest crack, the depth of the widest crack and the segment convergence deformation level of the target subway tunnel;

[0117] Step M3: adjusting the value of the displacement threshold according to the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack: if the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack exceeds 5%, then using Bayesian optimization to minimize the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack as the goal to redetermine the value of the displacement threshold and return to step M2, otherwise proceed to step M4. In this embodiment, the error between the depth of the widest crack output by the subway monitoring neural network model and the actual depth of the widest crack does not exceed 5%, and then proceed to step M4;

[0118] Step M4: If the health level of the interval tunnel of the target subway tunnel reaches 4 or above, a tunnel disease warning is issued, otherwise no tunnel disease warning is issued.

[0119] Example 13. This example is based on the above example. This example runs in the Windows operating system environment, relies on the Anacond3 environment, and uses the Open3D library to complete point cloud related processing. Keras is used as the algorithm framework of the subway monitoring neural network model. The thermal imager uses The ultrasonic radar is implemented by AK2 USS, the central processing module is implemented by Dell Precision T5820, and the temperature and humidity meter is implemented by KENTA temperature and humidity detector.

[0120] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0121] The present invention and its implementation methods are described above. Such description is not restrictive. What is shown in the accompanying drawings is only one of the implementation methods of the present invention. The actual structure is not limited thereto. In short, if ordinary technicians in this field are inspired by the present invention and do not deviate from the purpose of the invention, they can creatively design structural methods and embodiments similar to the technical scheme, which should all fall within the scope of protection of the present invention.

Claims

1. An artificial intelligence-based subway automatic monitoring and early warning system, characterized by: The subway automatic monitoring and early warning system based on artificial intelligence comprises: a thermometer and hygrometer, an ultrasonic radar, a thermal imager and a central processing module; The thermometer, the thermal imager and the ultrasonic radar are electrically connected to the central processing module respectively; The ultrasonic radar collects the ultrasonic point cloud of the tunnel and uploads it to the central processing module; The thermometer and hygrometer collect the temperature and relative humidity in the tunnel and upload them to the central processing module; The thermal imager collects tunnel infrared images and uploads them to the central processing module; The central processing module corrects the tunnel ultrasonic point cloud according to the temperature in the tunnel and the relative humidity in the tunnel, aligns the tunnel infrared image with the tunnel ultrasonic point cloud, assigns a value to each point in the tunnel ultrasonic point cloud according to the temperature in the tunnel infrared image to generate a temperature position point cloud, and constructs a three-dimensional model of the tunnel temperature position point cloud according to the temperature position point cloud; The central processing module constructs a subway monitoring neural network model and uses the subway monitoring neural network model for monitoring.

2. According to claim 1, an artificial intelligence-based subway automatic monitoring and early warning system is characterized in that: The process of the central processing module correcting the tunnel ultrasonic point cloud according to the temperature in the tunnel and the relative humidity in the tunnel includes the following steps: Step P1: extract the horizontal angle, vertical angle and pre-correction distance corresponding to each point from the tunnel ultrasonic point cloud, and divide the pre-correction distance corresponding to each point by 343 to obtain the sound wave round-trip time corresponding to each point; Step P2: Apply the ultrasonic distance correction formula to each point in the tunnel ultrasonic point cloud to determine the actual distance between the point and the ultrasonic radar; Step P3: Regenerate the tunnel ultrasonic point cloud according to the horizontal angle, vertical angle and actual distance between each point and the ultrasonic radar.

3. The subway automatic monitoring and early warning system based on artificial intelligence according to claim 2 is characterized by: The process of constructing the subway monitoring neural network model by the central processing module specifically includes the following steps: Step S1: construct tunnel disease dataset; Step S2: construct a point cloud feature extraction branch; Step S3: construct a crack feature analysis branch, and connect the point cloud feature extraction branch to the crack feature analysis branch; Step S4: construct a structural feature analysis branch, and connect the point cloud feature extraction branch to the structural feature analysis branch; Step S5: construct a disease diagnosis and early warning branch, and connect the crack feature analysis branch and the structure feature analysis branch to the disease diagnosis and early warning branch; Step S6: Construct a subway monitoring neural network model and use the tunnel disease dataset to train the subway monitoring neural network model.

4. The subway automatic monitoring and early warning system based on artificial intelligence according to claim 3 is characterized by: The step S2 specifically includes the following steps: Step S21: construct a custom displacement extraction layer; Step S22: constructing a first custom three-dimensional extreme value pooling layer and a second custom three-dimensional extreme value pooling layer, wherein the first custom three-dimensional extreme value pooling layer and the second custom three-dimensional extreme value pooling layer are constructed by calling the same custom three-dimensional extreme value pooling layer instance; Step S23: constructing a first three-dimensional convolutional layer, a second three-dimensional convolutional layer and a first splicing fusion layer, wherein the first three-dimensional convolutional layer and the second three-dimensional convolutional layer share weights; Step S24: connect the custom displacement extraction layer to the first custom three-dimensional extreme value pooling layer, connect the first custom three-dimensional extreme value pooling layer to the first three-dimensional convolutional layer, connect the second custom three-dimensional extreme value pooling layer to the second three-dimensional convolutional layer, and connect the first three-dimensional convolutional layer and the second three-dimensional convolutional layer to the first splicing fusion layer.

5. The subway automatic monitoring and early warning system based on artificial intelligence according to claim 4 is characterized by: The hyperparameters of the custom displacement extraction layer include a displacement threshold. The custom displacement extraction layer receives two point cloud 3D models as inputs. The custom displacement extraction layer traverses each point of the first point cloud 3D model of the two point cloud 3D models, and determines whether at least one point can be found in the second point cloud 3D model within an area with the displacement threshold as a radius of the traversed point in the first point cloud 3D model. If not found, the traversed point is marked. After the traversal of the first point cloud 3D model is completed, a new point cloud 3D model is generated as output based on all the marked points.

6. The subway automatic monitoring and early warning system based on artificial intelligence according to claim 5 is characterized by: The custom three-dimensional extreme value pooling layer instance has two three-dimensional pooling windows. The first three-dimensional pooling window of the two three-dimensional pooling windows takes the minimum value within the window to generate a pooling result, and the second three-dimensional pooling window of the two three-dimensional pooling windows takes the maximum value within the window to generate a pooling result. Finally, the pooling result of the first three-dimensional pooling window is superimposed with the pooling result of the second three-dimensional pooling window to generate a dual-channel three-dimensional feature map and then output.

Images