A method for real-time measurement of structural leakage based on multidimensional video analysis

Abstract

This invention discloses a real-time method for detecting structural water leakage based on multidimensional video analysis, and applies this technology to tunnel water leakage detection. The method uses a structural defect inspection robot as its main body, integrating sensors such as cameras, lidar, ultrasonic sensors, and infrared thermal imagers. Utilizing synchronous matching and multidimensional coupled imaging technology, distance, angle, depth, and temperature data are added in real time to the original visible light video to generate a new format of multidimensional video or images with multidimensional information. This new dynamic multidimensional video format can not only sense the location and boundaries of tunnel water leakage, but also, combined with artificial intelligence algorithms, determine the size and severity of the leakage. This method effectively improves the efficiency and accuracy of water leakage detection, and has significant reference value for structural defect detection.

Description

Technical Field

[0001] This invention relates to the field of structural water seepage detection technology, specifically to a method for real-time measurement of structural water leakage based on multi-dimensional video analysis. Background Technology

[0002] Tunnels are often built for transportation, mining, or other reasons. After the tunnel is completed, due to geological reasons, such as the development of joints and fissures or groundwater in the area, groundwater may seep through the fissures; or due to natural weathering or geological disasters that cause structural damage, water may seep or leak. This makes the daily maintenance of tunnels particularly important.

[0003] Currently, routine inspections of tunnels lack regularity and are mostly conducted manually. This method is labor-intensive and often results in delayed detection of defects, which can lead to serious accidents. Existing methods are often limited to single automated detection systems without the cooperation of the detection end, resulting in low overall detection efficiency, poor detection results, and untimely feedback on the specific state of disasters. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a real-time method for detecting structural water leakage based on multi-dimensional video analysis, thereby improving the efficiency and quality of water leakage detection in tunnels.

[0005] To achieve the above objectives, the present invention provides a real-time measurement method for structural leakage based on multidimensional video analysis. This measurement method is applied to a structural leakage detection system, which includes a structural defect detection robot. The structural defect detection robot integrates a camera, a lidar, an infrared thermal imager, and one or more modules for storing or executing the measurement method.

[0006] The specific automatic detection method includes the following steps:

[0007] To determine the location of a disaster, a structural defect detection robot moves along a track to the location of the disaster, collects disaster location information, and determines the time of disaster discovery.

[0008] The multidimensional video-based automatic detection method includes the following steps:

[0009] Based on the data collected in the tunnel, deep learning recognition algorithms were used to train a water seepage detection model and a water leakage detection model, respectively, and the embedded deployment was completed to detect the input images, videos and infrared thermal imaging videos.

[0010] The leak was detected, and the depth of the leak was detected by LiDAR. The LiDAR emitted a laser beam towards the leak and the tunnel wall. Two distances, L1 and L2, were obtained by reflection calculation. The crack depth at the leak in the tunnel is L = L1 - L2, and the distance L2 is the physical location of the leak by the structural defect detection robot.

[0011] The camera in the structural defect detection robot captures images of the leak location, then analyzes the image clarity. If the clarity is sufficient, the next step is performed; otherwise, the images are captured again until the clarity meets the analysis requirements. The outline of the leak is obtained by segmenting the video capture images. Based on the temperature changes inside and outside the leak area presented by infrared thermal imaging, the outline of the leak is determined by the infrared capture images, completing the outline segmentation. In addition, the outline segments of the above two types of images are merged. Based on the shape of the segmented leak outline, it is compared with the identification defect outline set in the self-built database to determine the specific type of leak.

[0012] The pixel area is calculated using Green's formula based on the outline of the leaking water using OpenCV's contourArea function.

[0013]

[0014] Green's formula (1) represents the curve φ L The circulation or line integral over the curve L represents the result of integrating Pdx and Qdy over the curve L, where P and Q are vector fields in the plane of L, and dx and dy represent insignificant path elements; ∫∫ on the right-hand side of the formula D Let represent the surface integral of region D, which represents the result of taking the divergence of the vector field within D, where and Let Q and P represent the partial derivatives of the vector fields Q and P in the x and y directions, respectively.

[0015] The final step of the inspection process involves comprehensively considering the depth of cracks in the water leakage, the actual area of ​​the damage, and the identification results of the type of damage to determine the severity of the damage.

[0016] The disaster location information, disaster discovery time information, and specific severity information of water leakage disaster are stored in the background data for detection and management.

[0017] Based on the aforementioned water leakage detection system, a multi-dimensional video tunnel water leakage measurement method integrates two-dimensional image information, water leakage depth information, and water leakage temperature data. This allows the structural defect detection robot to automatically detect water leakage disasters upon detecting anomalies and issue early warnings through an early warning system. By training different detection models for different water leakage situations, the accuracy of detection is improved. The multi-dimensional video tunnel water leakage measurement method, integrating two-dimensional image information, water leakage depth information, and water leakage temperature data, provides a more detailed assessment of the specific situation of water leakage disasters from multiple dimensions. This allows for more efficient and accurate development of emergency response plans, improving the efficiency and quality of handling water leakage disasters within tunnels. The severity of the damage is determined by comprehensively considering the depth of cracks, the actual area of ​​the damage, and the identification results of the damage type. This invention provides targeted early warnings for different types of water leakage, enhancing the ability to quickly grasp real-time disaster situations. It also reports detailed disaster location and status information, enabling management to comprehensively and accurately understand the disaster status and formulate targeted solutions. Therefore, compared with existing technical solutions, the technical solution of this invention is more comprehensive and accurate, improving the understanding of disasters and enabling more targeted solution development.

[0018] Based on the aforementioned measurement method, the robot integrates a camera, lidar, and infrared thermal imager. Using an artificial neural network, it adds multi-dimensional data such as the tunnel seepage area, seepage crack depth, and seepage area temperature in real time to the existing visible light video, generating a new multi-dimensional video format (e.g., .nmp4) and a new image format (e.g., .njpg) with multi-dimensional information.

[0019] According to another aspect of this application, the depth of water leakage is detected using lidar in multi-dimensional video, providing a reliable standard for determining the severity of water leakage damage. Measuring water leakage from this perspective represents a significant gap in the current technological field.

[0020] According to another aspect of this application, in order to obtain the contour of the leaking water, features are extracted, and the feature map matrix obtained by upsampling and subsampling is given by the following formula:

[0021] The feature map matrix of the same layer in the downsampling layer can be expressed by the following formula:

[0022] g(H g ×W g ×C g (2)

[0023] In formula (2), g refers to pooling; H gThis refers to the height of the feature map after pooling, representing the size of the feature map in the vertical direction; W g The width of the feature map after pooling indicates its size in the horizontal direction; C g The number of channels in the feature map after pooling indicates the feature dimension of each pixel in the feature map.

[0024] The feature map matrix of the same layer in the upsampling layer can be expressed by the following formula:

[0025] f(H f ×W f ×C f (3)

[0026] In formula (3), f refers to the convolution operation, used to extract features from the feature map; H f The height of the input feature map indicates its size in the vertical direction; W f This refers to the width of the input feature map, representing its size in the horizontal direction; C f This represents the feature dimension of each pixel in the input feature map, which can also be understood as the depth of the feature map.

[0027] This method enables it to calculate the contour area more accurately, making the calculated result less different from the actual area and improving the accuracy of its detection.

[0028] According to one aspect of this application: In order to make the contour detection of water leakage more accurate, the contour is segmented in the following manner:

[0029] S1. Obtained by performing a dot product on the feature map matrices of the two inputs in claim 1. and The weights are then summed to obtain the new feature weights, as shown in the following formula:

[0030]

[0031] In formula (4), w s This represents a neuron in the entire neural network; w g This represents a fully connected layer; g represents a pooling operation, used to downsample the feature map and reduce the feature size w. f The weight moments in the convolutional layer are represented by f; f represents the feature obtained after the convolution operation. This represents the result obtained by performing a linear transformation on the pooled features in the fully connected layer. This represents the result of a linear transformation of the features obtained after the convolution operation in the convolutional layer.

[0032] S2. The weighted features are processed by ReLU and then multiplied by a 1×1×1 convolution to obtain the attention intermediate matrix, as shown in the following formula:

[0033] q att =Φ T (σ1(w s (5)

[0034] In formula (5), q att This represents the weighted feature vector calculated through the attention mechanism, where att represents attention. Φ T Φ represents the transpose of the linear transformation matrix that maps input features to the attention space. Φ is this linear transformation matrix. Attention mechanisms typically map input features to a low-dimensional attention space for calculating attention scores and generating weighted feature vectors; σ1 represents the activation function; w s Represents the weighted output of a neuron or group of neurons in a neural network.

[0035] S3.q att The final space-based attention weight matrix is ​​obtained after applying the Sigmoid activation function, as shown in the following formula:

[0036] α=σ2(q att (f;δ att (6)

[0037] In formula (6), α represents the attention score vector calculated through another attention mechanism, used to calculate the weighted sum of the input features; the attention mechanism usually combines the input features and attention scores to calculate the weighted sum, resulting in a weighted feature vector; σ2 represents the activation function; q att δ represents the weighted feature vector calculated through the attention mechanism; f represents the input feature vector; δ att Hyperparameters representing the attention mechanism;

[0038] S4. Multiply the updated attention weight matrix α by the original input feature map f to obtain a more informative output feature map m, as shown in the following formula:

[0039] m=αf

[0040] After binarizing the acquired images, pixel-level segment maps are obtained. Each pixel is then classified, contrast is increased, and Canny edge detection is used to obtain the outline of the leaking water.

[0041] In one aspect of this application, a multi-dimensional video tunnel leakage measurement method is employed, integrating two-dimensional image information of leakage water, leakage depth information, and leakage temperature data. Based on the segmented leakage water outline shape, it is compared with the identification defect outline set in a self-built database to determine the specific type of leakage defect. This provides a more detailed understanding of the leakage disaster situation, enabling more efficient and accurate development of emergency response plans, thus improving the efficiency and quality of handling leakage disasters within tunnels. The severity of the defect is determined by comprehensively considering the crack depth, actual area of ​​the defect, and the defect type identification results.

[0042] In one aspect of this application, the method for determining the location of a disaster is as follows: the actual specific location of the disaster in the tunnel is obtained by the number of rotations of the robot gears as the structural defect detection robot moves on the track, the correspondence between the RFID on the track and the station number, and the reaction of the magnetic strip on the robot.

[0043] In this way, when determining the location, the robot's geographical coordinates can be determined by correlating the RFID tags on the track with the station numbers and the magnetic stripe on the robot. This means that when the robot is deployed in multiple tunnels, this method can effectively pinpoint which tunnel the specific disaster location is in. Then, by using the number of gear rotations and the RFID tags in conjunction, the robot's exact position within the tunnel can be determined, thus providing a precise understanding of the robot's location within the tunnel. Therefore, compared to existing devices, this method of disaster location determination is more accurate, facilitating the development of solutions tailored to the specific external environment.

[0044] The beneficial effects of this invention are reflected in the following aspects: By establishing and training multiple models, different models can be used to detect and handle different disaster situations (seepage, leakage); through a multi-dimensional video tunnel seepage measurement method that integrates two-dimensional image information of seepage, depth information of seepage defects, and temperature data of seepage, the specific situation of seepage disasters can be measured more specifically from multiple dimensions. This allows for more efficient and accurate development of handling plans for dangerous situations, improving the efficiency and quality of handling seepage disasters in tunnels. The severity of the defect is determined by comprehensively considering the depth of cracks in the seepage defects, the actual area of ​​the defects, and the identification results of the defect type. When water seepage occurs, a certain area of ​​the tunnel's inner wall becomes damp. Therefore, during automatic detection, the area of ​​seepage can be used to determine the extent of the damage. This data can be compared with the defect outline data in a self-built database of seepage maps to further identify the specific type of seepage defect. Finally, the detection process integrates the depth of cracks, the actual area of ​​the defect, and the defect type identification results to determine the severity of the defect. This judgment process improves the accuracy and efficiency of detection. The activation of the structural defect detection robot allows for more specific detection of the disaster location and situation, providing a more detailed understanding of the disaster situation. This enables more efficient and accurate development of emergency response plans. Attached Figure Description

[0045] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0046] Figure 1 This is a partial structural block diagram of a structural defect detection robot provided in an embodiment of the present invention;

[0047] Figure 2 This is an overall flowchart of the present invention;

[0048] Figure 3 This is a schematic diagram of the actual output provided in an embodiment of the present invention;

[0049] Figure 4 This is a block diagram of the detection section in one embodiment of the present invention;

[0050] Figure 5 for Figure 2 A schematic diagram of one embodiment of a measurement imaging schematic diagram.

[0051] In the attached diagram, the components include a structural defect detection robot 100, a data processor 101, a memory 102, a memory controller 103, a peripheral interface 104, a lidar 105, an infrared thermal imager 106, and a power supply 107. Detailed Implementation

[0052] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0053] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning understood by those skilled in the art, and numerous specific details are set forth in the following detailed description for the purpose of fully understanding the invention. However, those skilled in the art should understand that the invention can be implemented without these specific details.

[0054] Figures 1-4 The present invention discloses a method for real-time determination of structural leakage based on multi-dimensional video analysis. The detection system includes a structural defect detection robot 100 and a track. The structural defect detection robot 100 includes a sliding connection structure that can be attached to and slide along the track. The structural defect detection robot 100 includes a drive motor, a memory 102, a memory controller 103, one or more data processors 101, a peripheral interface 104, a lidar 105, an infrared thermal imager 106, and one or more modules, programs, or instruction sets for storing or executing automatic detection methods, a data storage unit, and a battery pack or other power supply port for providing power to the above components 107. These components are communicatively connected to a server or control terminal via one or more communication buses, signal lines, or wireless communication modules. The control terminal or server has a display screen that can display the detection terminal's implementation information. It should be understood that the structural defect detection robot 100 is only one example of this application, and the terminal device may have more or fewer components than shown in the figure. Furthermore, each component can be implemented using hardware, software, or a combination of both, including one or more data processors 101 and / or application-specific integrated circuits. The peripheral interface 104 connects to external detection devices, such as lidar, infrared thermal imagers, high-definition cameras, displays, etc., for detecting, transmitting, and displaying data about the external environment.

[0055] The drive motors include motors that generally convert electrical energy into driving torque, as well as servo motors. In some embodiments, servo motors are used to detect and collect data on the position of the structural defect detection robot 100.

[0056] The tracks include suspended tracks and tracks implemented through pre-embedding or other methods. In some embodiments, a suspended track is used in conjunction with the structural defect detection robot 100 to enable the terminal to slide along the track. The track has RFID tags for positioning and corresponding station numbers. The RFID tags and station numbers allow for real-time sensing of the structural defect detection robot 100's position on the track, and the position status can be further clarified by the number of rotations of the gears on the servo motor and the magnetic strip reaction of the robot.

[0057] The memory 102 controller can control access to the memory 102 by devices such as the data processor 101 in the structural defect detection robot 100. The memory 102 contains multiple software programs and / or instruction sets, and can connect inputs or outputs to the memory 102 or the processor through the peripheral interface 104. This enables the structural defect detection robot 100 to perform various detection functions and process data. In some embodiments, the memory 102 stores various detection models trained by image segmentation algorithms. Based on the detection purpose of this application, the detection models in this application are seepage detection models and leakage detection models. Through the detection models, various different disaster situations in the tunnel can be detected. Furthermore, the structural defect detection robot 100 can run the above-mentioned models and implement the detection method.

[0058] When the structural defect detection robot 100 detects an abnormality during routine inspections, it will trigger automatic detection and issue an early warning system. The automatic detection method includes the following steps:

[0059] To determine the location of a basic disaster, the structural defect detection robot moves along a track to the location of the disaster. While moving to the disaster location, it automatically collects the number of gear rotations. Then, based on the reaction between RFID and the magnetic strip on the robot, it accurately collects the disaster location information and simultaneously determines the time of disaster discovery.

[0060] The method for determining the leakage area is as follows:

[0061] Based on the data collected in the tunnel, deep learning recognition algorithms were used to train a water seepage detection model and a water leakage detection model, respectively, and the embedded deployment was completed to detect the input images, videos and infrared thermal imaging videos.

[0062] When water leakage is detected, lidar is used to detect the depth of the leakage. The lidar emits a laser beam towards the water leakage and the tunnel wall. By calculating the reflection, two distances L1 and L2 are obtained. The crack depth at the water leakage point in the tunnel is L = L1 - L2.

[0063] The structural defect detection robot captures video images and infrared photographs, then analyzes the image clarity. If the clarity is sufficient, it proceeds to the next step; otherwise, it re-captures images until the required clarity is met. The robot segments the captured video images to obtain the outline of the leak, and uses infrared thermal imaging to determine the temperature changes inside and outside the leak area. The robot also segments the infrared captured images to identify the leak outline, completing the outline segmentation. Furthermore, the outline segments of the two types of images are merged. Based on the shape of the segmented leak outline, it compares it with the identified defect outlines in a self-built database to determine the specific type of leak.

[0064] The pixel area is calculated using Green's formula based on the outline of the leaking water using OpenCV's contourArea function.

[0065]

[0066] In Green's formula (1), the circulation or line integral on curve L is represented. It represents the result of integrating Pdx and Qdy on curve L, where P and Q are vector fields on the plane of L, and dx and dy represent small path elements; ∫∫ on the right side of the formula D Let represent the double integral of region D, which represents the result of taking the divergence of the vector field within D, where and Let represent the partial derivatives of vector fields Q and P in the x and y directions, respectively.

[0067] The final step of the testing process involves combining the actual area of ​​the disease with the results of disease type identification to determine the severity of the disease.

[0068] The disaster location information, disaster discovery time information, and leakage area information are stored in the background data for detection and management.

[0069] The method for determining the outline of the above-mentioned water leakage includes the following steps:

[0070] like Figures 4-5 As shown, in a specific embodiment of this application, the contour is segmented in the following manner:

[0071] S1. Obtained by performing a dot product on the feature map matrices of the two inputs in claim 1. and The weights are then summed to obtain the new feature weights, as shown in the following formula:

[0072]

[0073] In formula (4), w s This represents a neuron in the entire neural network; w g represents a fully connected layer; g represents a pooling operation used to downsample the feature map and reduce the feature size. f The weight moments in the convolutional layer are represented by f; f represents the features obtained after the convolution operation. This represents the result obtained by performing a linear transformation on the pooled features in the fully connected layer. This represents the result of linear transformation of the features obtained after the convolution operation in the convolutional layer.

[0074] S2. The weighted features are processed by ReLU and then multiplied by a 1×1×1 convolution to obtain the attention intermediate matrix, as shown in the following formula:

[0075] q att =Φ T (σ1(w s (5)

[0076] In formula (5), q att This represents the weighted feature vector calculated through the attention mechanism, where att represents attention. Φ T Φ represents the transpose of the linear transformation matrix that maps input features to the attention space. Φ is this linear transformation matrix. Attention mechanisms typically map input features to a low-dimensional attention space for calculating attention scores and generating weighted feature vectors. σ1 represents the activation function; in a specific embodiment, the sigmoid function is used. This function converts the input real number into a real number between 0 and 1, used to represent the attention score. Specifically, a larger attention score indicates a greater contribution of the input features to the current task; therefore, the input features need to be weighted when calculating the weighted features. s This represents the weighted output of a neuron or group of neurons in a neural network, and its definition is the same as the formula explained earlier. Attention mechanisms typically use the output of the neural network as an input feature to calculate the attention score.

[0077] S3.q att The final space-based attention weight matrix is ​​obtained after applying the Sigmoid activation function, as shown in the following formula:

[0078] α=σ2(q att (f;δ att (6)

[0079] In formula (6), α represents the attention score vector calculated through another attention mechanism, used to calculate the weighted sum of the input features; the attention mechanism usually combines the input features and attention scores to calculate the weighted sum, resulting in a weighted feature vector; σ2 represents the activation function, and in a specific embodiment, the softmax function is selected. This function converts the input real number vector into a real number vector between 0 and 1, where each element represents the attention score at the corresponding position of the input vector. The softmax function is usually used to normalize the attention scores so that the sum of all attention scores is 1; q att δ represents the weighted feature vector calculated through the attention mechanism; f represents the input feature vector; δ att These represent the hyperparameters of the attention mechanism, used to control its behavior. Different attention mechanisms have different hyperparameter settings.

[0080] S4. Multiply the updated attention weight matrix α by the original input feature map f to obtain a more informative output feature map m, as shown in the following formula:

[0081] m=αf

[0082] After binarizing the acquired images, pixel-level segment maps are obtained. Each pixel is then classified, contrast is increased, and Canny edge detection is used to obtain the outline of the leaking water.

[0083] Based on the segmented outline of the leakage water, compare it with the identification disease outline set in the self-built database to determine the specific type of leakage water disease.

[0084] like Figure 5 As shown, the leakage width is calculated according to the proportional relationship based on the pinhole imaging principle as follows:

[0085] f: indicates the focal length of the lens: unit mm

[0086] Z: The distance between the lens and the object, in meters (m).

[0087] w and h represent the width and height of the lens target surface, in mm.

[0088] The resolution of W and H imaging, such as 1080P.

[0089] Based on the principle of triangle similarity, we can roughly estimate the target pixel value at that distance on 1080P, taking the width as an example.

[0090] Let w be the width of the leaking area. r (Unknown), image width W = 1920, focal length f = a (mm), shooting distance Z = b (mm), let the width of the leaking water pixel be x (pixel value).

[0091]

[0092] Where wl represents the width (mm) of the image formed by the leaking water on the target surface.

[0093]

[0094] That is, the width of the leakage is:

[0095]

[0096] In a specific embodiment, an inspection robot can be used as a real-time detection platform, combining detection methods to output three-dimensional or four-dimensional detection results. More specifically, when outputting three-dimensional results, the steps are as follows: A visible light camera and a lidar are simultaneously deployed on the tunnel inspection robot to detect water leakage within the tunnel. A deep learning neural network algorithm is used to segment the water leakage and calculate its area (two-dimensional). The lidar is used to measure and calculate the depth of cracks in the water leakage, and point cloud imaging is used to present the internal spatial state of the water leakage. This is then fused with the water leakage surface image segmented from the visible light video. This adds depth information to the two-dimensional visible light video to create a three-dimensional visual video effect. The output is then presented in new video and image formats such as ".3mp4" and ".3jpg" to provide a multi-dimensional and comprehensive view of the water leakage problems within the tunnel and to facilitate further analysis.

[0097] Furthermore, the steps for outputting a five-dimensional result are as follows: while deploying visible light cameras and lidar on the tunnel inspection robot to detect water leakage in the tunnel, an infrared thermal imager is used to detect the temperature information of the water leakage area. The detection system first uses a deep learning neural network algorithm to segment and calculate the area of ​​the leaking water (two-dimensional). It then uses lidar to measure and calculate the depth of cracks in the leaking water and presents the internal spatial state of the leaking water through point cloud imaging. This is fused with the leaking water surface image segmented from visible light video, thus creating a three-dimensional visual video effect by adding depth information to the two-dimensional visible light video. This three-dimensional visual effect is then fused with the temperature of the entire leaking water area captured by an infrared thermal imager and the crack angle of the leaking water calculated by an ultrasonic sensor. This adds the perception of temperature and crack angle of the leaking water defects to the three-dimensional visual detection, achieving a five-dimensional fusion leaking water detection method through multi-dimensional coupling. Finally, it outputs new video and image formats such as ".5mp4" and ".5jpg" to provide a multi-dimensional and comprehensive display and further analysis of the severity of leaking water defects in the tunnel.

[0098] Through the above embodiments, it is clear that each embodiment utilizes an artificial neural network to add multi-dimensional data such as distance and temperature in real time to the original visible light video to generate a new video format with multi-dimensional information. Based on this multi-dimensional real-time video, infrared data can be used to quickly locate the boundaries of the water leakage image, and LiDAR data can be used to measure the distance between the robot and the physical location of the water leakage. Combined with the improved Unet artificial intelligence algorithm, the dynamic calculation of the structural water leakage area can be achieved. This method effectively improves the efficiency and accuracy of water leakage detection.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for real-time determination of structural leakage water based on multi-dimensional video analysis, characterized in that: The determination method is applied to a structure water leakage detection system, the structure water leakage detection system comprising a structure disease detection robot, the structure disease detection robot being integrated with a camera, a laser radar, an infrared thermal imager and one or more modules for storing or executing the determination method; The structure water leakage real-time determination method based on multi-dimensional video analysis comprises the following steps: Disaster location determination, the structure disease detection robot moves to the disaster location through a track, collects disaster location information, and determines the disaster discovery time; The water leakage detection model and the water leakage detection model are trained respectively using a deep learning identification algorithm based on the data collected by the tunnel, embedded deployment is completed, and the input image video and infrared thermal imaging video are detected; The water leakage disease is found, and the laser radar is used to detect the depth of the disease, the laser radar emits laser to the water leakage disease location, two distances L1 and L2 are obtained through reflection calculation and screening, the crack depth L of the tunnel water leakage is L1-L2, and the distance L2 from the structure disease detection robot to the water leakage physical location; The camera takes a picture of the water leakage location, then analyzes the image clarity of the picture, if the clarity meets the analysis requirement, the next step is performed, if the clarity is not enough, the picture is re-taken until the clarity meets the analysis requirement, the water leakage contour is obtained through image segmentation of the video snapshot, the temperature change inside and outside the water leakage disease range presented by the infrared thermal imaging is analyzed, the infrared thermal imaging is image-captured, the water leakage contour is determined, the contour segmentation is completed, the water leakage disease type is determined by comparing the segmented water leakage contour shape with the identification disease contour set in the self-built library; The pixel area is calculated using the Green formula according to the water leakage contour using the contourArea function of openCV; Green's formula (1) represents the curve The circulation or line integral on curve L represents the coefficient of variation of Pdx and Pdx. The result obtained by integration, where and A vector field on the plane containing L. and Represents tiny path elements; the right side of the formula Let represent the surface integral of region D, which represents the result of taking the divergence of the vector field within D, where and Representing vector fields respectively and Partial derivatives in the x and y directions; The disease severity is determined by comprehensively considering the water leakage disease crack depth, the actual disease area and the disease type identification result in the detection process; The disaster location information, the disaster discovery time information and the water leakage disaster specific severity information are stored in the background data for detection and management.

2. The method of claim 1, wherein, In order to obtain the contour of the water leakage, the features are extracted through the feature map matrix obtained by upsampling and downsampling; The down-sampling layer feature map matrix is expressed by the following formula: wherein, in formula (2) refers to pooling; refers to the height of the pooled feature map, indicating the size of the feature map in the vertical direction; refers to the width of the pooled feature map, indicating the size of the feature map in the horizontal direction; refers to the number of channels of the pooled feature map, indicating the feature dimension of each pixel point of the feature map; The up-sampling layer feature map matrix is expressed by the following formula: wherein, formula (3) refers to a convolution operation used to extract features in the feature map; refers to the height of the input feature map, indicating the size of the feature map in the vertical direction; refers to the width of the input feature map, indicating the size of the feature map in the horizontal direction; refers to the depth of the feature map, indicating the feature dimension of each pixel point of the input feature map.

3. The method of claim 2, wherein The method for segmenting the water leakage contour comprises the following steps: S1. Get by point multiplication of two input feature map matrices in claim 2 and and add to get new feature weights, as shown in the following formula: Wherein, in formula (4) represents a nerve in the entire neural network; represents a fully connected layer; represents a pooling operation, which is used for down-sampling the feature map and reducing the feature; represents a weight matrix in the convolution layer; represents the feature obtained after the convolution operation represents the result obtained by linear transformation of the pooled feature in the fully connected layer; represents the result obtained by linear transformation of the feature obtained after the convolution operation in the convolution layer; S2. The weighted features are activated by a ReLU function and then multiplied by a 1x1x1 convolution to obtain an attention intermediate matrix, as shown in the following formula: wherein, in formula (5) represents a weighted feature vector calculated by an attention mechanism; represents the transpose of a linear transformation matrix for mapping input features to an attention space, is the linear transformation matrix, and the attention mechanism usually maps the input features to a low-dimensional attention space for calculating attention scores and generating a weighted feature vector; represents an activation function; represents the weighted output of a neuron or a group of neurons in a neural network; S3. The final spatial-based attention weight matrix is obtained through a sigmoid activation function, as shown in the following formula: wherein, in formula (6) represents an attention score vector calculated by another attention mechanism, which is used to calculate the weighted sum of the input features; the attention mechanism usually combines the input features and the attention scores to calculate the weighted sum to obtain a weighted feature vector; represents an activation function; represents a weighted feature vector calculated by the attention mechanism; represents an input feature vector; represents a hyperparameter of the attention mechanism; S4. The updated attention weight matrix and the input original feature map Dot product to get more information-rich output feature map m, as shown in the following formula: After image binarization of the collected image, a Segment map at the pixel level is obtained, the category of each pixel is judged, the contrast is increased, and the outline of the leaking water is obtained through Canny edge detection.

4. The method of claim 1, wherein, Through the multi-dimensional video tunnel water leakage determination method integrating the two-dimensional image information of the water leakage, the water leakage disease depth information and the water leakage temperature data, the water leakage disease type is determined by comparing the segmented water leakage contour shape with the identification disease contour set in the self-built library.

5. The method of claim 1, wherein, The method for determining the disaster location is that the actual specific location of the disaster in the tunnel is obtained by the gear rolling number of the structure disease detection robot during movement on the track, the RFID on the track corresponding to the stake number, and the magnetic stripe reaction on the robot.

Images