Industrial visual intelligent detection method for municipal water supply and drainage pipe network

Abstract

The invention relates to the technical field of pipe network detection, and discloses an industrial visual intelligent detection method for a municipal water supply and drainage pipe network, and the method comprises the steps: collecting the multi-modal data of a target object through a collection device; the collected multi-modal data is preprocessed; based on a convolutional neural network, performing attention feature extraction on the preprocessed data collected by the visible light camera to obtain feature data after attention enhancement; based on the preprocessed multi-modal data and the attention-enhanced feature data, performing defect identification and classification in combination with random forest and physical rule verification; outputting a final identification classification result and a defect position; according to the method, attention feature extraction is carried out on the preprocessed data collected by the visible light camera, the defect area features are highlighted, so that background interference is suppressed, and the defect detection accuracy is improved by adaptively adjusting the weight of the feature channel, highlighting the feature channel corresponding to the defect area and suppressing the feature channel of the background area.

Description

Technical Field

[0001] This invention relates to the field of pipeline inspection technology, and more specifically to an industrial visual intelligent inspection method for municipal water supply and drainage pipelines. Background Technology

[0002] Currently, the inspection of municipal water supply and drainage pipe networks mainly relies on manual inspection, which suffers from low inspection efficiency, poor accuracy, and high cost. With the development of artificial intelligence technology, industrial vision intelligent inspection technology has been widely used in various fields, but its application in the field of municipal water supply and drainage pipe network inspection is still immature.

[0003] Manual inspection is inefficient, has a high rate of missed inspections, and cannot detect hidden defects, such as corrosion inside underground pipes. It also relies on single visible light vision, which is poorly adapted to dark, underwater, and oily environments. Feature extraction is easily affected by noise, and traditional machine vision uses edge detection and texture analysis, which have low accuracy in identifying complex defects.

[0004] In view of this, the present invention proposes an industrial vision intelligent inspection method for municipal water supply and drainage pipe networks. The method applies industrial vision intelligent inspection technology, which is robust and accurate, so as to achieve high-precision and high-efficiency identification of defects in municipal water supply and drainage pipe networks. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides an industrial visual intelligent inspection method for municipal water supply and drainage pipe networks to solve the problems existing in the background art.

[0006] This invention provides the following technical solution: an industrial visual intelligent inspection method for municipal water supply and drainage pipe networks, comprising the following steps: Step 1: Collect multimodal data of the target object using acquisition equipment; Step 2: Preprocess the acquired multimodal data; the preprocessing includes noise removal, contrast enhancement, and multimodal alignment. Step 3: Based on the convolutional neural network, perform attention feature extraction on the preprocessed visible light camera data to obtain attention-enhanced feature data; Step 4: Based on the preprocessed multimodal data in Step 2 and the attention-enhanced feature data in Step 3, defect identification and classification are performed by combining random forest and physical rule verification. Step 5: Output the final identification and classification results and the location of the defect.

[0007] Preferably, the acquisition equipment includes a drone, a visible light camera, an infrared thermal imager, and an ultrasonic probe. The target object is the pipe to be inspected. The multimodal data includes pipe surface texture images, pipe surface color images, pipe temperature, pipe material, and pipe wall thickness. The visible light camera is used to acquire pipe surface texture images and pipe surface color images. The infrared thermal imager is used to detect pipe temperature. The ultrasonic probe is used to measure pipe wall thickness. The pipe surface texture images and pipe surface color images are used to obtain changes in pipe surface texture and changes in pipe surface color.

[0008] Preferably, the contrast enhancement specifically involves: applying contrast-limited adaptive histogram equalization to the image acquired by the visible light camera, including image segmentation, obtaining local histograms, obtaining contrast-limiting thresholds, adjusting histograms, histogram equalization, and pixel grayscale value replacement. The image segmentation specifically involves dividing the input image captured by the visible light camera into several sub-blocks of fixed size; the acquisition of the local histogram specifically involves calculating the gray-level histogram for each sub-block and counting the number of pixels at different gray levels within the sub-block; the acquisition of the contrast limit threshold specifically involves calculating the contrast limit threshold based on the histogram of the sub-block; the adjustment of the histogram specifically involves adjusting the number of pixels at a certain gray level in the histogram of each sub-block to the value of the contrast limit threshold if the number of pixels at a certain gray level in the histogram exceeds the contrast limit threshold; the histogram equalization specifically involves performing histogram equalization on each adjusted sub-block to convert the gray-level distribution of the sub-block from a non-uniform distribution to a uniform distribution; and the pixel gray-level value replacement specifically involves replacing the original gray-level value of the pixels in each sub-block with the equalized gray-level value, thereby obtaining an image with enhanced contrast.

[0009] Preferably, the multimodal alignment specifically involves registering the data acquired by the infrared thermal imager, the data acquired by the ultrasonic probe, and the image acquired by the visible light camera using SIFT feature matching, mainly including the following steps: Step 1: Feature point detection; Step 2, Feature point description; Step 3: Feature point matching; Step 4: Remove outliers; Step 5: Transformation model estimation; Step 6: Image registration.

[0010] Preferably, the attention feature extraction of the preprocessed multimodal data in step three specifically involves: The preprocessed data acquired by the visible light camera is input into a convolutional neural network. The data acquired by the visible light camera is recorded as a visible light image, and the feature map is obtained by processing the visible light image through the convolutional layer. The size is , For height, For width, Number of channels; For feature maps Global average pooling is performed to compress the two-dimensional feature map of each channel into a scalar value, expressed by the formula: ;in, Indicates the first The compression value of each channel, Representation of feature map The Middle line, number Column, No. The pixel values ​​of each channel; The squeezed value is processed by two fully connected layers, and the weight of each channel is calculated; of the two fully connected layers, the output dimension of the first fully connected layer is... , For compression ratio, use The function is used as the activation function, and the output dimension of the second fully connected layer is... ,use The function is used as the activation function; Combine the channel weight vector with the feature map Channel-by-channel weighting is performed to obtain the attention-enhanced feature map. .

[0011] Preferably, the calculation of the weight of each channel is expressed by the formula: ;in, Indicates the first The weight vector of each channel, express function, This represents the weight matrix of the second fully connected layer. express function, This represents the weight matrix of the first fully connected layer; The attention-enhanced feature map is expressed by the following formula: ;in, Represents the feature map after attention enhancement The Middle line, number Column, No. The pixel values ​​of each channel.

[0012] Preferably, step four specifically includes: The preprocessed multimodal data, attention-enhanced feature data, and pipeline prior information are used as input features. The random forest algorithm is used to identify and classify the input features. The identification and classification results of the random forest are verified by physical rules. The pipeline prior information consists of pipeline material and corresponding defect information. The defect information includes defect type, defect cause, and defect manifestation. The specific steps of using the random forest algorithm to identify and classify input features are as follows: The input features are fed into a trained random forest algorithm. Each decision tree outputs a recognition and classification result. The recognition and classification results of all decision trees are integrated by majority voting to obtain the final recognition and classification result, which is then output.

[0013] Preferably, the physical rule verification specifically includes: Obtain the final identification and classification results; Extract the physical rules corresponding to the final identification and classification results; Determine whether the final recognition and classification result meets the corresponding physical rules. If it does, the verification passes and the final recognition and classification result is retained. If it does not, the verification fails and the process returns to step three to reacquire the attention-enhanced feature data for re-recognition and classification.

[0014] Preferably, the physical rules are as follows: If the identification classification result is "corrosion", then check whether the pipe wall thickness of the target object is less than 80% of the design value; If the identification and classification result is "leakage", then check whether the infrared temperature difference of the target object is greater than 2℃; If the classification result is "crack", then check whether the visible light crack length of the target object is greater than ten centimeters.

[0015] Preferably, the output of the defect location is specifically as follows: A drone equipped with a positioning system is used to obtain the drone's latitude, longitude, altitude, and speed information in real time. The characteristics of the defect area are matched with the drone's coordinates to obtain the latitude and longitude coordinates of the defect. These coordinates are then mapped onto a GIS map to mark the defect location. Finally, the distance from the starting point to the defect location is calculated based on the drone's flight speed and flight time.

[0016] The technical effects and advantages of this invention are as follows: (1) The present invention has a third step, which is beneficial to extract attention features from the data collected by the preprocessed visible light camera, highlight the features of the defect area, thereby suppressing background interference. By adaptively adjusting the weights of the feature channels, the feature channels corresponding to the defect area are highlighted and the feature channels of the background area are suppressed, thereby improving the accuracy of defect detection. Traditional convolutional neural networks treat all feature channels equally and cannot adaptively highlight the feature channels corresponding to the defect area. However, the present invention uses attention feature extraction, which learns the importance of each channel and adaptively adjusts the weights of the feature channels, thus solving the problem of traditional convolutional neural networks. Performing attention feature extraction separately can make the model more focused on the features of the defect area, further improving the accuracy of defect detection.

[0017] (2) The present invention has a fourth step, which is beneficial to verify the identification and classification results of random forest by physical rules to ensure the rationality of the identification and classification results, thereby avoiding misjudgment by pure data-driven models; and for the first time, visible light camera, infrared thermal imager and ultrasonic probe are combined to cover all dimensions of defect information such as pipe surface, pipe temperature and pipe wall thickness, thereby improving the accuracy of random forest identification and classification results, effectively reducing the false negative rate, and effectively improving the accuracy of identification and classification of complex defects. It is robust and does not require manual inspection. Attached Figure Description

[0018] Figure 1 This is a flowchart of the industrial visual intelligent inspection method for municipal water supply and drainage pipe networks according to the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. In addition, the forms of the various structures described in the following embodiments are merely illustrative. The industrial visual intelligent detection method for municipal water supply and drainage pipe networks involved in the present invention is not limited to the structures described in the following embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figure 1 As shown, this invention provides an industrial visual intelligent inspection method for municipal water supply and drainage pipe networks, comprising the following steps: Step 1: Collect multimodal data of the target object using acquisition devices. The purpose is to obtain complementary information on different defect types. The acquisition devices include, but are not limited to, drones, visible light cameras, infrared thermal imagers, and ultrasonic probes. The target object is the pipe to be inspected. The multimodal data includes, but is not limited to, pipe surface texture images, pipe surface color images, pipe temperature, pipe material, and pipe wall thickness. The visible light camera can be used to acquire pipe surface texture and color images, the infrared thermal imager can be used to detect pipe temperature, and the ultrasonic probe can be used to measure pipe wall thickness. Changes in pipe surface texture and color can be obtained from the pipe surface texture and color images. Step 2: Preprocess the acquired multimodal data; the purpose of which is to remove noise and enhance defect features to adapt to complex environments; the preprocessing includes, but is not limited to, noise removal, contrast enhancement, and multimodal alignment. Step 3: Based on the convolutional neural network, attention feature extraction is performed on the preprocessed visible light camera data to obtain attention-enhanced feature data; the purpose is to highlight the features of the defect area, thereby suppressing background interference. Step 4: Based on the preprocessed multimodal data in Step 2 and the attention-enhanced feature data in Step 3, defect identification and classification are performed by combining random forest and physical rule verification. Step 5: Output the final identification and classification results and the location of the defect.

[0021] In this embodiment, it should be specifically explained that the noise removal specifically refers to: An adaptive Gaussian filter is used, with a filter kernel size of 3×3. The standard deviation of the adaptive Gaussian filter is expressed by the formula: ;in, This represents the standard deviation of the adaptive Gaussian filter. Image size, This indicates the number of pixels in the horizontal direction of the image. This indicates the number of pixels in the vertical direction of the image. Indicates the position of the original image at the th Line number Column pixel values, Indicates the position after median filtering. Line number The pixel values ​​in the column are used to remove salt and pepper noise; The contrast enhancement specifically involves applying contrast-limited adaptive histogram equalization to the image acquired by the visible light camera, including image segmentation, obtaining local histograms, obtaining contrast-limiting thresholds, adjusting histograms, histogram equalization, and pixel grayscale value replacement. The image segmentation specifically involves dividing the input image captured by the visible light camera into several sub-blocks of fixed size, thereby performing local enhancement on the brightness distribution of different regions of the image and avoiding noise amplification or loss of detail caused by global equalization. The process of obtaining the local histogram involves calculating the grayscale histogram for each sub-block and counting the number of pixels at different grayscale levels within the sub-block; the histogram reflects the brightness distribution characteristics of the sub-block. The specific method for obtaining the contrast limit threshold involves calculating the contrast limit threshold based on the histogram of the sub-blocks, expressed by the formula: ;in, Indicates the contrast limit threshold; This represents the adjustment coefficient, and its value range is... ; This represents the average grayscale value of the sub-block; The standard deviation of the sub-block is represented; the contrast limit threshold is used to limit the contrast enhancement of the sub-block to avoid excessive enhancement that would lead to excessive noise. The adjustment of the histogram specifically involves, for each sub-block, if the number of pixels at a certain gray level in the histogram exceeds the contrast limit threshold, adjusting the number of pixels at that gray level to the value of the contrast limit threshold. The adjusted histogram is more uniform, which is beneficial for subsequent histogram equalization. The histogram equalization specifically involves performing histogram equalization on each adjusted sub-block, transforming the grayscale distribution of the sub-block from a non-uniform distribution to a uniform distribution. The histogram equalization is expressed by the formula: ;in, Indicates the first after equalization The pixel value of the gray level; Indicates the th in the adjusted histogram The number of pixels at each gray level; Indicates the total number of pixels in the sub-block; The pixel grayscale value replacement specifically involves replacing the original grayscale value of each pixel within each sub-block with the equalized grayscale value, thereby obtaining an image with enhanced contrast. By using local histogram equalization, the grayscale difference between defective areas such as cracks and leaks and the background is improved, making the defective areas more prominent, which facilitates subsequent feature extraction and detection.

[0022] In this embodiment, it should be specifically explained that the multimodal alignment specifically involves: registering the data acquired by the infrared thermal imager, the data acquired by the ultrasonic probe, and the image acquired by the visible light camera through SIFT feature matching to ensure that defects at the same location are displayed synchronously in the multimodal data; mainly including the following steps: Step 1: Feature point detection; Feature points are detected using the SIFT algorithm and scale-invariant feature points in the image are detected using the difference of Gaussian pyramid. These feature points are robust to rotation, scaling, and noise. Step 2, Feature point description; Generate a descriptor for each feature point. The descriptor contains information such as the feature point's scale, orientation, and gradient. Step 3: Feature point matching; Use either Euclidean distance or Hamming distance to match feature points in different images to find corresponding feature point pairs in different modal images; Step 4: Outlier Removal; Use the random sampling consensus algorithm to remove outliers in the matching. Construct transformation models, such as affine transformation and perspective transformation, through random sampling, and obtain the consistency score of the transformation models. The feature point pairs corresponding to the models with high scores are interior points, and the rest are outliers. Step 5: Transformation model estimation; use interior point estimation to map feature point pairs of different modal images to the same coordinate system. For example, map the feature points of the image acquired by the infrared thermal imager to the coordinate system of the image acquired by the visible light camera through affine transformation. Step 6: Image registration; Based on the transformation model, the image acquired by the infrared thermal imager and the image acquired by the ultrasonic probe are registered in the coordinate system of the image acquired by the visible light camera, thereby obtaining the multimodal aligned image; The purpose is to align the feature spaces of different modal images so that the same defect is in the same position in different modal images, so as to facilitate subsequent feature fusion and defect detection.

[0023] In this embodiment, it should be specifically explained that the attention feature extraction of the preprocessed multimodal data in step three is as follows: The preprocessed data acquired by the visible light camera is input into a convolutional neural network. The data acquired by the visible light camera is recorded as a visible light image, and the feature map is obtained by processing the visible light image through the convolutional layer. The size is , For height, For width, Number of channels; For feature maps Global average pooling is performed to compress the two-dimensional feature map of each channel into a scalar value, expressed by the formula: ;in, Indicates the first The compression value of each channel, Representation of feature map The Middle line, number Column, No. The pixel values ​​of each channel; the purpose of the squeezing operation is to convert local features into global features and capture the global context information of each channel; The squeezed value is processed by two fully connected layers, and the weight of each channel is calculated; of the two fully connected layers, the output dimension of the first fully connected layer is... , For the compression ratio, this embodiment selects... ,use The function is used as the activation function, and the output dimension of the second fully connected layer is... ,use The function serves as the activation function; the calculation of the weight for each channel is expressed by the formula: ;in, Indicates the first The weight vector of each channel, express function, This represents the weight matrix of the second fully connected layer. express function, This represents the weight matrix of the first fully connected layer; the larger the weight vector value of a channel, the greater its contribution to defect detection. Combine the channel weight vector with the feature map Channel-by-channel weighting is performed to obtain the attention-enhanced feature map. This can be expressed as a formula: ;in, Represents the feature map after attention enhancement The Middle line, number Column, No. The pixel values ​​of each channel; attention enhancement is used to highlight the feature channels corresponding to the defective areas, thereby suppressing the feature channels of the background areas; By adaptively adjusting the weights of feature channels, the feature channels corresponding to defective regions are highlighted, while feature channels in background regions are suppressed, thereby improving the accuracy of defect detection. Feature channels corresponding to defective regions have larger weight values ​​and are therefore highlighted, while feature channels corresponding to background regions have smaller weight values ​​and are therefore suppressed. For example, the edge feature channels of crack regions have larger weight values ​​and are highlighted, while the texture feature channels of the background have smaller weight values ​​and are suppressed. Traditional convolutional neural networks treat all feature channels equally and cannot adaptively highlight feature channels corresponding to defective regions. This embodiment uses attention feature extraction, which learns the importance of each channel and adaptively adjusts the weights of feature channels, solving the problem of traditional convolutional neural networks. Performing attention feature extraction separately allows the model to focus more on the features of defective regions, further improving the accuracy of defect detection.

[0024] In this embodiment, it should be specifically explained that step four is as follows: Preprocessed multimodal data, attention-enhanced feature data, and pipeline prior information are used as input features. A random forest algorithm is employed to identify and classify these input features. The classification results of the random forest are then validated using physical rules to ensure their rationality and avoid misjudgments inherent in purely data-driven models. The pipeline prior information comprises the pipeline material and corresponding defect information. This defect information includes, but is not limited to, defect type, defect cause, and defect manifestation. Different pipeline materials produce different defect types and causes; corrosion of concrete pipelines is primarily caused by carbonation, while corrosion of PVC pipelines is mainly caused by chemical erosion. Leakage defects manifest as temperature anomalies in infrared images, while crack defects exhibit linear structures in visible light images. Therefore, incorporating pipeline prior information improves the accuracy of random forest defect identification and classification, while also laying the foundation for subsequent physical rule validation. The specific steps of using the random forest algorithm to identify and classify input features are as follows: The input features are fed into a trained random forest algorithm. Each decision tree outputs a classification result. The classification results of all decision trees are integrated by majority voting to obtain the final classification result, which is then output. The classification result includes, but is not limited to, cracks, leaks, corrosion, and no defects. The majority voting method is as follows: The identification and classification results of each decision tree are counted, and the identification and classification result with the most votes is selected as the final identification and classification result. For example, if the identification and classification results of three decision trees are cracks, then cracks have three votes; if the identification and classification results of two decision trees are leaks, then leaks have two votes. Therefore, cracks have the most votes, and the final identification and classification result is cracks. The training process of the random forest algorithm is as follows: T training subsets with replacement are extracted from the original training set through autopilot sampling, and each training subset is used to train a decision tree; historical multimodal data, attention-enhanced feature data, and pipeline prior information are used as the original training set. For each node split of each decision tree, instead of using all features from the original training set, a subset of features is randomly selected from all features. The number of these subset features is the square root or logarithm of the total number of features, in order to reduce the correlation between trees and enhance the model's generalization ability. The node is divided into two child nodes by selecting the optimal splitting feature from the partial features; the optimal splitting feature can be the feature with the largest information gain or the feature with the smallest Gini index, and the judgment criterion for the optimal splitting feature can be set or adjusted by those skilled in the art. Repeat the above process until the node meets the termination condition, and stop when all decision trees are trained; the termination condition is that the number of node samples is less than a threshold, which is set by those skilled in the art.

[0025] In this embodiment, it should be specifically explained that the physical rule verification is as follows: Obtain the final identification and classification results; Extract the physical rules corresponding to the final identification and classification results; Determine whether the final recognition and classification result meets the corresponding physical rules. If it does, the verification passes and the final recognition and classification result is retained. If it does not, the verification fails and the process returns to step three to reacquire the attention-enhanced feature data for re-recognition and classification. The physical rules are used to determine whether the defect classification conforms to common physical sense, specifically: If the identification classification result is "corrosion", then check whether the pipe wall thickness of the target object is less than 80% of the design value; If the identification and classification result is "leakage", then check whether the infrared temperature difference of the target object is greater than 2℃; If the classification result is "crack", then check whether the visible light crack length of the target object is greater than ten centimeters; The physical rules can be reasonably modified and set by those skilled in the art based on the actual situation of the target object, as long as they conform to common sense in physics; by setting physical rule verification, the final identification and classification results conform to physical principles, effectively avoiding model misjudgment and improving the accuracy of the model's final identification and classification results.

[0026] In this embodiment, it should be specifically noted that the output of the defect location is as follows: By using a drone equipped with a positioning system, the latitude, longitude, altitude, and speed information of the drone are obtained in real time. The characteristics of the defect area are matched with the coordinates of the drone to obtain the latitude and longitude coordinates of the defect. This coordinates are then mapped onto a GIS map to mark the location of the defect. The distance from the starting point to the defect location is then calculated based on the drone's flight speed and flight time. By using a GIS map, the distribution of defects can be displayed intuitively, which makes it easier for pipeline maintenance personnel to formulate maintenance plans.

[0027] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0028] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An industrial visual intelligent inspection method for municipal water supply and drainage pipe networks, characterized in that: Includes the following steps: Step 1: Collect multimodal data of the target object using acquisition equipment; Step 2: Preprocess the collected multimodal data; The preprocessing includes noise removal, contrast enhancement, and multimodal alignment; Step 3: Based on the convolutional neural network, perform attention feature extraction on the preprocessed visible light camera data to obtain attention-enhanced feature data; Step 4: Based on the preprocessed multimodal data in Step 2 and the attention-enhanced feature data in Step 3, defect identification and classification are performed by combining random forest and physical rule verification. Step 5: Output the final identification and classification results and the location of the defect.

2. The industrial visual intelligent inspection method for municipal water supply and drainage pipe networks according to claim 1, characterized in that: The data acquisition equipment includes a drone, a visible light camera, an infrared thermal imager, and an ultrasonic probe. The target object is the pipe to be inspected. The multimodal data includes pipe surface texture images, pipe surface color images, pipe temperature, pipe material, and pipe wall thickness. The visible light camera is used to acquire pipe surface texture images and pipe surface color images. The infrared thermal imager is used to detect pipe temperature, and the ultrasonic probe is used to measure pipe wall thickness. Changes in pipe surface texture and color are obtained through the pipe surface texture images and pipe surface color images.

3. The industrial visual intelligent inspection method for municipal water supply and drainage pipe networks according to claim 2, characterized in that: The contrast enhancement specifically involves applying contrast-limited adaptive histogram equalization to the image acquired by the visible light camera, including image segmentation, obtaining local histograms, obtaining contrast-limiting thresholds, adjusting histograms, histogram equalization, and pixel grayscale value replacement. The image segmentation specifically involves dividing the input image captured by the visible light camera into several sub-blocks of fixed size; the acquisition of the local histogram specifically involves calculating the gray-level histogram for each sub-block and counting the number of pixels at different gray levels within the sub-block; the acquisition of the contrast limit threshold specifically involves calculating the contrast limit threshold based on the histogram of the sub-block; the adjustment of the histogram specifically involves adjusting the number of pixels at a certain gray level in the histogram of each sub-block to the value of the contrast limit threshold if the number of pixels at a certain gray level in the histogram exceeds the contrast limit threshold; the histogram equalization specifically involves performing histogram equalization on each adjusted sub-block to convert the gray-level distribution of the sub-block from a non-uniform distribution to a uniform distribution; and the pixel gray-level value replacement specifically involves replacing the original gray-level value of the pixels in each sub-block with the equalized gray-level value, thereby obtaining an image with enhanced contrast.

4. The industrial visual intelligent inspection method for municipal water supply and drainage pipe networks according to claim 3, characterized in that: The multimodal alignment specifically involves registering data acquired by the infrared thermal imager, data acquired by the ultrasonic probe, and images acquired by the visible light camera using SIFT feature matching. This mainly includes the following steps: Step 1: Feature point detection; Step 2, Feature point description; Step 3: Feature point matching; Step 4: Remove outliers; Step 5: Transformation model estimation; Step 6: Image registration.

5. The industrial visual intelligent inspection method for municipal water supply and drainage pipe networks according to claim 4, characterized in that: The third step of extracting attention features from the preprocessed multimodal data specifically involves: The preprocessed data acquired by the visible light camera is input into a convolutional neural network. The data acquired by the visible light camera is recorded as a visible light image, and the feature map is obtained by processing the visible light image through the convolutional layer. The size is , For height, For width, Number of channels; For feature maps Global average pooling is performed to compress the two-dimensional feature map of each channel into a scalar value, expressed by the formula: ;in, Indicates the first The compression value of each channel, Representation of feature map The Middle line, number Column, No. The pixel values ​​of each channel; The squeezed value is processed by two fully connected layers, and the weight of each channel is calculated; of the two fully connected layers, the output dimension of the first fully connected layer is... , For compression ratio, use The function is used as the activation function, and the output dimension of the second fully connected layer is... ,use The function is used as the activation function; Combine the channel weight vector with the feature map Channel-by-channel weighting is performed to obtain the attention-enhanced feature map. .

6. The industrial visual intelligent inspection method for municipal water supply and drainage pipe networks according to claim 5, characterized in that: The calculation of the weight of each channel is expressed by the formula: ;in, Indicates the first The weight vector of each channel, express function, This represents the weight matrix of the second fully connected layer. express function, This represents the weight matrix of the first fully connected layer; The attention-enhanced feature map is expressed by the following formula: ;in, Represents the feature map after attention enhancement The Middle line, number Column, No. The pixel values ​​of each channel.

7. The industrial visual intelligent inspection method for municipal water supply and drainage pipe networks according to claim 6, characterized in that: Step four specifically involves: The preprocessed multimodal data, attention-enhanced feature data, and pipeline prior information are used as input features. The random forest algorithm is used to identify and classify the input features. The identification and classification results of the random forest are verified by physical rules. The pipeline prior information consists of pipeline material and corresponding defect information. The defect information includes defect type, defect cause, and defect manifestation. The specific steps of using the random forest algorithm to identify and classify input features are as follows: The input features are fed into a trained random forest algorithm. Each decision tree outputs a recognition and classification result. The recognition and classification results of all decision trees are integrated by majority voting to obtain the final recognition and classification result, which is then output.

8. The industrial visual intelligent inspection method for municipal water supply and drainage pipe networks according to claim 7, characterized in that: The physical rule verification specifically involves: Obtain the final identification and classification results; Extract the physical rules corresponding to the final identification and classification results; Determine whether the final recognition and classification result meets the corresponding physical rules. If it does, the verification passes and the final recognition and classification result is retained. If it does not, the verification fails and the process returns to step three to reacquire the attention-enhanced feature data for re-recognition and classification.

9. The industrial visual intelligent inspection method for municipal water supply and drainage pipe networks according to claim 8, characterized in that: The specific physical rules are as follows: If the identification classification result is "corrosion", then check whether the pipe wall thickness of the target object is less than 80% of the design value; If the identification and classification result is "leakage", then check whether the infrared temperature difference of the target object is greater than 2℃; If the identification and classification result is "crack", then check whether the visible light crack length of the target object is greater than ten centimeters.

10. The industrial visual intelligent inspection method for municipal water supply and drainage pipe networks according to claim 9, characterized in that: The specific output of the defect location is as follows: A drone equipped with a positioning system is used to obtain the drone's latitude, longitude, altitude, and speed information in real time. The characteristics of the defect area are matched with the drone's coordinates to obtain the latitude and longitude coordinates of the defect. These coordinates are then mapped onto a GIS map to mark the defect location. Finally, the distance from the starting point to the defect location is calculated based on the drone's flight speed and flight time.

Images