Pipeline abnormal state identification and positioning method and inspection robot thereof

Abstract

The invention discloses a pipeline abnormal state recognition and positioning method and an inspection robot thereof, relates to the technical field of machine vision recognition, and can solve the problems of low recognition precision and high false detection and omission ratio of structural posture abnormity such as falling, inclination and hanging drooping of an underground coal mine pipeline in the prior art. The pipeline abnormal state identification and positioning method comprises the following steps: S1, constructing a global map of a coal mine tunnel, and establishing a coordinate corresponding relation between an image acquisition position and the global map; s2, acquiring a pipeline image in the roadway and position information corresponding to the pipeline image, and performing noise reduction processing on the pipeline image to obtain a to-be-identified image; s3, judging whether the pipeline is in an abnormal posture state or not, and outputting a pipeline abnormity recognition result; and S4, according to the position information corresponding to the pipeline abnormity identification result, completing the positioning of the abnormal pipeline.

Description

Technical Field

[0001] This application relates to the technical field of machine vision recognition, specifically to a method for identifying and locating abnormal pipeline conditions and its inspection robot. Background Technology

[0002] Coal, as a crucial basic energy source in my country's energy structure, is vital for the stable operation of the national economy through its safe and efficient mining. In underground coal mine production systems, water supply and drainage pipelines, sprinkler systems, and other functional pipelines are key infrastructure ensuring safe production and normal equipment operation. Due to the complex and harsh environment of underground coal mines, including unstable geological structures, deformed surrounding rock, high dust concentrations, high humidity, and poor lighting conditions, pipelines are constantly under high load, making them prone to aging, corrosion, and loose connections. These issues, coupled with installation deviations, inadequate maintenance, or improper human operation, can easily lead to various pipeline abnormalities. Among these, pipeline leaks, loose fixing clips or hooks causing pipeline tilting, sag, or even falling to the ground are particularly prominent. These abnormalities not only waste water and energy resources and increase production and operating costs, but may also further induce secondary safety accidents such as water inrush, roof collapse, and falling debris injuries.

[0003] In existing technologies, most methods for detecting anomalies in underground coal mine pipelines focus on typical problems such as pipeline leaks, corrosion, cracks, and surface defects. For example, methods such as negative pressure waves, electromagnetic induction, machine vision, infrared detection, or binocular vision are used to identify and locate pipeline leak points, thermal anomaly areas, or surface defect areas. However, anomalies such as pipeline falls, sags, and attitude deviations are sudden structural anomalies, which are fundamentally different from leaks or surface defects. The former manifests as the entire or partial pipeline detaching from the original fixed structure, resulting in changes in spatial position, attitude tilting, or even falling to the ground. Identifying the former requires not only determining whether the target pipeline is abnormal, but also further analyzing the pipeline's attitude characteristics and its spatial relationship with the ground, support structures, and other surrounding environments. The latter, on the other hand, usually only involves changes in texture, temperature, or morphology in local areas, focusing on surface feature detection.

[0004] Therefore, existing technologies are not adaptable to such structural anomalies, especially in complex working conditions such as dust obstruction, insufficient lighting, cluttered backgrounds, narrow spaces, and equipment vibration in coal mines. These conditions can easily lead to problems such as poor recognition performance, insufficient detection accuracy, and high false positive and false negative rates. Summary of the Invention

[0005] To address this, this application provides a method for identifying and locating abnormal pipeline conditions and its inspection robot, thereby solving the problems of low accuracy and high false positive and false negative rates in the existing technology for identifying structural anomalies such as pipelines falling, tilting, and drooping in coal mines.

[0006] To achieve the above objectives, this application provides the following technical solution: A method for identifying and locating abnormal conditions in pipelines includes the following steps: S1, construct a global map of the coal mine roadway and establish the coordinate correspondence between the image acquisition location and the global map; S2, acquire pipeline images in the tunnel and the corresponding location information of the pipeline images, and perform noise reduction processing on the pipeline images to obtain the image to be identified; S3, input the image to be identified into the neural network model for identification, obtain the pipeline target area, and determine whether the pipeline is in an abnormal posture state based on the geometric features of the pipeline target area and the spatial relationship between the pipeline target area and the ground area, and output the pipeline abnormality identification result. S4. Based on the location information corresponding to the pipeline anomaly identification result, the location of the abnormal pipeline is mapped to the global map to complete the localization of the abnormal pipeline.

[0007] Optionally, in S3, the geometric features of the pipeline target area include at least the inclination angle and aspect ratio, and the spatial relationship features between the pipeline target area and the ground area include at least one of the overlap, horizontal distance and vertical distance.

[0008] Optionally, in S3, geometric features are extracted based on the contour or circumscribed rectangle of the pipeline target area, and spatial relationship features are calculated based on the positional relationship between the pipeline target area and the ground area; the geometric features and spatial relationship features are compared with preset thresholds to determine whether the pipeline is in an abnormal posture state.

[0009] Optionally, in S3, the overlap between the pipeline target area and the ground area is first used to determine whether the pipeline is in a state of falling to the ground; if the pipeline is not in a state of falling to the ground, the inclination angle of the pipeline target area is used to determine whether the pipeline is in a state of hanging or tilting.

[0010] Optionally, the neural network model is a YOLOv8 model, which includes: The backbone network uses MobileNetV4 to perform multi-layer feature extraction on the input image; The feature enhancement module, located in the backbone network, is used to enhance the feature representation of abnormal pipeline targets. Spatial pyramid pooling module is used to expand the receptive field; Feature fusion networks are used to upsample, stitch together, and fuse features from different levels. A multi-scale detection head is used to output the location and category information of abnormal pipeline targets.

[0011] Optionally, in S1, environmental feature data of the coal mine roadway is collected based on lidar. After denoising and registration preprocessing of the environmental feature data, a prior map of the coal mine roadway is constructed using the Cartographer algorithm.

[0012] Optionally, in S1, the local environmental feature data of the current location is acquired in real time by the sensor, the local environmental feature data is matched with the map features in the prior map, and combined with the odometry data for fusion calculation to determine the pose information of the current location in the global map coordinate system.

[0013] Optionally, in S4, when an abnormal pipeline is detected, the pose information corresponding to the time of abnormal identification is mapped to the global map as the associated positioning information of the abnormal pipeline, so as to realize the location marking of the abnormal pipeline in the global map and output alarm information.

[0014] Optionally, in S2, a median filtering method is used to denoise the pipeline image to suppress dust interference and sensor noise, thereby obtaining the image to be identified.

[0015] This application also discloses an inspection robot, including a mobile chassis and an image acquisition device, an environmental sensing device, an inertial measurement device, an alarm device, and a controller mounted on the mobile chassis; The image acquisition device is used to acquire images of pipelines inside the tunnel; Environmental sensing devices are used to acquire environmental characteristic data within coal mine roadways; Inertial measurement units are used to acquire the attitude information of inspection robots; The controller is connected to the image acquisition device, the environmental sensing device, the inertial measurement device, and the alarm device, and is configured to perform the pipeline abnormality identification and location method as described above. The alarm device is used to output alarm information when an abnormal pipeline is detected.

[0016] Compared with the prior art, this application has at least the following beneficial effects: A global map of the coal mine roadway is constructed, and a coordinate correspondence is established between the image acquisition location and the global map. This allows each subsequently acquired pipeline image to be associated with its actual spatial location within the roadway, achieving the conversion from image information to map location information. Subsequently, pipeline images within the roadway and their corresponding location information are acquired, and noise reduction processing is performed on the pipeline images to mitigate the impact of underground coal dust, humidity, insufficient lighting, and sensor noise on image quality, improving the recognizability of pipeline edges, contours, and abnormal areas in the images to be identified. Based on this, the images to be identified are input into a neural network model for recognition, obtaining the pipeline... The target area is analyzed, and combined with the geometric features of the pipeline target area and its spatial relationship with the ground area, it is determined whether the pipeline is in an abnormal posture state. This not only identifies abnormal pipeline targets but also further distinguishes between normal hanging states and abnormal postures such as tilting, drooping, or falling to the ground, improving the targeting and accuracy of anomaly identification. Finally, based on the location information corresponding to the pipeline anomaly identification results, the location of the abnormal pipeline is mapped onto a global map, realizing the positioning and marking of the abnormal pipeline within the roadway. This transforms the anomaly detection results from simple image recognition results into spatial location information that can be used for on-site inspection, verification, and maintenance. Through the coordinated operation of the above steps, this implementation method combines pipeline anomaly identification with roadway map positioning, which not only improves the ability to identify pipeline posture anomalies in the complex environment of underground coal mines but also enables rapid positioning and visual marking of abnormal pipelines. This helps reduce the risk of false detection and missed detection, and improves inspection efficiency and on-site fault handling efficiency. Attached Figure Description

[0017] To more intuitively illustrate the prior art and this application, several exemplary figures are provided below. It should be understood that the specific shapes and structures shown in the figures should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary figures, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).

[0018] Figure 1 A flowchart illustrating a pipeline abnormality identification and location method provided in one embodiment of this application; Figure 2 A YOLOv8 model structure diagram of a pipeline abnormality identification and location method provided in one embodiment of this application; Figure 3 This is a schematic diagram showing the location of abnormal pipelines during an inspection by an inspection robot provided in one embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] refer to Figure 1 This application discloses a method for identifying and locating abnormal conditions in pipelines, comprising the following steps: S1, construct a global map of the coal mine roadway and establish the coordinate correspondence between the image acquisition location and the global map; S2, acquire pipeline images in the tunnel and the corresponding location information of the pipeline images, and perform noise reduction processing on the pipeline images to obtain the image to be identified; S3, input the image to be identified into the neural network model for identification, obtain the pipeline target area, and determine whether the pipeline is in an abnormal posture state based on the geometric features of the pipeline target area and the spatial relationship between the pipeline target area and the ground area, and output the pipeline abnormality identification result. S4. Based on the location information corresponding to the pipeline anomaly identification result, the location of the abnormal pipeline is mapped to the global map to complete the localization of the abnormal pipeline.

[0021] First, a global map of the coal mine roadway is constructed, and a coordinate correspondence is established between the image acquisition location and the global map. This ensures that each subsequent frame of pipeline image can be associated with its actual spatial location within the roadway, achieving the conversion from image information to map location information. Next, pipeline images within the roadway and their corresponding location information are acquired. Noise reduction processing is then applied to the pipeline images to mitigate the impact of underground coal mine dust, humidity, insufficient lighting, and sensor noise on image quality, improving the recognizability of pipeline edges, contours, and abnormal areas in the image to be identified. Based on this, the image to be identified is input into a neural network model for recognition, obtaining the pipeline... The system first identifies the target area of ​​the pipeline and then, combined with the geometric features of the pipeline target area and its spatial relationship with the ground area, determines whether the pipeline is in an abnormal posture state. This not only identifies abnormal pipeline targets but also further distinguishes between normal hanging states and abnormal postures such as tilting, drooping, or falling to the ground, improving the targeting and accuracy of anomaly identification. Finally, based on the location information corresponding to the pipeline anomaly identification results, the location of the abnormal pipeline is mapped onto a global map, realizing the positioning and marking of the abnormal pipeline within the roadway. This transforms the anomaly detection results from simple image recognition results into spatial location information that can be used for on-site inspection, verification, and maintenance. Through the coordinated operation of the above steps, this implementation method combines pipeline anomaly identification with roadway map positioning, which not only improves the ability to identify pipeline posture anomalies in the complex environment of underground coal mines but also enables rapid positioning and visual marking of abnormal pipelines. This helps reduce the risk of false detection and missed detection, and improves inspection efficiency and on-site fault handling efficiency.

[0022] In S3, the geometric features of the pipeline target area include at least the inclination angle and aspect ratio, and the spatial relationship features between the pipeline target area and the ground area include at least one of the overlap, horizontal distance and vertical distance.

[0023] By combining the geometric features of the pipeline target area with its spatial relationship features relative to the ground area into the attitude anomaly determination process, anomaly determination is no longer limited to a single target detection result, but rather further integrates the target shape and spatial position relationship for comprehensive analysis. Specifically, the tilt angle characterizes the degree of deflection of the pipeline target area relative to the normal installation direction (main direction). When the pipeline experiences support loosening, suspension point detachment, or local instability, the main direction of its target area will change significantly relative to the horizontal reference direction, thus reflecting whether the pipeline has tilted, drooping, or other attitude anomalies. The aspect ratio characterizes the projected shape features of the pipeline target area. When the pipeline changes from a normal suspended state to an abnormal drooping or partial fall state, its lateral extension and vertical span in the image usually change, leading to a change in the aspect ratio, thus serving as an auxiliary basis for attitude anomaly determination. Meanwhile, overlap reflects the degree of overlap between the pipeline target area and the ground area in the image, which can be used to determine whether the pipeline is close to the ground or has fallen to the ground; vertical distance characterizes the vertical distance between the pipeline target area and the ground area. When this distance decreases, it indicates that the pipeline is closer to the ground, which can provide a direct criterion for falling or sagging anomalies; horizontal distance characterizes the relative offset between the pipeline target area and the ground area in the lateral direction. Especially in uphill sections, sloping ground areas, or scenarios where the ground boundary is not regularly horizontally distributed, if only the overlap or vertical distance is used for judgment, the accuracy of the judgment is easily affected by changes in the projection shape of the ground area. Combining the horizontal distance can further reflect the lateral offset state of the pipeline relative to the ground reference area, thereby improving the reliability of the judgment of abnormal postures under complex terrain conditions. By simultaneously employing spatial relationship features such as tilt angle, aspect ratio, overlap, horizontal distance, and vertical distance for comprehensive judgment, the abnormal state of pipelines can be characterized from three levels: directional change, shape change, and spatial adjacency relationship. This enhances the ability to identify abnormal postures such as pipeline tilting, hanging, and falling to the ground, and improves the accuracy and environmental adaptability of anomaly judgment.

[0024] In S3, geometric features are extracted based on the contour or circumscribed rectangle of the pipeline target area, and spatial relationship features are calculated based on the positional relationship between the pipeline target area and the ground area. The geometric features and spatial relationship features are compared with preset thresholds to determine whether the pipeline is in an abnormal posture state.

[0025] By extracting geometric features based on the contour or circumscribed rectangle of the pipeline target area, the target recognition results output by the neural network model can be further transformed into quantifiable attitude representation parameters. The contour of the pipeline target area realistically reflects the boundary shape and extension trend of the pipeline in the image, while the circumscribed rectangle facilitates a unified description of the overall shape, orientation, and size of the target area, thus providing a foundation for subsequent extraction of geometric features such as tilt angle and aspect ratio. Simultaneously, spatial relationship features are calculated based on the positional relationship between the pipeline target area and the ground area, characterizing the pipeline's proximity to the ground, projected adjacency, and relative distribution. This allows anomaly detection to not only rely on changes in the pipeline's own morphology but also to comprehensively analyze its external spatial environment. After obtaining the geometric and spatial relationship features, comparing them with preset thresholds distinguishes between normal suspension states and abnormal postures such as tilting, drooping, or falling to the ground: when the extracted feature parameters fall within the normal range, it can be determined that the pipeline has not experienced any posture abnormalities; when the extracted feature parameters exceed the corresponding threshold, it indicates that the pipeline has deviated from the normal state in terms of orientation, shape, or positional relationship with the ground, thus determining that the pipeline is in an abnormal posture state.

[0026] In S3, the overlap between the pipeline target area and the ground area is first used to determine whether the pipeline is in a state of falling to the ground; if the pipeline is not in a state of falling to the ground, the inclination angle of the pipeline target area is used to determine whether the pipeline is in a state of hanging or tilting.

[0027] A hierarchical judgment method is adopted to identify abnormal pipeline posture. First, it is determined whether the pipeline has fallen to the ground. After ruling out the possibility of falling to the ground, it is further determined whether the pipeline is hanging, drooping or tilted. This gives the abnormal posture identification process a clear judgment sequence and clear quantitative basis.

[0028] In some embodiments, the overlap between the pipeline target area and the ground area can be obtained first, and this can be used as a priority criterion for determining the fall status. Let the detection bounding box of the pipeline target area (with coordinates of the upper left corner as...) be... The coordinates of the lower right corner are The coordinates of the upper left corner of the detection frame in the ground area are: The coordinates of the lower right corner are Based on the above coordinate information, the relative positional relationship between the pipeline target area and the ground area in the image can be further obtained, and the horizontal distance, vertical distance, and projection overlap between the two can be calculated. The areas of the pipeline target area and the ground area are respectively: ; ; Based on this, the intersection area can be calculated according to the overlap of the two detection boxes. If the two detection boxes intersect, the area of ​​the intersection is: ; This allows us to determine the overlap between the pipeline target area and the ground area: ; Where S represents the intersection area of ​​the pipeline target area and the ground area, S pipe S represents the area of ​​the target region for the pipeline. ground This represents the area of ​​the ground region. The above calculations can determine the degree of overlap between the pipeline target area and the ground region in the image. When the overlap Z > 0, it indicates that the pipeline target area and the ground region have already overlapped in the image plane, and the pipeline can be preliminarily determined to be in a fallen state. When the overlap Z = 0, it indicates that the pipeline has not yet overlapped with the ground region; in this case, the tilt angle of the pipeline target area is further used to determine whether it is in a suspended, drooping, or tilted state.

[0029] For pipeline targets that have not fallen to the ground, their inclination angle can be calculated using the coordinates of two feature points a1 and a2 on the pipeline target area. These two feature points can be the two endpoints along the main direction of the pipeline target area. In one implementation, it can also be approximately determined by the coordinates of the upper left and lower right corners of the pipeline detection frame. The formula for calculating the inclination angle is: ; in, and These represent the coordinates of the two feature points, This represents the deflection angle of the main direction of the pipeline target area relative to the horizontal reference direction. This formula can quantitatively characterize the degree of attitude change of the pipeline. Under normal laying conditions, the pipeline is usually in a near-horizontal state, with its tilt angle close to 0. When the pipeline experiences issues such as clip detachment, support failure, or partial collapse, its main direction will deviate significantly from the normal range, resulting in a significant increase in the tilt angle.

[0030] In one embodiment, experimental analysis can be performed based on images of normally laid pipelines, images of normal pipelines acquired by camera angle offset, and images of fallen pipelines to obtain the distribution range of tilt angles under normal and abnormal conditions. Experiments show that the tilt angle of normal pipelines in the images typically satisfies 3.7° < θ < 13.6°, while the tilt angle of fallen or obviously abnormal pipelines typically satisfies 16° < θ < 62°. Based on the experimental results, 15°, the midpoint between 13.6° and 16°, can be taken as the threshold for distinguishing between normal and abnormal pipelines. That is, when θ > 15°, the pipeline can be determined to be in a suspended or tilted state; when θ < 15°, the pipeline is determined to be in a suspended or tilted state. At 15°, it can be considered that the pipeline has not experienced any obvious abnormal posture.

[0031] By first determining the ground-fall status based on overlap and then determining the hanging sagging or tilting status based on tilt angle, this method effectively distinguishes between ground-fall anomalies and non-ground-falling attitude deviation anomalies, avoiding confusion between different anomaly states. Furthermore, the use of explicit overlap and tilt angle formulas for quantitative determination enhances the interpretability and feasibility of the attitude anomaly identification process. Simultaneously, because this method further extracts geometric and spatial relationship parameters from the detection results for comprehensive analysis, it more effectively suppresses the risk of misjudgment caused by underground dust, lighting changes, background clutter, and camera perspective shifts compared to identification methods that rely solely on neural network output category results. This improves the accuracy and stability of pipeline anomaly identification in the complex environment of underground coal mines.

[0032] refer to Figure 2 The neural network model is the YOLOv8 model, which includes: The backbone network uses MobileNetV4 to perform multi-layer feature extraction on the input image; The feature enhancement module, located in the backbone network, is used to enhance the feature representation of abnormal pipeline targets. Spatial pyramid pooling module is used to expand the receptive field; Feature fusion networks are used to upsample, stitch together, and fuse features from different levels. A multi-scale detection head is used to output the location and category information of abnormal pipeline targets.

[0033] The neural network model uses an improved YOLOv8 model to detect and identify abnormal pipeline targets in the image to be identified. The improvements are mainly reflected in three aspects: the backbone feature extraction structure, the feature enhancement method, and the feature fusion method. Specifically, in the backbone network, MobileNetV4 is used to replace the conventional, heavier backbone extraction structure. Multi-layer feature extraction is performed on the input image, allowing the model to maintain its ability to extract target features such as pipeline outlines, leaking areas, and fallen areas while effectively reducing the number of model parameters and computational complexity. This improves the model's deployment adaptability and real-time detection capabilities on the local computing platform of the inspection robot. A feature enhancement module is set at the rear of the backbone network to weight and strengthen the features related to abnormal pipelines. This enhances the model's ability to express weak features in slender pipeline targets, low-contrast abnormal areas, and targets with weak features in complex backgrounds, thereby improving the accuracy of identification under conditions of dust interference, insufficient lighting, and cluttered backgrounds in coal mines. Simultaneously, a spatial pyramid pooling module is set up to expand the feature receptive field by introducing pooling operations of different scales. This allows the model to focus on local abnormal details while incorporating broader contextual information, thereby improving the overall perception capability of abnormal targets at different scales, such as fallen pipelines, hanging pipelines, and leaking areas. In the feature fusion network, features at different levels are upsampled, concatenated, and fused. The fused features are further processed using the C2F and ODConv modules shown in the diagram. The C2F module enhances the flow and reuse between shallow detail information and deep semantic information, while the ODConv module adaptively adjusts the convolutional response based on the input features, improving the network's ability to model features of abnormal targets with different shapes and postures, thus enhancing detection robustness under complex conditions. Finally, a multi-scale detection head detects feature maps at different scales, outputting the location and category information of abnormal pipeline targets to accommodate the detection needs of targets of different sizes, such as small leaking areas, medium-scale abnormal areas, and entire fallen pipelines. Through these improvements, the YOLOv8 model not only effectively identifies abnormal pipeline targets but also possesses good lightweight design, real-time performance, and adaptability to complex environments, which is beneficial for improving the accuracy, stability, and engineering practicality of abnormal pipeline identification in coal mines.

[0034] MobileNetV4 is used to perform multi-layer feature extraction on the input image to reduce the number of model parameters and computational complexity while ensuring the ability to represent the features of abnormal pipeline targets. Specifically, let the feature map of the input sample be represented as... ,in, and These represent the height and width of the input image, respectively. The number of channels in the input image is represented by the Stem layer. The backbone network first extracts initial features from the input image through the Stem layer. The Stem layer uses a 3×3 convolution to process the input RGB image, converting the original image into a high-dimensional feature map and completing the initial downsampling.

[0035] Furthermore, the feature extraction units in the backbone network can employ a lightweight bottleneck block structure. The lightweight bottleneck block is preferably constructed using residual connections to enhance feature transfer capability and alleviate gradient decay issues during deep network training. In one implementation, the lightweight bottleneck block can first expand the input feature channels using 1×1 convolutions to enhance feature representation; then, spatial features are extracted using depthwise convolutions, and finally, the output channels are compressed using pointwise convolutions, thereby reducing computational complexity. Here, depthwise convolution refers to performing k×k convolutions independently on each input channel to extract spatial dimension features; pointwise convolution refers to linearly combining and compressing the feature channels after depthwise convolution using 1×1 convolutions to obtain the output features. By combining depthwise convolutions and pointwise convolutions, effective feature extraction can be achieved with lower computational cost.

[0036] In the lightweight bottleneck block, a lightweight attention mechanism can be further introduced to enhance the backbone network's responsiveness to key target regions. Specifically, global pooling and convolution operations can be used to generate channel-dimensional and spatial-dimensional weight information to enhance the features related to abnormal pipelines, thereby improving the model's ability to identify slender pipeline targets, low-contrast abnormal regions, and weak-feature targets in complex backgrounds. Simultaneously, under the condition of matching the number of channels and stride, residual connections can be introduced to directly add the input and output features, preserving shallow detail information and enhancing network training stability.

[0037] Regarding the choice of activation function, the backbone network can adopt the HardSwish activation function to balance the non-linear representation capability of features with the computational efficiency of mobile and edge computing. The HardSwish activation function can be expressed as: This activation method can reduce computational overhead while ensuring model accuracy, thereby improving the model's adaptability to deployment on the local computing platform of the inspection robot.

[0038] The improved YOLOv8 model further incorporates the iEMA attention mechanism to enhance the representation of features related to abnormal pipelines, thereby improving the model's ability to identify slender pipeline targets, low-contrast abnormal regions, and weakly featured targets in complex backgrounds. The iEMA attention mechanism can be viewed as a fusion structure of the iRMB attention unit and the EMA attention branch. The iRMB attention unit is primarily used for local feature extraction and channel transformation, while the EMA attention branch is mainly used to extract global statistical information and generate channel attention weights, thus achieving synergistic enhancement of both local and global features.

[0039] Specifically, let the input feature map be X, and its size be... ,in, This represents the number of channels in the input feature map. Let W and W represent the height and width of the feature map, respectively. First, in the iRMB attention unit, channel compression is performed on the input feature map to reduce computational cost while preserving key feature information. In one implementation, the input channels can be compressed using a 1×1 convolution. Its expression is: ; ; in, To achieve dimensionality reduction of the channel dimension, where B represents the batch size. This indicates the channel compression ratio. Channel compression reduces the amount of redundant background information while preserving key feature channels related to anomalies such as pipe corrosion textures, crack edges, and leak outlines.

[0040] After channel compression, depthwise convolution is further used to extract local spatial features. In one implementation, a 3×3 depthwise convolution can be used to process the compressed feature map, and its expression is as follows: ; ; in, This represents the output feature map after depthwise convolution, used to characterize local spatial features in the input feature map. This represents the kernel weight parameters corresponding to depthwise convolution. This represents the bias parameter corresponding to depthwise convolution; depthwise convolution can extract local spatial detail features of pipeline targets with low computational cost, providing a basis for subsequent abnormal region enhancement.

[0041] Subsequently, 1×1 convolution is used to restore and expand the channels of local features to enhance their expressive power. The expression is as follows: ; ; in, This represents the output feature map after channel restoration and expansion via 1×1 convolution, used to enhance feature representation and restore channel dimensions to match subsequent network processing. This represents the kernel weight parameters corresponding to a 1×1 convolution. This represents the bias parameters corresponding to a 1×1 convolution. This represents the number of channels after expansion. Through the above "compression-extraction-expansion" process, local spatial features can be preserved while reducing computational complexity, and channel-consistent feature input can be provided for subsequent fusion with the EMA branch.

[0042] On the other hand, in the EMA attention branch, channel-level attention weights are generated by performing global statistics on the input feature map, thereby strengthening abnormally relevant channels and suppressing background interference. In one implementation, adaptive average pooling can be performed on the input feature map first to obtain global statistical features. ; in, It aggregates local-global information from the input feature map. Subsequently, two fully connected layers can be used to transform the global statistical features, generating attention weights that match the number of input channels. The expression for this weight is: ; ; ; ; in, It is a channel-level attention weight, which can assign higher weights to feature channels related to pipeline anomalies and lower weights to background or irrelevant feature channels, thereby achieving anomaly feature filtering. This represents the intermediate feature vector after the input features have undergone the first fully connected layer transformation, which is used for preliminary mapping and compression of global statistical information; This represents the weight parameters corresponding to the first fully connected layer; This represents the bias parameters corresponding to the first fully connected layer; This represents the feature vector after the output of the first fully connected layer is further transformed by the second fully connected layer, which is used to recover the channel dimension and generate an attention response that matches the number of input feature channels; This represents the weight parameters corresponding to the second fully connected layer; This represents the bias parameters corresponding to the second fully connected layer.

[0043] In the feature fusion stage, the local enhancement features output by the iRMB attention unit are fused with the channel attention weights obtained from the EMA branch. In one implementation, the local enhancement features are first multiplied element-wise with the channel attention weights to obtain the weighted features. ; Subsequently, the original input features and attention-enhanced features are fused through residual connections, and the output features are further calibrated by convolution and activation functions, as expressed in the following expression: ; in, This is the final output characteristic of the iEMA module. This represents the kernel weight parameters corresponding to a 1×1 convolution. This represents the bias parameter corresponding to a 1×1 convolution. Through the above residual fusion method, on the one hand, the original input features can be preserved, preventing useful information from being weakened during attention weighting; on the other hand, it can utilize... We enhance anomaly-related features and further improve the robustness of feature representation through the HardSwish activation function.

[0044] Furthermore, in one implementation, the input feature map can also extract multi-scale features through parallel 1×1 convolution and 3×3 convolution, and introduce an attention weight generation process in different feature groups to achieve joint modeling of local and global information, thereby improving the model's adaptability to abnormal targets at different scales.

[0045] By introducing the iEMA attention mechanism into a lightweight YOLOv8 model, the model's focus on key areas of abnormal pipelines is enhanced, improving its ability to extract detailed features such as cracks, leaks, fallen edges, and sagging contours. Furthermore, it suppresses irrelevant responses caused by background noise, dust interference, and lighting changes in complex underground coal mine environments, thereby improving the model's accuracy and stability in detecting abnormal pipeline targets. Simultaneously, because the iEMA attention mechanism employs a lightweight design combining channel compression, depthwise convolution, and channel attention, it enhances feature representation capabilities without significantly increasing the overall computational burden of the model, making it suitable for deployment on the local computing platform of inspection robots.

[0046] In some embodiments, when training the YOLOv8 model, the dataset of abnormal pipelines is expanded by augmentation methods. A multi-dimensional data augmentation strategy is adopted to effectively simulate the scene changes that may be encountered in actual downhole inspection: the brightness of the collected abnormal pipeline images is reduced to simulate the dim conditions in the well; then, Gaussian noise with a standard deviation of 15 is added to the collected abnormal pipeline images; finally, the abnormal pipeline images are subjected to horizontal motion blur processing, which can effectively simulate the scene changes that may be encountered in actual downhole inspection robot inspection, and complete the construction of the abnormal pipeline dataset.

[0047] The improved YOLOv8 model's feature fusion network replaces the original standard convolution with a full-dimensional dynamic convolution (ODConv) to enhance the model's adaptive modeling capability for multi-dimensional features of abnormal pipeline targets. Traditional standard convolution uses fixed parameters for feature extraction in the kernel space, input channel, and output channel dimensions, making it difficult to adequately adapt to changes in the shape, scale, pose, and background interference of abnormal pipeline targets in the complex environment of underground coal mines. ODConv, by introducing multi-dimensional attention weights during convolution calculation, can adaptively adjust the convolution response according to different input features, thereby improving the network's feature representation capability and detection robustness for abnormal pipeline targets.

[0048] Let the input feature map be X, which can be compressed and mapped to a length equal to the number of input channels. Corresponding feature vectors are used to reduce the computational complexity of the subsequent dynamic weight generation process. In one implementation, the input features can first be dimensionality-reduced by a fully connected layer, bringing them into a low-dimensional feature space; then, a nonlinear transformation is performed on the mapping result by a modified linear unit to enhance feature representation and suppress invalid responses. Preferably, the ReLU activation function can be used to process the low-dimensional features, setting the negative values ​​to zero, thereby improving the stability of the subsequent dynamic convolutional weight generation process.

[0049] Building upon this, multiple parallel branches generate various attention weights related to the convolution process. Preferably, four parallel branches generate attention parameters corresponding to the convolution kernel spatial dimension, input channel dimension, output channel dimension, and the convolution kernel itself, respectively. Specifically, the convolution kernel spatial dimension attention characterizes the importance of the convolution response at different spatial locations, the input channel dimension attention characterizes the contribution of each input channel feature, the output channel dimension attention characterizes the selective enhancement capability of each output channel, and the convolution kernel attention dynamically weights and combines multiple candidate convolution kernels. Through this multi-dimensional attention mechanism, ODConv can simultaneously adaptively adjust the convolution process across different dimensions, thereby overcoming the limitations of fixed standard convolution parameters and limited expressive power.

[0050] In one implementation, the convolution output of a full-dimensional dynamic convolution can be represented as: ; in, This indicates element-wise multiplication. Represents the convolution operation; for, , , and These represent the attention weights corresponding to the convolution kernel space dimension, input channel dimension, output channel dimension, and convolution kernel dimension, respectively. Multiple candidate convolutional kernels are used. By jointly weighting multiple candidate convolutional kernels and multidimensional attention parameters, the convolutional layer can adaptively select a more suitable feature extraction method based on the current input features.

[0051] By introducing ODConv into the feature fusion network of a lightweight YOLOv8 model, and by upsampling, concatenating, and fusing features at different levels, the fusion features can be further enhanced to respond to key information such as details of abnormal pipelines, contour boundaries, drop patterns, and leakage areas. Especially in the context of dust interference, uneven lighting, complex backgrounds, and significant changes in target pose in underground coal mines, ODConv can dynamically adjust convolution parameters based on input features, making the feature fusion results more targeted and discriminative, thereby improving the ability of subsequent detection heads to locate and classify abnormal pipeline targets.

[0052] By replacing the standard convolutions in the feature fusion network of the lightweight YOLOv8 model with full-dimensional dynamic convolutions, on the one hand, the expressive power of the convolutional layers can be improved without significantly increasing the overall structural complexity of the model, enabling the network to better adapt to changes in the scale, shape, and background conditions of abnormal pipeline targets; on the other hand, by using multi-dimensional attention parameters to finely adjust the convolution process, the abnormality-related features can be enhanced and the background redundant response can be suppressed, thereby improving the detection accuracy, stability, and robustness of the model in the complex environment of underground coal mines.

[0053] The specific process of the horizontal motion blur data augmentation method is as follows: The original image is I, with dimensions M×N (M is the number of rows, N is the number of columns), and pixel coordinates are... grayscale value ; Set the blur direction and blur length; Design a fuzzy kernel: The weights follow a one-dimensional Gaussian distribution. ; in (Core center location) For Gaussian standard deviation, the final result is... Normalization process.

[0054] In one embodiment, the specific process of constructing the dataset is as follows: Abnormal pipe types are categorized as follows: Leakage (warning 1): 457 images; Falling (warning 2): 1238 images, representing cases of pipes falling to the ground and hanging from walls.

[0055] The collected images were classified according to the type of damage to ensure that the images of each category were evenly distributed; normal: 256 images, leaking: 457 images, dropped: 1238 images, total: 1951 images.

[0056] Dataset partitioning: The dataset was divided into training, validation, and test sets in an 8:1:1 ratio. Specific partitioning: Training set: 1568 images (80%), including 204 normal images, 374 images with leaks, and 990 images with drops; Validation set: 171 images (10%), including 20 normal images, 27 images with leaks, 124 images with drops, and 30 images with broken threads or wear; Test set: 212 images (10%), including 22 normal images, 56 images with leaks, and 124 images with drops.

[0057] Perform uniform preprocessing steps on each type of image, including resizing, adjusting all images to the same resolution (e.g., 640×640 pixels), and normalizing pixel values ​​to between 0 and 1.

[0058] Randomly sample from each dataset to ensure image quality and labeling accuracy. Confirm the balance of data distribution to avoid having too many or too few samples of any one class.

[0059] Images of different types are stored in different folders and named according to damage type. Detailed information about the dataset is recorded, including sample source, damage type, quantity, and partitioning, for later use and reference.

[0060] In S1, environmental feature data of the coal mine roadway is collected based on lidar. After denoising and registration preprocessing of the environmental feature data, the prior map of the coal mine roadway is constructed using the Cartographer algorithm.

[0061] First, environmental feature data within the coal mine roadway is collected using LiDAR. This data includes spatial information such as the roadway wall, ground, roof contours, and equipment and support structures distributed along the roadway. Due to the high dust concentration, humidity, strong reflection interference, and sensor noise during robot movement typically present in underground coal mines, the directly collected raw environmental feature data often contains discrete noise points, invalid points, and locally distorted data. Therefore, before using this data for mapping, it undergoes denoising and registration preprocessing to filter out abnormal data introduced by dust interference and sensor noise, and to improve the alignment consistency between environmental data collected at different times. After preprocessing, the Cartographer algorithm is used to map the environmental feature data. The Cartographer algorithm estimates the pose of continuously collected local environmental data and corrects the global trajectory using closed-loop optimization, thereby gradually constructing a priori map reflecting the spatial structure of the coal mine roadway.

[0062] By adopting the above methods, on the one hand, the overall spatial distribution of the tunnel environment can be restored more accurately, providing a unified map basis for location matching and anomaly localization in subsequent inspection processes; on the other hand, by performing denoising and registration preprocessing on the original environmental feature data, the impact of the complex underground environment on the accuracy and stability of mapping can be effectively reduced, map drift and local distortion problems can be reduced, thereby improving the reliability and usability of the prior map, and providing support for mapping the abnormal pipeline identification results to the global spatial location of the tunnel.

[0063] The Cartographer algorithm performs front-end SLAM processing and back-end global optimization on the collected environmental feature data to generate a two-dimensional grid map of the coal mine roadway. Specifically, the Cartographer algorithm first estimates the current pose of the inspection robot in real time based on the matching relationship between continuous laser scan frames and local sub-maps; then, it corrects the accumulated error by combining the back-end pose map optimization with the closed-loop detection results, thereby obtaining a more consistent global map.

[0064] In the front-end SLAM processing, the current laser scan frame can be matched with the local sub-map being constructed to obtain the pose constraints of the current scan frame relative to the local map. The scan frame is then inserted into the corresponding sub-map to update the grid occupancy probability in the sub-map. Through this processing method, the motion trajectory of the inspection robot in the local map can be continuously obtained, and multiple local sub-maps can be gradually formed.

[0065] In the backend optimization process, Cartographer uses a sparse pose graph to optimize the global pose. Specifically, the global pose of the robot corresponding to all laser scan frames and the global pose corresponding to all local subgraphs are associated through pose constraints generated by matching scan frames and subgraphs, and global consistency optimization is performed on this basis. The optimization objective can be expressed as: ; in, This represents the set of global robot poses corresponding to all laser scan frames. Indicates the scan frame number; This represents the set of global poses corresponding to all local subgraphs. Indicates the subgraph number; This represents the relative pose constraint obtained by matching the scan frame with the local subgraph; Information matrix corresponding to constraints; This represents the robust loss function, used to reduce the adverse effects of outlier matching on the overall optimization results.

[0066] In one implementation, the error term can be expressed as: ; in, The residual between the current pose relation and the observation constraints can be expressed as: ; in, This represents the rotation matrix corresponding to the pose of a local subgraph. and These represent the translation components in the local subgraph pose and translation pose, respectively. Rotational angular components in the pose of a local subgraph; The rotational angular component in the laser scanning frame pose. By minimizing the above residuals, better consistency between the poses of each scanning frame and the poses of the sub-image can be achieved in the global coordinate system.

[0067] Furthermore, when the robot returns to a previously traversed area during inspection, the Cartographer algorithm can utilize loop closure detection to generate additional pose constraints and incorporate these constraints into the pose graph optimization to correct for cumulative errors introduced by long-term motion. Through the collaborative processing of front-end scanning matching and back-end loop closure optimization, the global consistency of map construction can be effectively improved, reducing map distortion caused by cumulative mileage errors and local matching drift. This results in a more accurate and stable 2D grid map, providing a reliable map foundation for subsequent robot localization, correlation between abnormal pipeline identification results and map location, and inspection path planning.

[0068] In S1, the local environmental feature data of the current location is acquired in real time by the sensor. The local environmental feature data is matched with the map features in the prior map and combined with the odometry data for fusion calculation to determine the pose information of the current location in the global map coordinate system.

[0069] By acquiring real-time local environmental feature data of its current location through sensors, the inspection robot continuously perceives the environmental information of its current area during movement. This local environmental feature data can include features such as the contours of nearby alleyways, wall boundaries, ground morphology, and local equipment or support structures. Subsequently, the local environmental feature data is matched with map features in a pre-constructed prior map. By comparing the consistency of the current local environment with the corresponding environmental features at each location in the prior map, candidate locations for the inspection robot in the prior map are determined. Based on this, odometry data is combined for fusion calculation. Leveraging the continuity advantage of odometry data in short-term continuous motion estimation, the matching results are compensated and corrected to obtain the pose information of the current location in the global map coordinate system.

[0070] By adopting the above method, on the one hand, it can avoid the position jump or mismatch problem caused by relying solely on the matching of a single local environment, and also reduce the positioning drift caused by relying solely on the cumulative error of the odometer, thereby improving the accuracy and stability of the pose solution; on the other hand, the obtained pose information can serve as the position basis corresponding to the image acquisition time, providing a basic support for establishing the correlation between the subsequent abnormal pipeline identification results and map coordinates, so that the abnormal detection results can not only reflect the abnormal state in the image, but also be further mapped to the corresponding position in the global map of the tunnel.

[0071] In some embodiments, the Cartographer algorithm is used to construct a global map of coal mine roadways. The specific process is as follows: A lidar, IMU, and wheeled odometer are installed on the inspection robot. Each sensor establishes a communication connection with the processing unit (controller) through a bus unit to ensure real-time transmission of multi-source data and ensure that the data transmission delay is no more than 10ms. The processing unit subscribes to LiDAR point cloud data, IMU angular velocity / acceleration data, and odometer displacement data through ROS nodes, and performs preprocessing on the LiDAR point cloud data, including voxel filtering and outlier removal, to reduce noise interference and improve subsequent matching accuracy. Based on IMU data and odometry displacement data, the initial pose of the current laser scanning frame is predicted based on extended Kalman filter. The Fast Correlative Scan Matching algorithm is used to match the current corrected laser point cloud with the grid map being constructed. The matched laser point cloud is then inserted into the current sub-map, and the grid occupancy probability is updated. Based on the optimized sub-map global pose, the raster data of all sub-maps are merged into a global map under a unified coordinate system. The raster probability weighted average method is used during the fusion. Convert the global map to standard formats, including probabilistic raster maps and sparse point cloud maps, to support subsequent localization and path planning modules.

[0072] Based on the preset inspection path planning scheme, real-time motion control commands for the inspection robot are generated. The inspection robot moves along the preset inspection path according to the motion control commands, and collects image data of the pipeline along the way in real time through the pipeline detection sensors on board during the movement. At the same time, the collected pipeline image data is correlated with the robot pose information at the corresponding time in the prior map.

[0073] In S4, when an abnormal pipeline is detected, the pose information corresponding to the time of abnormal identification is mapped to the global map as the associated location information of the abnormal pipeline, so as to mark the location of the abnormal pipeline in the global map and output alarm information.

[0074] Step S4 locates abnormal pipelines based on the global map. The specific process is as follows: The inspection robot performs inspection tasks in the global map. When the camera mounted on the robot detects an abnormal pipeline target, it triggers the abnormal location and alarm process and marks the location information corresponding to the abnormal target in the global map constructed in step S1.

[0075] The sensor synchronization module of the inspection robot is activated to uniformly calibrate the timestamps of the image acquisition device, environmental perception device, and inertial measurement device. In one embodiment, the Time Synchronizer in ROS can be called to synchronize the timestamps of multiple source sensors such as LiDAR and vision camera. By using a preset time synchronization threshold, the time difference between data acquisition of different sensors can be reduced, thereby ensuring the consistency of image data, environmental perception data, and robot pose data in the time dimension when identifying abnormal pipelines and reducing abnormal positioning deviations caused by time delays or asynchrony.

[0076] Based on multi-source data after time alignment, and combined with a preset abnormal pipeline judgment mechanism, pipeline status detection is completed. When an abnormal pipeline is detected, the inspection robot immediately initiates an alarm process. On the one hand, it outputs a warning signal on-site through the robot's audible and visual alarm module. On the other hand, it packages and uploads the abnormality type, detection confidence level, and corresponding raw sensor data to the background monitoring system. At the same time, it calls the robot's pose information corresponding to the time of abnormality identification in real time, and uses the mapping relationship of the global map coordinate system to write the pose information as the associated positioning information of the abnormal pipeline into the global map, realizing the correspondence between the abnormal pipeline and the map location.

[0077] The recorded abnormal information is generated into standardized abnormal information points and displayed explicitly on the global map according to preset visualization marking rules. At the same time, the abnormal layer or abnormal information layer in the global map is updated so that the abnormal information points are spatially associated with the original environmental features of the map. This results in the updated global map containing not only the geometric information of the tunnel environment, but also the abnormal status information of the pipeline, which is convenient for subsequent maintenance tracing, historical abnormality query, and secondary inspection path optimization.

[0078] When the system detects an abnormal pipeline, it acquires the pose information corresponding to the anomaly identification result at that moment and maps this pose information as the associated location information of the abnormal pipeline onto the global map, thereby realizing the transformation from "abnormal target in the image" to "abnormal location on the map". Specifically, since the pipeline image is acquired in real time during the movement of the inspection robot, at the same time the anomaly is identified, the current position and pose information of the inspection robot in the global map coordinate system can be read according to the acquisition time corresponding to the anomaly identification result, and a correspondence can be established between this pose information and the anomaly identification result; then, the correspondence is written to the corresponding position on the global map to form a location marker for the abnormal pipeline on the map. At the same time, after completing the location marking, an alarm message is output, enabling the system not only to visually identify the abnormal pipeline, but also to simultaneously notify on-site operators or the back-end monitoring platform that the abnormal event has occurred.

[0079] The above methods effectively link anomaly identification results with the spatial location of the tunnel, transforming abnormal pipelines from mere image-level identification into queryable, verifiable, and traceable map location information. Furthermore, by synchronizing multi-source sensor data over time, the time deviation between anomaly identification results and robot pose information is effectively reduced, improving the reliability of abnormal pipeline location marking. Additionally, by simultaneously completing map marking, on-site alarms, and background information uploads upon anomaly detection, the response speed and handling efficiency after anomaly detection are significantly improved. This allows maintenance personnel to quickly reach the abnormal area for on-site verification and repair based on the global map, while also providing data support for subsequent anomaly statistical analysis, maintenance tracing, and inspection path optimization.

[0080] refer to Figure 3 The sub-figures illustrate the process by which the inspection robot identifies abnormal pipelines and maps the anomaly information onto the global map at different inspection positions and observation directions. The base map is the global map of the alleyway constructed in step S1. The dark shaded area represents the local observation area or image acquisition area of ​​the inspection robot at the current moment. The arrows represent the mapping relationship between the anomaly identification results and the corresponding positions in the global map, and the black dots represent the marked anomaly information points. As the inspection robot moves along the preset path, when the camera acquires images of abnormal pipelines at different positions and completes anomaly identification, the system reads the robot's pose information corresponding to the anomaly identification moment and writes it into the global map as the associated positioning information of the abnormal pipeline, thereby forming the corresponding anomaly location marker on the map. Figure 3 During the inspection process of the AF, the inspection robot found a total of 5 abnormal pipeline location points (black dots).

[0081] In S2, median filtering is used to denoise the pipeline image to suppress dust interference and sensor noise, thus obtaining the image to be identified.

[0082] Median filtering is used to denoise pipeline images. This method leverages the median filtering's strong ability to suppress impulse noise and discrete noise points, smoothing out abnormal pixels in the acquired pipeline images. Specifically, a preset neighborhood window is selected on the image to be processed. The grayscale values ​​of each pixel within the window are sorted, and the median value is used to replace the value of the center pixel of the window. This reduces random bright spots, dark spots, and local noise interference caused by suspended dust, damp reflections, light fluctuations, and sensor acquisition errors in coal mines.

[0083] In the above processing, median filtering differs from simple mean smoothing. While removing noise, it can better preserve structural information such as pipeline edges, contour directions, and abnormal region boundaries. Therefore, it is more conducive to the subsequent stable identification of pipeline target areas by the neural network model. By using median filtering to denoise pipeline images, on the one hand, it can improve the signal-to-noise ratio of the input image and reduce the adverse effects of the complex downhole environment on image quality; on the other hand, it can enhance the recognizability of pipeline contours and abnormal regions in the image, reduce false detections and missed detections caused by noise in the subsequent identification process, thereby improving the accuracy and stability of abnormal pipeline identification.

[0084] Specifically, let the input image signal be... For a given window size It can sort the pixel grayscale values ​​within a window and take the middle value as the output. Its expression is: ; Based on this, the adaptive median filtering process can be divided into process A and process B.

[0085] In process A, the grayscale values ​​of the pixels within the current window are first counted, and the minimum grayscale value is obtained for each pixel. Median gray level and maximum grayscale value Then determine whether the conditions are met: ; If the above conditions are met, it indicates that the median gray level within the current window is not a significant noise point, and the process can proceed to step B. If the above conditions are not met, it indicates that the pixel gray level distribution within the current window is abnormal, for example, showing... = or = In this case, it can be assumed that the median point within the current window may be affected by noise pollution. It is necessary to appropriately expand the window size and re-search for non-noise statistical features in a larger neighborhood before proceeding to process B.

[0086] In process B, the grayscale value of the center pixel of the current window is further determined. Does the condition meet: ; If the above conditions are met, it means that the current center pixel is not a noise point and its original grayscale value can be retained; if the above conditions are not met, then... = or = In such cases, the center pixel can be determined to be a noise point, and the current window value can be used. The original grayscale value is replaced, thus achieving noise suppression.

[0087] Through the aforementioned A and B two-stage discrimination mechanism, the adaptive median filtering method can not only remove impulse noise and discrete noise points from images, but also dynamically adjust the processing range according to the gray-level characteristics of local image regions. This avoids the problems of insufficient noise removal when the fixed window is too small, or blurred edges when the window is too large. Especially in the underground environment of coal mines, due to factors such as suspended dust, damp reflection, local illumination fluctuations, and sensor errors, random bright spots, dark spots, and local gray-level abrupt changes are prone to appear in images. Adaptive median filtering can effectively reduce the impact of noise on subsequent abnormal pipeline identification while better preserving the detailed features of pipeline contour edges, crack boundaries, and leaking areas. This improves the quality of the image to be identified and the accuracy and stability of subsequent neural network model recognition.

[0088] In some other embodiments, in addition to employing the YOLOv8-based image recognition algorithm, the abnormal pipeline identification algorithm can also be implemented using target segmentation methods. Specifically, the target region in the pipeline image can be finely segmented to obtain the segmented region corresponding to the abnormal pipeline. Based on the segmented region, the center point position, contour boundary, main direction, or shape features can be further extracted to determine whether the pipeline has abnormal postures such as tilting, drooping, or falling to the ground. Compared with the recognition method based on target detection boxes, the target segmentation method can provide more accurate target boundary information, which is beneficial to improving the description accuracy of slender pipeline targets, local fallen areas, and abnormal regions in complex backgrounds, thereby providing more fine-grained image basis for abnormal posture determination.

[0089] In some other embodiments, besides employing a positioning scheme that fuses abnormal pipeline identification results with a priori maps, the abnormal pipeline location method can also utilize a positioning method based on a Wireless Sensor Network (WSN). Specifically, wireless tags can be pre-deployed at key locations along the pipeline, and multiple sensor nodes can be deployed within the tunnel. When a pipeline anomaly is detected, communication is established between the wireless tags and the multiple sensor nodes. The spatial coordinates of the abnormal pipeline are calculated using either the Received Signal Strength Indication (RSSI) algorithm or the Time Difference of Arrival (TDOA) algorithm. The calculated coordinates are then mapped onto a global map of the tunnel, thereby completing the location of the abnormal pipeline. This method allows for direct determination of the spatial location of the abnormal area based on the location of the tag corresponding to the abnormal pipeline, enhancing the directness of anomaly location to a certain extent.

[0090] This application also discloses an inspection robot, including a mobile chassis and an image acquisition device, an environmental sensing device, an inertial measurement device, an alarm device, and a controller mounted on the mobile chassis; The image acquisition device is used to acquire images of pipelines inside the tunnel; Environmental sensing devices are used to acquire environmental characteristic data within coal mine roadways; Inertial measurement units are used to acquire the attitude information of inspection robots; The controller is connected to the image acquisition device, the environmental sensing device, the inertial measurement device, and the alarm device, and is configured to perform the pipeline abnormality identification and location method as described above. The alarm device is used to output alarm information when an abnormal pipeline is detected.

[0091] The inspection robot uses a mobile chassis as its motion platform, enabling autonomous or remote-controlled inspections along coal mine roadways. It integrates an image acquisition device, environmental sensing device, inertial measurement unit, alarm device, and controller, all mounted on the mobile chassis, to form a unified anomaly identification and localization system. The image acquisition device acquires real-time images of pipelines within the roadway, providing image data for subsequent pipeline target identification and posture anomaly determination. The environmental sensing device collects environmental feature data within the coal mine roadway to support global roadway map construction and position matching and localization during the inspection process. The inertial measurement unit acquires the robot's posture information during movement, supporting pose calculation, motion state compensation, and the correlation between anomaly identification results and spatial location. The controller uniformly receives, processes, and schedules the data collected by each device, and executes the aforementioned pipeline anomaly identification and localization methods, enabling the inspection robot to achieve a complete functional closed loop from pipeline image acquisition, environmental sensing, anomaly identification to map localization and alarm output. When an abnormal pipeline is detected, the alarm device outputs an alarm message to alert operators or the monitoring system to promptly address the anomaly.

[0092] Through the above structural configuration, the pipeline anomaly identification function can be integrated with the robot inspection platform, enabling the inspection robot to not only acquire images and environmental information, but also to simultaneously complete anomaly judgment, location marking and alarm prompts during the inspection process. This improves the automation level, identification efficiency and on-site handling timeliness of pipeline anomaly inspection in coal mines, and helps reduce the labor intensity of manual inspection and the risk of missed inspections.

[0093] Among them, the image acquisition device can be an industrial camera, a visible light camera, etc.; the environmental sensing device can be a lidar; the inertial measurement device can be an IMU (inertial measurement unit), which generally includes an accelerometer and a gyroscope; the alarm device can be an audible alarm device, an optical alarm device, or an audible and visual alarm device, such as a buzzer, a warning light, or an integrated audible and visual alarm.

[0094] The data samples were collected from the coal mine site, including images of pipelines in normal operation, to ensure the authenticity and representativeness of the samples.

[0095] The technical features of the above embodiments can be combined in any way (as long as there is no contradiction in the combination of these technical features). For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; these embodiments not explicitly written should also be considered to be within the scope of this specification.

[0096] The present application has been described in a relatively specific and detailed manner above through general descriptions and specific embodiments. It should be understood that, based on the technical concept of the present application, several conventional adjustments or further innovations can be made to these specific embodiments; however, as long as they do not depart from the technical concept of the present application, the technical solutions obtained by these conventional adjustments or further innovations also fall within the protection scope of the claims of the present application.

Claims

1. A method for identifying and locating abnormal conditions in pipelines, characterized in that, Includes the following steps: S1, construct a global map of the coal mine roadway and establish the coordinate correspondence between the image acquisition location and the global map; S2, acquire pipeline images in the tunnel and the corresponding location information of the pipeline images, and perform noise reduction processing on the pipeline images to obtain the image to be identified; S3, input the image to be identified into the neural network model for identification, obtain the pipeline target area, and determine whether the pipeline is in an abnormal posture state based on the geometric features of the pipeline target area and the spatial relationship between the pipeline target area and the ground area, and output the pipeline abnormality identification result. S4. Based on the location information corresponding to the pipeline anomaly identification result, the location of the abnormal pipeline is mapped to the global map to complete the localization of the abnormal pipeline.

2. The pipeline abnormality identification and location method according to claim 1, characterized in that, In S3, the geometric features of the pipeline target area include at least the inclination angle and the aspect ratio. The inclination angle is used to characterize the degree of deflection of the pipeline target area relative to the main direction of the pipeline target area, and the aspect ratio is used to characterize the projected shape features of the pipeline target area. The spatial relationship features between the pipeline target area and the ground area include at least one of the overlap, horizontal distance and vertical distance.

3. The pipeline abnormality identification and location method according to claim 2, characterized in that, In S3, geometric features are extracted based on the contour or circumscribed rectangle of the pipeline target area, and spatial relationship features are calculated based on the positional relationship between the pipeline target area and the ground area. The geometric features and spatial relationship features are compared with preset thresholds to determine whether the pipeline is in an abnormal posture state.

4. The pipeline abnormality identification and location method according to claim 3, characterized in that, In S3, the overlap between the pipeline target area and the ground area is first used to determine whether the pipeline is in a state of falling to the ground; if the pipeline is not in a state of falling to the ground, the inclination angle of the pipeline target area is used to determine whether the pipeline is in a state of hanging or tilting.

5. The pipeline abnormality identification and location method according to claim 1, characterized in that, The neural network model is the YOLOv8 model, which includes: The backbone network uses MobileNetV4 to perform multi-layer feature extraction on the input image; The feature enhancement module, located in the backbone network, is used to enhance the feature representation of abnormal pipeline targets. Spatial pyramid pooling module is used to expand the receptive field; Feature fusion networks are used to upsample, stitch together, and fuse features from different levels. A multi-scale detection head is used to output the location and category information of abnormal pipeline targets.

6. The pipeline abnormality identification and location method according to claim 1, characterized in that, In S1, environmental feature data of the coal mine roadway is collected based on lidar. After denoising and registration preprocessing of the environmental feature data, the prior map of the coal mine roadway is constructed using the Cartographer algorithm.

7. The pipeline abnormality identification and location method according to claim 6, characterized in that, In S1, local environmental feature data of the current location is acquired in real time by the sensor. The local environmental feature data is matched with map features in the prior map and combined with odometry data for fusion calculation. The fusion calculation uses the extended Kalman filter algorithm to determine the pose information of the current location in the global map coordinate system.

8. The pipeline abnormality identification and location method according to claim 7, characterized in that, In S4, when an abnormal pipeline is detected, the pose information corresponding to the time of abnormal identification is mapped to the global map as the associated location information of the abnormal pipeline, so as to mark the location of the abnormal pipeline in the global map and output alarm information.

9. The method for identifying and locating abnormal pipeline conditions according to claim 1, characterized in that, In S2, median filtering is used to denoise the pipeline image to suppress dust interference and sensor noise, thus obtaining the image to be identified.

10. An inspection robot, characterized in that, It includes a mobile chassis and image acquisition devices, environmental sensing devices, inertial measurement devices, alarm devices, and controllers mounted on the mobile chassis; The image acquisition device is used to acquire images of pipelines inside the tunnel; Environmental sensing devices are used to acquire environmental characteristic data within coal mine roadways; Inertial measurement units are used to acquire the attitude information of inspection robots; The controller is connected to the image acquisition device, the environmental sensing device, the inertial measurement device, and the alarm device, and is configured to perform the pipeline abnormality identification and location method according to any one of claims 1-9; The alarm device is used to output alarm information when an abnormal pipeline is detected.

Images