A kind of drifting capsule robot image pipeline defect detection system

Abstract

This invention discloses an image-based pipeline defect detection system for a drifting capsule robot. The system simultaneously acquires image and operational data, calculates physical prior dehazing results based on operational information, and enhances details through data-driven residual learning. Simultaneously, it extracts fog concentration probability maps and defect type weight vectors, generates an adaptive fusion weight matrix through dynamic routing, and performs pixel-level fusion of the physical prior results and data-driven results to obtain the final dehazed image, which is then input into the defect detection module. During the training phase, a joint loss function is constructed using dehazing reconstruction loss, physical consistency loss, and detection task loss to achieve end-to-end joint optimization of each module. This invention can improve image restoration quality and defect detection accuracy under complex water fog interference and is suitable for online pipeline inspection of drifting capsule robots.

Description

Technical Field

[0001] This invention relates to the field of pipeline inspection image processing and defect detection technology, and in particular to a pipeline defect detection system based on images from a drifting capsule robot. Background Technology

[0002] With the continuous expansion of urban underground drainage pipe networks, industrial transmission pipelines and municipal integrated pipe corridors, the use of drifting capsule robots for online inspection of the internal condition of pipelines has become an important technical means. Drifting capsule robots have advantages such as compact structure, ability to move with the flow, and suitability for long-distance continuous operation. They can acquire continuous images of the inside of pipelines with less human intervention, providing a data foundation for the rapid investigation of typical defects such as cracks, ruptures, and deformations.

[0003] However, in actual pipeline environments, drifting capsule robots are easily affected by factors such as high humidity, low illumination, local water mist dispersion, and droplet adhesion, leading to problems such as decreased contrast, color shift, blurred edges, and even loss of local information in the acquired images. Moreover, as the robot drifts with the water flow, it is affected by changes in posture, speed fluctuations, and changes in pipeline curvature, resulting in significant non-uniformity and dynamics in image degradation. In response to these issues, traditional image enhancement methods are difficult to stably restore image details. If degraded images are directly input into a defect detection network, defect features are easily submerged by the fogged background, leading to missed detections, false detections, or positioning errors.

[0004] In existing technologies, some solutions first perform general dehazing preprocessing on the image, and then input the processing result into the object detection network. Although such methods can improve image clarity to a certain extent, the dehazing process usually only focuses on the image reconstruction quality and lacks specific constraints for subsequent defect detection tasks, which can easily lead to overly smoothed dehazing results, weakened defect edges, or the appearance of local artifacts. Other solutions attempt to directly use deep learning object detection networks to identify the original image, but under low illumination, strong fogging, and high noise interference, the stability and robustness of its feature extraction are low.

[0005] In addition, some technical solutions introduce physical models into the defogging process or introduce multi-scale feature fusion mechanisms in the detection stage. However, overall, such technologies lack unified modeling between physical priors and data-driven features, and have limited adaptability to non-uniform fog fields and dynamic disturbances. There is a lack of explicit feedback constraints between the defogging results and the defect detection task, making it difficult to achieve collaborative optimization for defect identification. Furthermore, there is a lack of an integrated detection system covering the entire process for collecting working condition data, image degradation features, and defect semantic information in the scenario of the drifting capsule robot.

[0006] Therefore, for pipeline inspection scenarios using drifting capsule robots, there is an urgent need to build an image-based pipeline defect detection system that can integrate working condition information, physical priors, dynamic routing, and task feedback to improve image availability, defect identification accuracy, and system robustness in complex water mist environments. Summary of the Invention

[0007] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0008] Therefore, to solve the above-mentioned technical problems, the present invention provides the following technical solution: a drifting capsule robot image pipeline defect detection system, including a drifting capsule robot, and an image acquisition unit, a working condition acquisition unit and a main control unit disposed on the drifting capsule robot; the image acquisition unit and the working condition acquisition unit are both communicatively connected to the main control unit.

[0009] The image acquisition unit is used to acquire raw image data inside the pipeline, and the image acquisition unit can be a CMOS industrial camera; the operating condition acquisition unit is used to synchronously acquire operating condition data inside the pipeline; the operating condition acquisition unit can include at least an IMU sensor, a light sensor and a temperature and humidity sensor, and is connected to the main control unit; the operating condition data includes at least temperature value, relative humidity value, three-axis angular velocity value, three-axis attitude angle and drift velocity.

[0010] The main control unit timestamps the original image data and operating condition data based on a unified clock source, and aligns the two according to the timestamps to form a timestamp-aligned drifting capsule robot image and synchronized operating condition data, thereby ensuring that each frame of the image has corresponding operating condition parameters as a source of physical constraints.

[0011] The system also includes a preprocessing module connected to the main control unit, used to preprocess the images of the drifting capsule robot; the preprocessing module sequentially performs size normalization, pixel value normalization and Gaussian noise suppression on the images of the drifting capsule robot to obtain a standard input image; The preprocessing module also extracts low-level visual features from the standard input image, including at least dark channel features, local contrast features, saturation features, and edge gradient features. The above four features are concatenated in the channel dimension to obtain a low-level visual feature matrix.

[0012] The system also includes a physical prior estimation module for receiving standard input images and synchronous operating condition data. This module queries a pre-built temperature and humidity-fog scattering coefficient lookup table based on temperature and relative humidity values, and uses piecewise linear interpolation to obtain a baseline fog scattering coefficient when the temperature and humidity fall between adjacent table nodes. The module calculates a disturbance intensity factor based on triaxial angular velocity and drift velocity, fine-tuning the baseline fog scattering coefficient to obtain the final fog scattering coefficient. Using a pre-calibrated cylindrical pipe geometry model, camera mounting angle, camera focal length parameters, and triaxial attitude angles, the module calculates the scene depth for each pixel by intersecting the ray with the cylindrical surface. Based on the fog scattering coefficient and scene depth, the initial transmittance is calculated. The physical prior estimation module also employs a dark channel prior method, which uses the dark channel statistical information of the standard input image to construct the initial atmospheric light parameters for the RGB three channels. The physical prior estimation module also includes a lightweight physical parameter network for residual correction of the initial physical parameters. The lightweight physical parameter network receives a spliced ​​tensor of the standard input image, initial transmittance, and initial atmospheric light parameters, and outputs transmittance correction and atmospheric light correction for the RGB three channels after multi-layer convolution. The physical prior estimation module adds the initial transmittance and the transmittance correction and performs a cropping operation to obtain the corrected transmittance, and adds the initial atmospheric light parameters and the atmospheric light correction to obtain the corrected atmospheric light parameters. Based on the inverse transform of the atmospheric scattering model, the physical prior estimation module uses the corrected transmittance and the corrected atmospheric light parameters to calculate the physical prior dehazing result pixel by pixel for the RGB three channels.

[0013] The system also includes a data-driven dehazing module for residual learning based on physical priors. The data-driven dehazing module uses a lightweight dehazing network consisting of an encoder, a decoder, and an independent residual compensation branch. The encoder contains multiple downsampling blocks, the decoder is symmetrically arranged with the encoder, and the residual compensation branch is used to compensate for high-frequency details at defect edges. The data-driven dehazing module stitches together the standard input image, the physical prior dehazing result, the corrected transmittance, and the corrected atmospheric light parameters after spatial replication and expansion in the channel dimension to form a multi-channel input tensor. After network processing, it outputs a residual correction map, and adds the residual correction map to the physical prior dehazing result pixel by pixel to obtain the data-driven dehazing result.

[0014] The system also includes a fog concentration estimation submodule, which receives the channel concatenation result of a standard input image and a low-level visual feature matrix; the fog concentration estimation submodule contains a lightweight convolutional neural network, which outputs a fog concentration probability map with the same spatial resolution as the input image after joint learning.

[0015] The system also includes a defect type encoding submodule, which is used to receive data-driven dehazing results. The defect type encoding submodule contains a small convolutional neural network and a global average pooling layer, which are used to extract multi-scale semantic feature maps and generate a three-dimensional defect type weight vector through global average pooling. The three dimensions of the three-dimensional defect type weight vector correspond to three defect types: crack, fracture, and deformation. The values ​​of each dimension represent the sensitivity of the corresponding defect type to edge, contrast, and structural information.

[0016] The system also includes a fog concentration-aware dynamic routing controller, used to receive a fog concentration probability map, corrected transmittance, and defect type weight vector. The fog concentration-aware dynamic routing controller first performs global average pooling, max pooling, and block statistics on the fog concentration probability map and the corrected transmittance, respectively, to obtain fog concentration statistical vectors and transmittance statistical vectors. Then, the fog concentration statistical vector, transmittance statistical vector, and defect type weight vector are concatenated to form a low-dimensional routing input vector, which is input into a lightweight routing network. The lightweight routing network outputs a one-dimensional vector after being mapped by a fully connected layer and is reshaped into a low-resolution fusion weight map. It is then upsampled to the same spatial resolution as the standard input image through interpolation and normalized by applying a Sigmoid activation function to obtain an adaptive fusion weight matrix.

[0017] The system also includes a fusion module, which performs pixel-level weighted fusion of the physical prior dehazing result and the data-driven dehazing result based on an adaptive fusion weight matrix to obtain the final dehazed image. The adaptive fusion weight matrix is ​​a single-channel pixel-by-pixel weight map. For each pixel position, the same weight value is applied to the three RGB channels, thereby achieving regional adaptive fusion based on fog concentration and defect characteristics while maintaining color consistency.

[0018] The system also includes a defect detection module for detecting defects in the final dehazed image; the defect detection module uses an instance segmentation network built on the YOLOv5s backbone network. During the inference phase, the defect detection module outputs defect category labels, bounding box coordinates, and segmentation masks. The defect category labels include at least cracks, fractures, and deformations. During the training phase, the detection task loss output by the defect detection module is not only used to update the parameters of the defect detection module itself, but also passed to the defect type encoding submodule and the fog concentration sensing dynamic routing controller through the gradient backpropagation mechanism. This enables the above modules to adaptively adjust their parameters based on the detection feedback signal, thereby enhancing their responsiveness to defect discrimination features. The detection task loss, together with the defogging reconstruction loss and the physical consistency loss, constitute the total loss function, which is used to perform end-to-end joint optimization of each module of the system.

[0019] The system also includes a joint optimization control module, used to perform end-to-end joint optimization of each trainable module within the system during the training phase. The joint optimization control module calculates a joint loss function consisting of a weighted average of dehazing reconstruction loss, physical consistency loss, and detection task loss. The dehazing reconstruction loss is used to constrain the consistency between the final dehazed image and the clear reference image in pixel space. The physical consistency loss is used to constrain the difference between the fog map reconstructed after substituting the final dehazed image into the atmospheric scattering model and the standard input image. The detection task loss includes classification loss, localization loss, and segmentation mask loss, used to measure the deviation between the detection output and the true label.

[0020] During the training iteration, the joint optimization control module backpropagates the gradient of the joint loss function through the gradient descent optimization algorithm, and synchronously updates the trainable parameters of the lightweight physical parameter network, data-driven dehazing module, fog concentration estimation submodule, defect type encoding submodule, fog concentration-aware dynamic routing controller, and defect detection module in the physical prior estimation module. This achieves coordinated optimization of the dehazing and detection processes, enabling each module to adjust its parameters collaboratively under the constraints of a unified loss function.

[0021] The beneficial effects of this invention are: 1. This invention introduces synchronous operating condition data such as temperature, relative humidity, triaxial angular velocity, triaxial attitude angle and drift velocity to estimate fog scattering coefficient, scene depth and atmospheric light parameters, correct transmittance and generate physical prior defogging results, so that the defogging process is coupled with the actual pipeline environment, which can better adapt to complex operating conditions such as non-uniform fogging, attitude disturbance and local illumination changes, and enhance the physical consistency and interpretability of the defogging results.

[0022] 2. Based on the physical prior dehazing results, this invention introduces a data-driven dehazing module for residual compensation, enabling the model to further restore local textures and defect edges while adhering to atmospheric scattering laws. Compared with existing technologies that rely solely on physical models or deep learning models, this invention reduces the risks of over-enhancement, color distortion, and edge blurring, and is particularly beneficial for revealing small defects such as cracks and fractures.

[0023] 3. This invention generates a fog concentration probability map based on low-level visual features, then generates a defect type weight vector based on the dehazing result, and constructs an adaptive fusion weight matrix based on this vector. This mechanism enables the system to dynamically adjust the contribution ratio of the physical prior branch and the data-driven branch according to the local fog concentration and defect semantic differences in the image, strengthen physical constraints in dense fog areas, and retain more texture information in thin fog or detail areas, thereby improving the problem of insufficient adaptability of fixed fusion strategies.

[0024] 4. The fusion module provided by this invention is based on a single-channel pixel-by-pixel weight map. It performs pixel-by-pixel weighted fusion of the physical prior dehazing result and the data-driven dehazing result, and the weight values ​​are shared among the RGB three channels. This design not only ensures color consistency, but also enables the fusion strategy to be applied precisely to different areas, avoiding color shift and local distortion caused by independent weighting of different channels, and improving the usability of the final dehazed image.

[0025] 5. This invention unifies the dehazing reconstruction loss, physical consistency loss, and detection task loss into the same optimization framework. During the training phase, the detection task loss not only affects the detection network itself, but also influences the trainable parameters of multiple modules such as physical prior estimation, data-driven dehazing, fog concentration estimation, defect type encoding, and dynamic routing through gradient backpropagation. This makes the dehazing process directly constrained by the detection target, prompting the system to prioritize the retention of image features that have a discriminative effect on defect identification, thereby avoiding the problem of good dehazing effect but poor detection effect in traditional two-stage methods. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a system architecture diagram of the present invention.

[0027] Figure 2 This is an overall workflow diagram of Embodiment 2 of the present invention.

[0028] Figure 3 This is a flowchart of step S3 in Embodiment 2 of the present invention. Figure 4 This is a flowchart of step S7 in Embodiment 2 of the present invention. Figure 5 This is a flowchart of step S10 in Embodiment 2 of the present invention. Detailed Implementation

[0029] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0030] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0031] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention; the phrase "in an embodiment" appearing in different places in this specification does not all refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0032] Example 1 Reference Figure 1 The first embodiment of the present invention provides a drifting capsule robot image pipeline defect detection system, including a drifting capsule robot, and an image acquisition unit, a working condition acquisition unit and a main control unit disposed on the drifting capsule robot; the image acquisition unit and the working condition acquisition unit are both communicatively connected to the main control unit.

[0033] The image acquisition unit is used to acquire raw image data inside the pipeline, and the image acquisition unit can be a CMOS industrial camera; the operating condition acquisition unit is used to synchronously acquire operating condition data inside the pipeline; the operating condition acquisition unit can include at least an IMU sensor, a light sensor and a temperature and humidity sensor, and is connected to the main control unit; the operating condition data includes at least temperature value, relative humidity value, three-axis angular velocity value, three-axis attitude angle and drift velocity.

[0034] The main control unit timestamps the original image data and operating condition data based on a unified clock source, and aligns the two according to the timestamps to form a timestamp-aligned drifting capsule robot image and synchronized operating condition data, thereby ensuring that each frame of the image has corresponding operating condition parameters as a source of physical constraints.

[0035] The system also includes a preprocessing module connected to the main control unit, used to preprocess the images of the drifting capsule robot; the preprocessing module sequentially performs size normalization, pixel value normalization and Gaussian noise suppression on the images of the drifting capsule robot to obtain a standard input image; The preprocessing module also extracts low-level visual features from the standard input image, including at least dark channel features, local contrast features, saturation features, and edge gradient features. The above four features are concatenated in the channel dimension to obtain a low-level visual feature matrix.

[0036] The system also includes a physical prior estimation module for receiving standard input images and synchronous operating condition data. This module queries a pre-built temperature and humidity-fog scattering coefficient lookup table based on temperature and relative humidity values, and uses piecewise linear interpolation to obtain a baseline fog scattering coefficient when the temperature and humidity fall between adjacent table nodes. The module calculates a disturbance intensity factor based on triaxial angular velocity and drift velocity, fine-tuning the baseline fog scattering coefficient to obtain the final fog scattering coefficient. Using a pre-calibrated cylindrical pipe geometry model, camera mounting angle, camera focal length parameters, and triaxial attitude angles, the module calculates the scene depth for each pixel by intersecting the ray with the cylindrical surface. Based on the fog scattering coefficient and scene depth, the initial transmittance is calculated. The physical prior estimation module also employs a dark channel prior method, which uses the dark channel statistical information of the standard input image to construct the initial atmospheric light parameters for the RGB three channels. The physical prior estimation module also includes a lightweight physical parameter network for residual correction of the initial physical parameters. The lightweight physical parameter network receives a spliced ​​tensor of the standard input image, initial transmittance, and initial atmospheric light parameters, and outputs transmittance correction and atmospheric light correction for the RGB three channels after multi-layer convolution. The physical prior estimation module adds the initial transmittance and the transmittance correction and performs a cropping operation to obtain the corrected transmittance, and adds the initial atmospheric light parameters and the atmospheric light correction to obtain the corrected atmospheric light parameters. Based on the inverse transform of the atmospheric scattering model, the physical prior estimation module uses the corrected transmittance and the corrected atmospheric light parameters to calculate the physical prior dehazing result pixel by pixel for the RGB three channels.

[0037] The system also includes a data-driven dehazing module for residual learning based on physical priors. The data-driven dehazing module uses a lightweight dehazing network consisting of an encoder, a decoder, and an independent residual compensation branch. The encoder contains multiple downsampling blocks, the decoder is symmetrically arranged with the encoder, and the residual compensation branch is used to compensate for high-frequency details at defect edges. The data-driven dehazing module stitches together the standard input image, the physical prior dehazing result, the corrected transmittance, and the corrected atmospheric light parameters after spatial replication and expansion in the channel dimension to form a multi-channel input tensor. After network processing, it outputs a residual correction map, and adds the residual correction map to the physical prior dehazing result pixel by pixel to obtain the data-driven dehazing result.

[0038] The system also includes a fog concentration estimation submodule, which receives the channel concatenation result of a standard input image and a low-level visual feature matrix; the fog concentration estimation submodule contains a lightweight convolutional neural network, which outputs a fog concentration probability map with the same spatial resolution as the input image after joint learning.

[0039] The system also includes a defect type encoding submodule, which is used to receive data-driven dehazing results. The defect type encoding submodule contains a small convolutional neural network and a global average pooling layer, which are used to extract multi-scale semantic feature maps and generate a three-dimensional defect type weight vector through global average pooling. The three dimensions of the three-dimensional defect type weight vector correspond to three defect types: crack, fracture, and deformation. The values ​​of each dimension represent the sensitivity of the corresponding defect type to edge, contrast, and structural information.

[0040] The system also includes a fog concentration-aware dynamic routing controller, used to receive a fog concentration probability map, corrected transmittance, and defect type weight vector. The fog concentration-aware dynamic routing controller first performs global average pooling, max pooling, and block statistics on the fog concentration probability map and the corrected transmittance, respectively, to obtain fog concentration statistical vectors and transmittance statistical vectors. Then, the fog concentration statistical vector, transmittance statistical vector, and defect type weight vector are concatenated to form a low-dimensional routing input vector, which is input into a lightweight routing network. The lightweight routing network outputs a one-dimensional vector after being mapped by a fully connected layer and is reshaped into a low-resolution fusion weight map. It is then upsampled to the same spatial resolution as the standard input image through interpolation and normalized by applying a Sigmoid activation function to obtain an adaptive fusion weight matrix.

[0041] The system also includes a fusion module, which performs pixel-level weighted fusion of the physical prior dehazing result and the data-driven dehazing result based on an adaptive fusion weight matrix to obtain the final dehazed image. The adaptive fusion weight matrix is ​​a single-channel pixel-by-pixel weight map. For each pixel position, the same weight value is applied to the three RGB channels, thereby achieving regional adaptive fusion based on fog concentration and defect characteristics while maintaining color consistency.

[0042] The system also includes a defect detection module for detecting defects in the final dehazed image; the defect detection module uses an instance segmentation network built on the YOLOv5s backbone network. During the inference phase, the defect detection module outputs defect category labels, bounding box coordinates, and segmentation masks. The defect category labels include at least cracks, fractures, and deformations. During the training phase, the detection task loss output by the defect detection module is not only used to update the parameters of the defect detection module itself, but also passed to the defect type encoding submodule and the fog concentration sensing dynamic routing controller through the gradient backpropagation mechanism. This enables the above modules to adaptively adjust their parameters based on the detection feedback signal, thereby enhancing their responsiveness to defect discrimination features. The detection task loss, together with the defogging reconstruction loss and the physical consistency loss, constitute the total loss function, which is used to perform end-to-end joint optimization of each module of the system.

[0043] The system also includes a joint optimization control module, used to perform end-to-end joint optimization of each trainable module within the system during the training phase. The joint optimization control module calculates a joint loss function consisting of a weighted average of dehazing reconstruction loss, physical consistency loss, and detection task loss. The dehazing reconstruction loss is used to constrain the consistency between the final dehazed image and the clear reference image in pixel space. The physical consistency loss is used to constrain the difference between the fog map reconstructed after substituting the final dehazed image into the atmospheric scattering model and the standard input image. The detection task loss includes classification loss, localization loss, and segmentation mask loss, used to measure the deviation between the detection output and the true label.

[0044] During the training iteration, the joint optimization control module backpropagates the gradient of the joint loss function through the gradient descent optimization algorithm, and synchronously updates the trainable parameters of the lightweight physical parameter network, data-driven dehazing module, fog concentration estimation submodule, defect type encoding submodule, fog concentration-aware dynamic routing controller, and defect detection module in the physical prior estimation module. This achieves coordinated optimization of the dehazing and detection processes, enabling each module to adjust its parameters collaboratively under the constraints of a unified loss function.

[0045] This invention forms a collaborative closed loop encompassing physical modeling, residual enhancement, dynamic routing, fusion control, detection feedback, and joint optimization. Operating data directly participates in the physical prior solution, while the fog concentration probability map and defect type weight vector jointly drive dynamic routing. Finally, the defogging image is subject to feedback constraints from the detection task loss. There are clear data dependencies, functional couplings, and gradient closed-loop relationships among the modules, enabling more stable image enhancement, more accurate defect localization, and more robust system deployment in complex water mist pipeline environments.

[0046] Example 2 Reference Figures 2-5 This is the second embodiment of the present invention, which differs from the first embodiment in that it also provides specific implementation steps for the above system: Step S1: Acquire images and synchronized operating data of the drifting capsule robot; After the system starts up, the data acquisition module actively acquires the raw data, as follows: A1. Raw image data of the inside of the pipeline is acquired by an image acquisition unit installed on the drifting capsule robot. The image acquisition unit is communicatively connected to the main control unit and is preferably a CMOS industrial camera.

[0047] A2. The operating condition data is collected through the operating condition acquisition unit, which is connected to the main control unit and includes at least an IMU sensor, a light sensor, and a temperature and humidity sensor.

[0048] A3. The main control unit timestamps the image data and operating condition data based on a unified clock source to form a synchronous record; the operating condition data includes at least: three-axis acceleration, three-axis angular velocity, three-axis attitude angle, ambient illuminance value, temperature value, relative humidity value, and drift velocity calculated by IMU integration and estimated by combining continuous frame optical flow fusion. The fusion process is used to reduce the cumulative drift error of the IMU; wherein, the optical flow is estimated by adjacent frame images.

[0049] A4. Align the image data and operating condition data according to the timestamp to form a one-to-one corresponding input sample.

[0050] Preferably, this step uses the MIPI CSI interface to transmit image data and the I2C bus to transmit low-speed operating data.

[0051] It should be noted that: since temperature, humidity and attitude data directly affect the fog scattering state inside the pipeline, and the operating condition data is a necessary input for the subsequent physical model to solve for transmittance and fog scattering coefficient, this step collects image data and operating condition data, and the main control unit achieves timestamp synchronization through a unified clock source, thereby ensuring a one-to-one correspondence between image data and operating condition data; unlike the traditional pure data-driven method that only relies on the image itself, this embodiment uses operating condition data as a quantifiable source of physical constraints, providing an interpretable environmental basis for the subsequent defogging process.

[0052] Step S2: Preprocess the input image and generate an initial fogging characterization; This step inputs the synchronized and aligned image of the drifting capsule robot output from step S1. Then, the image quality is optimized by performing the following processing: A1. Images of the drifting capsule robot The standard input image is obtained by performing size normalization, pixel value normalization, and Gaussian noise suppression. ; Preferably, the image size after size normalization is adjusted to 640×640 pixels; pixel value normalization is achieved by dividing each pixel value by 255, mapping it to a range. Gaussian noise suppression is achieved by performing convolution operations using a Gaussian filter with a standard deviation of 0.8.

[0053] A2. From standard input image Low-level visual features are extracted; these low-level visual features include dark channel features, local contrast features, saturation features, and edge gradient features. For example, the dark channel features are extracted by: processing the standard input image... The minimum value of each pixel in the RGB three channels is taken to obtain a single-channel minimum value map. Then, a minimum value filter is applied to the single-channel minimum value map using a 15×15 pixel rectangular window to output a single-channel dark channel feature map with the same spatial resolution as the input image. For example, the local contrast features are extracted by: taking the standard input image... If the image is converted to a grayscale image, the local standard deviation is calculated using a 5×5 pixel rectangular window on the grayscale image, and the output is a single-channel local contrast feature map with the same spatial resolution as the input image. For example, the saturation feature is extracted by: taking the standard input image... The color space is converted from RGB to HSV, the S channel is extracted as a saturation feature map, and a single-channel saturation feature map with the same spatial resolution as the input image is output. For example, edge gradient features are extracted by taking a standard input image... Switch to grayscale images and apply the Sobel operator in the horizontal direction. and the vertical Sobel operator Perform convolution operations to obtain the horizontal gradient map. and vertical gradient plot Calculate the gradient magnitude using the following formula. As an edge gradient feature map: ; in, , The edge gradient feature map is a single-channel feature map with the same spatial resolution as the input image.

[0054] A3. Concatenate the above dark channel features, local contrast features, saturation features, and edge gradient features to obtain a low-level visual feature matrix. ; For example, low-level visual feature matrix Its dimensions are 640×640×4.

[0055] It should be noted that this step will use the original image of the drifting capsule robot. Convert to standard input image And explicitly construct a low-level visual feature matrix with physical meaning. The dark channel feature is positively correlated with the fog concentration in a local area of ​​the image, directly reflecting the degree of fogging. The local contrast feature quantifies the spatial variation of image sharpness. The saturation feature is related to the degree of color degradation caused by water fog. The magnitude of the edge gradient feature reflects the sharpness of the pipe defect edges. Compared with the purely data-driven approach that relies solely on implicit feature extraction via convolutional neural networks in existing technologies, this technical solution explicitly calculates physically interpretable visual features, providing a traceable and understandable basis for subsequent fog concentration estimation and dynamic routing decisions. This enables the system to adaptively enhance defogging recovery capabilities in dense fog areas while avoiding unnecessary texture loss in light fog areas.

[0056] Step S3: Calculate transmittance and atmospheric optical parameters based on the physical prior estimation module; This step takes the standard input image output from step S2. The synchronous operating condition data output from step S1 is then processed as follows: A1. The physical prior estimation module describes the standard input image based on an atmospheric scattering model. The degradation process, the atmospheric scattering model is a well-known physical model describing the formation of foggy images, and its mathematical expression is defined as: ; in, The standard input image serves as the observed fog image. To recover a clear image, Transmittance represents the proportion of light that penetrates the fog medium to reach the camera, and its value range is... , This is a global atmospheric light parameter, representing the atmospheric light intensity at infinity. This parameter is an RGB three-channel vector to reflect the attenuation differences of different color channels.

[0057] A2. According to the physical definition of the atmospheric scattering model above, the above transmittance... The calculation formula is further expressed as: ;in, This is the fog scattering coefficient, in units of It characterizes the degree of light attenuation per unit distance; Scene depth, unit: , representing pixels The distance from the corresponding spatial point to the camera's optical center; For example, the fog scattering coefficient The methods for obtaining it are as follows: First, a temperature and humidity-fog scattering coefficient lookup table was constructed in advance through calibration experiments: at different temperatures and relative humidity Under the combined conditions, the image contrast attenuation rate is measured using a standard grayscale target, and the corresponding... Values ​​are used to form a two-dimensional lookup table. .

[0058] During system operation, a lookup table is queried based on the temperature and relative humidity values ​​in the synchronized operating data. When the actual temperature and humidity fall into the adjacent nodes in the table and When the time interval is reached, piecewise bilinear interpolation is used to obtain the reference fog scattering coefficient. : ; in, For nodes The calibration value at that location.

[0059] Furthermore, considering the dynamic impact of water flow disturbance within the pipeline on local fog concentration, a dimensionless disturbance intensity factor is calculated based on the triaxial angular velocity and drift velocity from the synchronous operating data. The calculation formula is as follows: ; in, These are the three-axis angular velocity values ​​from the synchronous operating data, in units of... ; This represents the upper limit of the angular velocity range of the IMU sensor, in units of... ; The drift velocity in the synchronized operating data, in units of ; For the maximum designed drifting speed of the drifting capsule robot, an example is taken as follows: ; A contribution coefficient for a preset linear velocity, as exemplified. The preferred value is 0.2.

[0060] The above disturbance intensity factor Substituting into the following formula for the reference fog scattering coefficient Fine-tuning was performed to obtain the final fog scattering coefficient. The calculation formula is as follows: ; in, The preset disturbance sensitivity coefficient has an exemplary value range of [value range missing]. The preset parameters in the above formula are preferably calibrated offline based on historical inspection samples during the training phase, and finely adjusted during the deployment phase according to the temperature and humidity statistical distribution and drift velocity distribution of the current inspection section. For example, scene depth The methods for obtaining it are as follows: Pre-calibrate the camera intrinsic parameter matrix of the drifting capsule robot Mounting angle of the camera optical axis relative to the robot axis and pipe radius (For pipes with a known diameter) or pipe cross-sectional profile parameters obtained through structured light pre-scanning; during the drifting process, it is assumed that the robot is at the geometric center of the pipe, or the camera optical center position is obtained through real-time structured light correction. The offset relative to the pipeline axis can be obtained through joint calibration of the IMU and structured light. For standard input image any pixel in The camera perspective projection model is used to back-project the view direction vector into the camera coordinate system in a three-dimensional view direction vector. : ; Construct a rotation matrix using the three-axis attitude angles from the synchronous operating data. , the direction vector of the line of sight Direction vector from camera coordinate system to world coordinate system (pipeline coordinate system) .

[0061] Based on the cylindrical geometric model of a pipe, the inner wall of the pipe can be represented by the equation of a cylindrical surface. ,in Let be the unit vector along the pipe axis; combine the positions of the camera's optical center and the unit vector. With line of sight The ray equation is: Solve for positive real numbers The intersection point If the equation of the cylindrical surface is satisfied, then the scene depth corresponding to this pixel is... That is: ; in, This refers to the effective optical path length of light from the optical center of the camera to the tube wall.

[0062] A3. Calculate initial atmospheric optical parameters using the dark channel prior method. Specifically, this involves calculating the standard input image. Dark passage diagram ,in In pixels A local window centered on the image; select the pixel positions in the dark channel image whose brightness values ​​are in the top 0.1%, and take these positions in the standard input image. The average pixel values ​​of each channel constitute the initial atmospheric light parameters for the RGB three channels. ; Construct a lightweight physical parameter network to perform residual correction on the initial physical parameters obtained from table lookup and geometric model; For example, the lightweight physical parameter network adopts a dual-head structure: The input layer receives standard input images. Initial transmittance and initial atmospheric light parameters The splicing tensor; where the initial atmospheric light parameters Expanded to match the standard input image through spatial copying. Three-channel feature maps of consistent size are then used for channel stitching. Then it goes through 3 convolutional layers (each convolutional kernel size is...). (with a step size of 1, padding of 1, and activation function of ReLU), and the number of channels is 32, 64, and 128 respectively; After this, it split into two branches: Transmittance correction head: After passing through one 3×3 convolution layer (outputting 1 channel), it directly outputs the spatial resolution of the standard input image. Consistent transmittance correction ; Atmospheric light correction head: After passing through a global average pooling layer and a fully connected layer, it outputs the atmospheric light correction amount for RGB three channels. ; Corrected transmittance The calculation formula is: ; Corrected atmospheric light parameters The calculation formula is: ; in, This means that the values ​​are element-wise limited to between 0.1 and 1 to avoid division by zero or over-enhancement problems caused by excessively low transmittance; the corrected atmospheric light parameters It is an RGB three-channel vector.

[0063] A4. Based on the corrected transmittance and corrected atmospheric light parameters The physical prior defogging results are calculated based on the inverse transform of the atmospheric scattering model. The calculation formula is as follows: ; In this formula, division and addition / subtraction operations are performed independently pixel-by-pixel on each of the RGB three channels, resulting in an image identical to the standard input image. Physical prior dehazing results of the same size .

[0064] The above-mentioned corrected atmospheric light parameters Employing an RGB three-channel vector format ensures consistency across color image channels and avoids color shift caused by a single scalar atmospheric light source; the resulting physically prior dehazing result is calculated accordingly. It follows the physical laws of fog imaging, provides strong recovery constraints in dense fog areas, effectively reduces color distortion and artifacts, and retains the interface for co-optimization with subsequent data-driven modules.

[0065] It should be noted that this step assigns the fog scattering coefficient by combining a temperature and humidity lookup table with IMU perturbation fine-tuning. With adaptive working conditions, each pixel is given geometrically consistent scene depth through a cylindrical pipe geometry model and IMU attitude calculation. Thus, an initial transmittance with physical rationality is obtained. Finally, a lightweight physical parameter network is used to perform data-driven residual correction on the initial parameters to compensate for the deviation between the ideal physical model and the actual non-uniform fog field and local illumination changes. Unlike the existing technology that relies solely on the dark channel of a single image or deep learning end-to-end estimation of transmittance, this embodiment uses a three-stage coupling mechanism of fog scattering coefficient determination driven by working condition data, optical path estimation constrained by geometric model, and network residual correction to enable the transmittance estimation to have both physical interpretability and data adaptability.

[0066] Step S4: Perform residual learning on the physical prior results using the data-driven dehazing module; The specific steps are as follows: A1. The data-driven dehazing module adopts a lightweight UNet variant structure, including an encoder, a decoder, and an independent residual compensation branch; Preferably, the encoder includes four downsampling blocks, each of which includes two 3×3 convolutional layers, a ReLU activation function, and a BatchNorm layer, with the number of channels being 64, 128, 256, and 512 respectively; the decoder is symmetrically arranged with the encoder; the residual compensation branch is used to compensate for detail textures and includes three 3×3 convolutional layers with the number of channels being 64, 32, and 3 respectively.

[0067] A2. Input standard image Physical prior defogging results Corrected transmittance and the corrected atmospheric light parameters Channel concatenation is performed to form a 10-channel input tensor; this 10-channel input tensor is then used to drive the dehazing module; the encoder extracts multi-scale texture features, the decoder recovers structural information, and the residual compensation branch preserves high-frequency information of defect edges, ultimately outputting a residual correction map. ; For example, residual correction plot The preferred dimensions are 640×640×3, with data-driven dehazing results. Size and standard input image Consistent.

[0068] A3. Based on the residual correction diagram Generate data-driven dehazing results The calculation formula is as follows: ; It should be noted that this step is based on the physical prior defogging results. Based on this, nonlinear residual information is learned through an explicit UNet structure to compensate for local occlusion, self-heating fogging, and edge blurring problems that are difficult for the physical model to fully describe; the above 10-channel input simultaneously includes physical prior results and physical parameters. , This approach ensures explicit guidance from physical constraints while facilitating engineering implementation. Compared to traditional two-stage pure data-driven methods, the residual learning process in this embodiment can simultaneously consider physical consistency and texture fidelity, ensuring that the dehazing results retain both physical rationality and defect edge details.

[0069] Step S5: Generate a fog concentration probability map using the fog concentration estimation submodule; A1. Input standard image With low-level visual feature matrix The components are spliced ​​together to form a new tensor, the preferred size of which is 640×640×7. Preferably, the fog concentration estimation submodule is a lightweight convolutional neural network with a structure of 3 layers of 3×3 convolutions, ReLU activation function and BatchNorm layer, with the number of channels being 16, 32 and 1 respectively, and the terminal adopts Sigmoid activation function.

[0070] A2. Input the newly formed tensor into the fog density estimation submodule. The fog density estimation submodule jointly learns dark channel, local contrast, saturation, and edge gradient information, and outputs a fog density probability map of the same size as the input image. Each pixel value is normalized to the interval [0, 1], and the larger the value, the higher the fog concentration at that location.

[0071] It should be noted that this step uses a lightweight convolutional neural network to transform low-level visual features into pixel-by-pixel fog density probability maps. This provides a spatialized and quantifiable decision-making basis for subsequent dynamic routing; unlike the fixed-weight fusion method in existing technologies, the fog concentration probability map... It can distinguish between dense fog and light fog, automatically enhancing physical constraints in dense fog areas while preserving the original texture as much as possible in light fog areas, thus achieving differentiated enhancement; due to the fog concentration probability map It has pixel-by-pixel spatial properties, so the subsequent fusion weights can be adaptively adjusted for local regions, rather than using a uniform processing strategy for the entire image.

[0072] Step S6: Generate a defect type weight vector through the defect type encoding submodule; Specifically, data-driven dehazing results Input the defect type encoding submodule, extract multi-scale semantic feature maps through the defect type encoding submodule, and generate defect type weight vectors through global average pooling. ; Preferably, the defect type encoding submodule is a small convolutional neural network with a structure of 2 layers of 3×3 convolutions, global average pooling, and fully connected layers 128→64→3.

[0073] Preferably, the defect type weight vector It is a 3-dimensional vector, corresponding to cracks, fractures and deformations. The value range of each dimension is the interval [0, 1], which is used to represent the sensitivity weight of this type of defect to edge, contrast and structural information.

[0074] It should be noted that this step encodes defect type sensitivity into an explicit defect type weight vector. This allows routing decisions to consider both fog concentration and defect characteristics; although the defect type weight vector Dehazing results driven by data before fusion Extraction, but through end-to-end joint training, the defect detection module improves the final dehazed image. The loss gradient can be backpropagated to the defect type encoding submodule via the subsequent fusion step S8 and routing controller S7, thereby achieving data-driven dehazing results. With the final dehazed image The mapping relationship between them is constrained in joint training; Specifically, for crack-type defects, when the defect type weight vector As the dimension of the crack increases, the system can automatically enhance the preservation of edge information; when the dimension of the fracture or deformation increases, it places greater emphasis on structural continuity and regional contrast; compared with the approach of uniformly applying the same fusion strategy to all defects, this embodiment helps to reduce the false negative rate of small cracks; in addition, the above-mentioned multi-scale semantic feature map is based on the input image It uses high-level semantic representations, rather than pre-specified candidate defect regions, thus avoiding the logical jump of introducing candidate regions before defect detection is completed.

[0075] Step S7: Generate an adaptive fusion weight matrix based on the fog concentration sensing dynamic routing controller; The specific steps are as follows: A1. The fog concentration sensing dynamic routing controller first processes the fog concentration probability map. and corrected transmittance Global average pooling, max pooling, and 4×4 block statistics are performed respectively to obtain fog concentration statistical vector and transmittance statistical vector. Among them, global average pooling is used to extract the overall fog concentration level, max pooling is used to capture local dense fog peaks, and block statistics are used to preserve the spatial distribution trend of fog concentration.

[0076] A2. Combine the fog concentration statistical vector, transmittance statistical vector, and defect type weight vector. The vectors are concatenated into a low-dimensional routing input vector and input into a lightweight routing network. The lightweight routing network then outputs a low-resolution fused weight map. For example, the lightweight routing network adopts a two-layer fully connected structure: the first layer maps the input dimension to 256 dimensions, and the second layer further maps it to 1600 dimensions, both layers using the ReLU activation function; after performing a nonlinear mapping on the low-dimensional routing input vector, the network outputs a 1600-dimensional vector, which is then reshaped into a 40×40×1 low-resolution fused weight map; subsequently, this low-resolution fused weight map is matched to the standard input image. The spatial size was determined by upsampling to a resolution of 640×640 using bilinear interpolation to obtain a preliminary adaptive fusion weight matrix. Finally, a Sigmoid activation function was applied to each element of this matrix, and its value was normalized to the interval [0, 1] to obtain the final adaptive fusion weight matrix. ; The above-described low-resolution generation, upsampling, and normalization process reduces the computational load of the routing network while ensuring the pixel-by-pixel correspondence between the weight map and the image to be fused in space.

[0077] A3. During the training phase, the fog concentration sensing dynamic routing controller will output the fog concentration probability map in step S5. The corrected transmittance output in step S3 The statistical characteristics of the data, and the defect type weight vector V output in step S6, are used as joint inputs to perform parameter learning through the lightweight routing network to generate the adaptive fusion weight matrix. ; The lightweight routing network aims to minimize the joint loss function L defined in step S10. It automatically learns the mapping relationship between each input feature and the fusion weights through gradient backpropagation, thereby enabling the adaptive fusion weight matrix to... The spatial distribution adaptively satisfies the following physical and task semantic constraints: In areas with high fog concentration (i.e., fog concentration probability map) When the pixel value is large, the adaptive fusion weight matrix The value at the corresponding position tends to increase, thereby improving the physical prior dehazing results. The proportion of contribution; In areas with low fog concentration (i.e., fog concentration probability map) When the pixel value is small, the adaptive fusion weight matrix The values ​​at the corresponding positions tend to decrease, thereby improving the results of data-driven dehazing. The proportion of contribution; When the component corresponding to the crack feature in the defect type weight vector V is large, the lightweight routing network further enhances the fusion weight with the corresponding position of the defect edge region to improve the edge detail preservation capability. For example, the preset fog concentration threshold ranges from 0.5 to 0.7, preferably 0.6; the preset crack threshold ranges from 0.4 to 0.6, preferably 0.5; the above thresholds are not used as hard control rules, but only to describe the adaptive fusion weight matrix after training. The statistical distribution characteristics of the metric can be adjusted according to the actual application scenario.

[0078] It should be noted that this step involves physical transmission prior to the a priori hypothesis and fog concentration probability map. Defect type weight vector A common input is used to the explicit route controller, and a spatially adaptive adaptive fusion weight matrix is ​​generated by combining statistical summary features with the lightweight routing network. Compared to directly flattening a 640×640 image and inputting it into a fully connected network, using statistical summarization features can reduce computational cost and improve engineering feasibility; adaptive fusion of weight matrices... The system can implement a dynamic fusion strategy that emphasizes physical properties in dense fog, preserves texture in thin fog, and is sensitive to defects. The routing input in this step uses pooling and block statistics to make the routing module focus on the statistical characteristics of fog concentration distribution and transmittance distribution, thereby making the module lighter, more stable, and easier to deploy on the edge computing platform of the drift capsule robot.

[0079] Step S8: Fuse the two types of dehazing results according to the adaptive fusion weight matrix; The specific steps are as follows: Based on the adaptive fusion weight matrix For the physical prior defogging results and data-driven dehazing results Perform pixel-level weighted fusion to generate the final dehazed image. The calculation formula is: ; in, This represents a pixel-wise dot product operation, that is, for each pixel position... The weight matrix The scalar weight value in the code applies simultaneously to all three RGB channels at the corresponding position, meaning the weight value is shared across the channel dimension; physical prior dehazing results. Data-driven dehazing results and the final dehazed image All are of the following dimensions: RGB three-channel image; adaptive fusion weight matrix A single-channel pixel-wise weighted map with a size of 640×640; During the fusion process, for each pixel position, the weight value applies synchronously to the three RGB channels, thereby ensuring color consistency while achieving adaptive fusion based on pixel-level fog density and defect characteristics; the design of sharing single-channel weights among the RGB channels helps to avoid color shift problems caused by inconsistent fusion ratios between different channels. Through the above fusion operation, the system can flexibly adjust the contribution ratio of physical priors and data-driven results according to the actual fog concentration and defect characteristics of each pixel location.

[0080] It should be noted that this step utilizes the aforementioned adaptive fusion weight matrix. This method achieves refined, pixel-level controllable fusion of two types of dehazing results, ensuring that the fused image inherits both the radiometric fidelity of the physical model and the texture restoration capability of the deep learning model. Compared to purely physical methods, the final dehazed image... Richer details are preserved in hazy areas and defect edges; compared to purely data-driven methods, the final dehazed image... Reduced artifacts and color distortion in dense fog areas; Specifically, in the fog concentration probability map In dense fog regions with large numerical values, the adaptive fusion weight matrix... The corresponding increase makes the fusion result more dependent on the physical prior dehazing result. This enhances physical constraints and suppresses noise amplification; in thin fog regions with low fog concentration, the adaptive fusion weight matrix... The corresponding reduction makes the fusion result more dependent on data-driven dehazing results. This allows for the preservation of texture details and defect edge information of the pipe's inner wall to the greatest extent possible; this region-differentiated adaptive fusion strategy results in a final dehazed image. It can effectively cope with complex working conditions such as uneven distribution of water mist and drastic changes in concentration inside actual pipelines.

[0081] Step S9: Dehaze the final image Input the defect detection module to perform defect detection; The defect detection module employs an instance segmentation network built upon the YOLOv5s backbone network. For example, a parallel mask prediction branch is added to the detection head of YOLOv5s to form a lightweight instance segmentation network that can output object detection results and instance segmentation masks at the same time. Its backbone network is CSPDarknet, the detection head adopts 3-scale output, and the mask branch shares feature maps with the detection head to output pixel-level segmentation masks while maintaining lightweightness.

[0082] The specific steps are as follows: A1. The final dehazed image Input defect detection module; during the inference phase, the defect detection module performs forward propagation on the input image and directly outputs the detection results; The detection results include: defect category labels (covering at least three categories: cracks, fractures, and deformations), bounding box coordinates for locating defects, and segmentation masks describing the precise contours of defects; wherein, the defect category labels are output by the classification branch, the bounding box coordinates are output by the regression branch, and the segmentation masks are output by the mask prediction branch.

[0083] A2. During the training phase, the defect detection module outputs the detection task loss. It is not only used to update the network parameters of the defect detection module itself, but also transmitted to the defect type encoding submodule and the fog concentration sensing dynamic routing controller through the gradient backpropagation mechanism. This enables the above modules to adaptively adjust their internal parameters according to the feedback signal of the defect detection results, thereby enhancing the response capability to defect discrimination-related features and improving the pertinence and effectiveness of subsequent fusion weight allocation.

[0084] Furthermore, the detection task loss Together with the dehazing reconstruction loss and physical consistency loss in subsequent step S10, it constitutes a joint loss function and participates in the end-to-end joint optimization process of the entire system in step S10.

[0085] It should be noted that this step will result in a high-quality final dehazed image. By directly inputting the defect detection module, the defect detection module can receive input images that are more conducive to defect feature recognition and localization without changing its overall framework.

[0086] Step S10: Perform joint optimization during the training phase; The specific steps are as follows: A1. Construct a joint optimization control module and calculate a joint loss function consisting of dehazing reconstruction loss, physical consistency loss, and detection task loss to collaboratively supervise the training process of the entire system: Furthermore, defogging reconstruction loss Used to evaluate the final dehazed image With clear reference image The formula for calculating consistency in pixel space is: ; in, This refers to the total number of pixels in the image used in the calculation, for example, 409,600 pixels in a 640×640 image; This represents the L1 norm, which is the average of the absolute values ​​of the differences between the three RGB channels. This represents the spatial coordinates of the i-th pixel in the image.

[0087] Furthermore, physical consistency loss Used to measure the final dehazed image Whether the atmospheric scattering imaging law is satisfied is calculated using the following formula: ; ; in, For the final dehazed image Corrected transmittance Corrected atmospheric light parameters The reconstructed fogged image; The standard input image is the output of step S2; , All of these originate from step S3; this loss term is used to force the defogging result to remain consistent with the actual foggy imaging mechanism, thereby reducing artifacts that do not conform to physical laws.

[0088] Furthermore, the detection task loss This is used to measure the deviation between the predicted results output by the defect detection module and the pre-labeled true labels in the training set; the detection task loss. Classification loss Location loss and segmentation mask loss The weighted average consists of three parts: ; in, , , The preset sub-loss weight coefficients are 1.0, 0.05, and 0.5, respectively.

[0089] The classification loss Multi-class cross-entropy loss is used to determine whether the predicted bounding box correctly identifies the target category: ; in, To predict the total number of bounding boxes; This represents the total number of categories, excluding background categories. and These are the true label and the predicted probability, respectively.

[0090] Location loss CIoU loss is preferably used to accurately constrain the positional relationship, center point distance, and aspect ratio consistency between the predicted bounding box and the ground truth bounding box. The specific calculation method can adopt the CIoU loss form known in the art, which will not be elaborated here to avoid repeating the known technology.

[0091] Segmentation mask loss Binary cross-entropy loss is preferred for refining pixel-level classification within the predicted bounding box, ensuring accurate differentiation between defect areas and background. The calculation formula is as follows: ; in, This represents the total number of pixels within the segmentation mask region corresponding to all predicted bounding boxes. Let i be the true mask label of the i-th pixel. ; This represents the probability predicted by the defect detection module that the pixel belongs to a defect region, with a value range of [value range missing]. The output is determined by the Sigmoid function.

[0092] A2. The total loss is calculated by weighted summation of the above three losses. The definition is as follows: ; in, , and These are the weighting coefficients that control the relative importance of each loss term; For example, , and The values ​​range from 0.2 to 0.5, with preferred values ​​of 0.4, 0.3, and 0.3, respectively, to achieve a balance between visual quality, physical rationality, and detection accuracy.

[0093] A3. During the training iteration process, based on the calculation of the total loss... The total loss is solved using an automatic differentiation mechanism. Regarding the gradients of all trainable parameters, gradient descent optimization algorithms (such as Adam) are used to backpropagate the gradients and simultaneously update the trainable parameters of the following six core modules: physical prior estimation module, data-driven dehazing module, fog concentration estimation submodule, defect type encoding submodule, fog concentration-aware dynamic routing controller, and defect detection module. Through the above end-to-end joint optimization, each module is coordinated and adjusted under the constraint of a unified loss function, so that the image dehazing process takes into account both physical consistency and visual quality, while improving the accuracy and robustness of defect detection, thus forming a closed-loop optimization mechanism for defect detection tasks.

[0094] It should be noted that this step unifies pixel-level visual quality constraints, physical model consistency constraints, and high-level semantic task constraints into a single optimization framework by constructing a joint loss function that includes dehazing reconstruction loss, physical consistency loss, and detection task loss. Specifically, the dehazing reconstruction loss ensures the basic visual quality of the dehazed image, the physical consistency loss constrains the dehazing result to conform to imaging physics, and the detection task loss guides the dehazing process to retain key features that are crucial for defect identification. Through this multi-objective joint optimization mechanism, the parameters of each module are updated collaboratively under a unified objective, thus balancing image clarity, physical plausibility, and defect detection performance under complex water fog interference conditions.

[0095] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A pipeline defect detection system for a drifting capsule robot, characterized in that: The system includes a drifting capsule robot, an image acquisition unit, a working condition acquisition unit, a main control unit, a preprocessing module, a physical prior estimation module, a data-driven defogging module, a fog concentration estimation submodule, a defect type encoding submodule, a fog concentration sensing dynamic routing controller, a fusion module, a defect detection module, and a joint optimization control module. The image acquisition unit and the working condition acquisition unit are both communicatively connected to the main control unit, which acquires timestamp-aligned images of the drifting capsule robot and synchronized working condition data. The preprocessing module is used to preprocess the images of the drifting capsule robot, extract low-level visual features, and output a standard input image and a low-level visual feature matrix. The physical prior estimation module is used to calculate the corrected transmittance and the corrected atmospheric light parameters based on synchronous operating condition data and standard input image, and to calculate the physical prior defogging result based on the corrected transmittance and the corrected atmospheric light parameters. The data-driven dehazing module is used to perform residual learning based on the standard input image, the physical prior dehazing result, and the corrected transmittance, and output the data-driven dehazing result. The fog concentration estimation submodule is used to generate a fog concentration probability map based on the standard input image and the low-level visual feature matrix; The defect type encoding submodule is used to generate a defect type weight vector based on the data-driven dehazing results; The fog concentration sensing dynamic routing controller is used to receive the fog concentration probability map, the corrected transmittance and defect type weight vector, and generate a spatially adaptive fusion weight matrix. The fusion module is used to perform pixel-level weighted fusion of the physical prior dehazing result and the data-driven dehazing result based on the adaptive fusion weight matrix, and output the final dehazing image. The defect detection module is used to detect defects in the final dehazed image and output the defect category, bounding box coordinates, and segmentation mask. The joint optimization control module is used to perform end-to-end joint optimization of trainable parameters in the physical prior estimation module, data-driven dehazing module, fog concentration estimation submodule, defect type encoding submodule, fog concentration-aware dynamic routing controller, and defect detection module during the training phase using a joint loss function composed of a weighted sum of dehazing reconstruction loss, physical consistency loss, and detection task loss.

2. The image pipeline defect detection system for a drifting capsule robot as described in claim 1, characterized in that: The main control unit is also used to timestamp and align the images and operating data of the drifting capsule robot to form a synchronous record; the operating data includes at least temperature value, relative humidity value, three-axis angular velocity value, three-axis attitude angle and drift speed; The physical prior estimation module determines the fog scattering coefficient based on temperature and relative humidity values, and fine-tunes the fog scattering coefficient by combining the triaxial angular velocity and drift velocity values. At the same time, it uses the pipeline geometry model and triaxial attitude angles to calculate the scene depth, thereby obtaining the initial transmittance and initial atmospheric light parameters.

3. The image pipeline defect detection system for a drifting capsule robot as described in claim 2, characterized in that: The physical prior estimation module also includes a lightweight physical parameter network, which receives a standard input image, initial transmittance and initial atmospheric light parameters, and outputs transmittance correction and atmospheric light correction. The physical prior estimation module obtains the corrected transmittance and corrected atmospheric light parameters based on the transmittance correction and atmospheric light correction.

4. The image pipeline defect detection system for a drifting capsule robot as described in claim 3, characterized in that: The preprocessing module is used to perform size normalization, pixel value normalization and Gaussian noise suppression on the image of the drifting capsule robot, and extract dark channel features, local contrast features, saturation features and edge gradient features from the standard input image to obtain a low-level visual feature matrix. The fog concentration estimation submodule is used to generate a fog concentration probability map based on the standard input image and the low-level visual feature matrix.

5. The image pipeline defect detection system for a drifting capsule robot as described in claim 4, characterized in that: The data-driven dehazing module employs a dehazing network consisting of an encoder, a decoder, and a residual compensation branch. The data-driven dehazing module is used to perform residual learning on the physical prior dehazing results to obtain the data-driven dehazing results.

6. The image pipeline defect detection system for a drifting capsule robot as described in claim 5, characterized in that: The defect type encoding submodule is used to extract semantic features from the data-driven dehazing results and generate a three-dimensional defect type weight vector, which corresponds to three defect types: crack, fracture, and deformation.

7. The image pipeline defect detection system for a drifting capsule robot as described in claim 6, characterized in that: The fog concentration sensing dynamic routing controller is used to extract statistical features from the fog concentration probability map and the corrected transmittance, and then fuse them with the defect type weight vector and input them into the lightweight routing network to generate a low-resolution fused weight map. The low-resolution fused weight map is then upsampled to obtain an adaptive fused weight matrix.

8. The image pipeline defect detection system for a drifting capsule robot as described in claim 7, characterized in that: The fusion module is used to perform pixel-level weighted fusion of the physical prior dehazing result and the data-driven dehazing result based on the adaptive fusion weight matrix. The adaptive fusion weight matrix is ​​a single-channel pixel-by-pixel weight map, which shares weight values ​​among the RGB three channels.

9. The image pipeline defect detection system for a drifting capsule robot as described in claim 8, characterized in that: The defect detection module uses an instance segmentation network built on the YOLOv5s backbone network. During the training phase, the detection task loss output by the defect detection module is not only used to update the parameters of the defect detection module itself, but also passed to the defect type encoding submodule and the fog concentration sensing dynamic routing controller through the gradient backpropagation mechanism. This enables the above modules to adaptively adjust their parameters based on the detection feedback signal, thereby enhancing their responsiveness to defect discrimination features. The detection task loss, defogging reconstruction loss, and physical consistency loss together constitute the total loss function, which is used to perform end-to-end joint optimization of each module of the system.

10. The image pipeline defect detection system for a drifting capsule robot as described in claim 9, characterized in that: The joint optimization control module is used to solve the gradient of the total loss function with respect to each trainable parameter based on the joint loss function through an automatic differentiation mechanism, and then synchronously update the trainable parameters of the following modules through a gradient descent optimization algorithm: physical prior estimation module, data-driven defogging module, fog concentration estimation submodule, defect type encoding submodule, fog concentration-aware dynamic routing controller, and defect detection module, thereby achieving end-to-end joint optimization and enabling each module to coordinately adjust its parameters under the constraint of a unified loss function.

Images