A method for detecting micro-defects of a catheter based on artificial intelligence and visual technology

Abstract

The application discloses a catheter micro-defect detection method based on artificial intelligence and visual technology, a catheter inner wall circumferential image sequence is collected through an endoscopic industrial camera, and a defect data set is constructed after preprocessing; an improved DETR visual detection branch is constructed to extract multi-scale features and generate a visual saliency response map; an image domain physical information neural network branch is constructed to output a physical residual error map; a physical priori enhanced target query is generated to guide defect prediction, and a physical quality enhanced Hungarian matching and a joint loss function are used for end-to-end training. The application realizes the automatic detection of catheter inner wall scratches, cracks and other micro-defects, and improves the detection precision, recall rate and robustness.

Description

Technical Field

[0001] This application belongs to the field of computer vision inspection technology, specifically relating to a method for detecting micro-defects in conduits based on artificial intelligence and vision technology. Background Technology

[0002] Conduits, serving as channels for the transmission of fluids, gases, or micro-devices, are widely used in industrial hydraulic systems, pneumatic control systems, and medical devices. The surface quality of the conduit's inner wall directly affects the system's operational stability, sealing performance, and service life. During manufacturing, processing, and service, microscopic surface defects such as scratches and cracks can easily occur on the conduit's inner wall. These defects may propagate under high pressure, high frequency, or corrosive conditions, potentially leading to media leakage or structural failure, posing a threat to system safety. Therefore, efficient and accurate detection of micro-defects on the conduit's inner wall has become an urgent technical problem to be solved in the quality control process of related industrial fields.

[0003] Currently, the detection methods for defects on the inner wall surface of catheters mainly rely on manual visual inspection and automated detection based on classical image processing algorithms. Manual visual inspection usually requires operators to observe the inner wall of the catheter segment by segment with the aid of an endoscope or microscope. The accuracy and consistency of the inspection results are limited by the operator's visual fatigue, subjective judgment experience, and concentration. The inefficiency of manual inspection is particularly prominent when dealing with long catheters with small diameters, and the ability to visually identify tiny scratches or cracks is limited, resulting in a high probability of missed detection.

[0004] Automatic detection methods based on classical image processing algorithms attempt to overcome the subjective limitations of manual detection. These methods typically rely on predefined defect geometric features or gray-level distribution rules, such as edge detection operators, thresholding algorithms, and morphological filtering, to perform region filtering and feature extraction on acquired duct inner wall images. However, duct inner wall images often contain backgrounds with uneven textures, variations in light reflection, and interference factors such as oil stains and processing marks. Classical image processing algorithms, dependent on manually set fixed parameters and rules, lack adaptability to complex and changing background environments. When the defect morphology or scale changes unexpectedly, or when background noise and the defect target exhibit high similarity in gray-level distribution, rule-based methods struggle to effectively distinguish between real and false defects, leading to a significant increase in false detection or false negative rates. Furthermore, traditional methods also have limitations in spatial localization accuracy for small targets.

[0005] In summary, among the existing technologies for detecting micro-defects in the inner wall of catheters, manual visual inspection has the disadvantages of low detection efficiency and strong subjectivity of results, while automatic detection methods based on classical image processing algorithms are limited in terms of stability and accuracy when faced with complex background interference and micro-defect targets with varied shapes. Summary of the Invention

[0006] To address the technical problems of low efficiency and high subjectivity of manual visual inspection in the existing catheter inner wall micro-defect detection process, as well as the insufficient adaptability of classical image processing algorithms to complex backgrounds and small defects, this paper proposes a catheter micro-defect detection method based on artificial intelligence and vision technology that can achieve high-precision and automated identification and localization.

[0007] To achieve the above technical objectives, this application specifically employs the following technical solution: One aspect of this application provides a method for detecting micro-defects in conduits based on artificial intelligence and vision technology, comprising the following steps: S1. Image acquisition inside the conduit: Insert an endoscopic industrial camera into the inside of the pipe to be tested, control the endoscopic industrial camera to move and rotate along the pipe axis to take pictures, and obtain a circumferential image sequence of the inner wall of the pipe. S2. Image preprocessing and defect sample construction: The acquired duct inner wall images are denoised, normalized, sized, illuminated, augmented, and labeled with defects to construct a dataset containing defect category labels, bounding box labels, and optional defect mask labels. S3. Constructing an improved DETR visual detection branch: The improved DETR visual detection branch includes a convolutional backbone feature extraction module, a multi-scale feature fusion module, a small target perception query initialization module, a Transformer encoder, a Transformer decoder, a classification head, and a bounding box regression head, which are used to extract multi-scale visual features of the inner wall of the duct and generate a visually salient response map. S4. Constructing the image domain physical information neural network branch: The image domain physical information neural network branch takes the image pixel coordinates, image gray value and visual features extracted by the improved DETR visual detection branch as input, and outputs the normal pipe wall apparent field, illumination field, defect anomaly field, defect potential function field and physical residual map. S5. Physical Prior Enhancement Target Query and Defect Prediction: Based on the visual saliency response map output by the improved DETR visual detection branch and the physical residual map output by the image domain physical information neural network branch, a physical prior enhancement response map is generated. The physical prior enhancement target query vector is constructed from the physical prior enhancement response map, input into the Transformer decoder, and outputs the duct inner wall defect category, defect bounding box, defect probability map and physical consistency score. S6. Physical quality enhancement matching and joint loss training: Hungarian matching is performed based on classification cost, bounding box cost, geometric overlap cost, physical residual cost and defect morphology cost, and the model is trained end-to-end using detection loss, scale-sensitive localization loss, image domain physical residual loss, defect boundary constraint loss and physical prior consistency loss. S7. Model Evaluation and Defect Detection Results Output: The trained model is evaluated using the test set, and mAP, accuracy, precision, recall, and physical consistency score are calculated. The defect category, defect location, bounding box, confidence score, defect probability map, and defect detection report are output.

[0008] In one embodiment, in step S1, the endoscopic industrial camera has a resolution of 1280×1080P and a frame rate of 30fps; the circumferential image sequence of the inner wall of the pipe includes scratch defect images and crack defect images.

[0009] In one implementation, step S2 specifically includes: S21. Perform preliminary screening on the original images of the inner wall of the catheter acquired in step S1, and remove samples that are severely blurred, overexposed, underexposed, severely obscured, have unidentifiable defect areas, or have failed to acquire images, and retain effective images that can reflect the true surface condition of the inner wall of the catheter. S22. Due to the narrow imaging space inside the duct, the image is easily affected by low illumination, sensor noise, local reflection and texture interference. Therefore, Gaussian filtering is used to denoise the original image. S23. Perform uniform size adjustment and normalization on the denoised image; the normalization process is calculated as follows:

[0010] In the formula, Represents the original image. and These represent the minimum and maximum pixel values ​​in the current image, respectively. This represents the normalized image; Uniform resizing scales an image to a fixed input size. During scaling, the original aspect ratio is maintained and edge padding is used; S24. To address common issues in catheter inner wall images such as uneven brightness, localized reflections, shadows, and low contrast, Gamma correction is applied to the images after size standardization to enhance illumination. S25. Use a labeling format to annotate the location of defects on the inner wall of the conduit with bounding boxes, and record the corresponding defect category labels; the bounding box is represented as... ,in, and Indicates the coordinates of the center of the defect area. and These represent the width and height of the bounding box, respectively, and 'c' represents the defect category label. S26. For samples requiring defect boundary constraints or image-domain PINN physical residual supervision, further construct pixel-level defect masks:

[0011] in, Indicates the area where the defect is located. This represents a defect mask. Correspondingly, the mask for the normal pipe wall region is represented as: Defect masks are used to constrain the defect anomaly field and the defect potential function field, while normal region masks are used to constrain the normal pipe wall apparent field and illumination field to satisfy the smoothness and low-frequency variation law.

[0012] S27. In the data preprocessing stage, data augmentation strategies are introduced, including random flipping, random cropping, brightness perturbation, contrast adjustment, and scale transformation. S28. Divide the preprocessed image data into training set, validation set and test set according to a preset ratio.

[0013] In one implementation, step S3 specifically includes: S31. Using a convolutional neural network as the backbone feature extraction network, multi-layer visual features are extracted from the duct image:

[0014] in Preserve edge and texture information of minor defects. Provides high-level semantic features, The enhanced image obtained in step S2; S32. Introduce a multi-scale feature pyramid network to fuse the different resolution features output by the backbone feature extraction network to obtain a high-resolution feature map containing details of minute defects:

[0015]

[0016] in, This represents the fused multi-scale feature map; S33, Utilizing high-resolution fusion feature maps Generate visually salient response maps: ,in, This represents the visual response map of suspected defective areas. Then, the area with the highest response value is selected from the visually salient response map. One position: in, This represents the initial set of candidate defect locations for visual analysis, and then for each candidate location... Extract visual features and location encoding to generate an initial visual query vector: ,in, This represents the initial visual query vector. This vector indicates the locations where visual defects may exist, and is used for subsequent physical prior augmentation. S34. Flatten the fused features, add positional encoding, and then input them into the Transformer encoder:

[0017] Self-attention is calculated as follows:

[0018] Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and d represents the feature dimension; S35, Output multi-scale visual features Encoder features Visual saliency response map and visual initial query vector This serves as input for subsequent PINN branches and the physical prior enhancement query module.

[0019] In one implementation, step S4 specifically includes: S41. For the preprocessed catheter inner wall image obtained in step S2 Establish a physical information neural network branch in the image pixel coordinate domain, assuming the position of any pixel in the image is: Meanwhile, to facilitate network learning, pixel coordinates are normalized to a fixed range. ,in:

[0020] Represents the original pixel coordinates ， These represent the image width and height, respectively, enabling the physical branch to learn the apparent changes in the inner wall of the duct, changes in illumination, and abnormal distribution of defects in a unified image coordinate space. S42. Perform multi-scale position encoding on the normalized pixel coordinates so that the network can simultaneously perceive large-scale illumination changes and local minor defects. S43. The image pixel coordinates, image grayscale values, and visual features extracted by the improved DETR visual branch are fused to form the input of the physical information neural network branch:

[0021] in Represents pixel coordinate encoding. This represents the grayscale value of the preprocessed catheter inner wall image. This indicates the corresponding positional visual features extracted by the improved DETR visual detection branch; S44. The image domain physical information neural network outputs four types of physical fields based on the above inputs and outputs:

[0022] in, This represents the normal apparent field of the pipe wall, used to describe the normal texture and grayscale distribution that the inner wall of the conduit should have under defect-free conditions; It represents the illumination field and is used to describe the low-frequency brightness changes caused by supplemental lighting, reflection, shadows, and uneven brightness during endoscopic imaging. It represents a defect anomaly field, used to indicate local anomaly regions that cannot be explained by normal pipe wall appearance and changes in illumination; This represents the defect potential function field, used to describe the defect boundary and its geometric continuity; S45. The image of the inner wall of the catheter is considered to be composed of the normal appearance of the catheter wall, changes in illumination, defects and anomalies, and noise. The basic relationship is as follows:

[0023] Based on the above relationships, the image reconstruction residuals are constructed as follows:

[0024] Used to measure whether the normal appearance field, illumination field, and defect / anomaly field output by the physical branch can reasonably explain the input image; S46. Construct apparent smoothing residuals and low-frequency illumination residuals to suppress the model from misclassifying normal textures, reflections, and shadows as defects. For defect boundaries, constrain their boundary continuity through a defect potential function field to make the boundary representation of slender defects such as cracks and scratches more stable. S47. The image reconstruction residual, apparent smoothing residual, low-frequency illumination residual, defect boundary residual, and defect anomaly field gradient are fused to obtain the physical residual map:

[0025] in, Represents the physical residual plot. This indicates the normal apparent residual of the pipe wall. Indicates illumination residual, This represents the residual of the defect potential function. to These are learnable weights; In one implementation, step S5 specifically includes: S51. Calculate the gradient magnitude of the enhanced image: It is used to reinforce areas of localized abrupt changes such as cracks, scratches, and corrosion boundaries; S52. Generate a physically prior enhanced response map by fusing visual salient response, physical residual, and image gradient:

[0026] in, To obtain a visually salient response map for S3, Obtain the physical residual map for S4. Then, obtain the physical prior augmentation response map. Select the top response values These locations were selected as the final candidate defect locations. These locations represent highly plausible candidate defect regions that are visually suspicious, physically abnormal, or exhibit abrupt changes at the edges. S53. Fuse the initial visual query from S3 with the physical information to generate the final query vector:

[0027] in, This represents the initial visual query obtained by S3. This represents the physical residual value at the candidate location. This represents the image gradient at the candidate location. This represents the grayscale value of the image at the candidate location; S54. Set the final target query set Global features output by the Transformer encoder Input the Transformer decoder together:

[0028] The decoder searches for corresponding defect regions in the global image features based on these query vectors and outputs target-level features; S55. The classification head outputs the defect category, the bounding box regression head outputs the defect location, and the optional defect probability map prediction head outputs the defect region probability map. Simultaneously, a physical consistency score is calculated based on the mean physical residual within the predicted bounding box region.

[0029] Indicates the first The physical consistency score of each predicted defect is calculated. If a region has a high detection confidence score and a high physical consistency score, it is more likely that the region is a real defect. If the detection confidence score is high but the physical consistency score is low, the region may be affected by reflection, shadow, or noise interference, and further verification is required.

[0030] In one implementation, step S6 specifically includes: S61. During model training, the improved DETR-PINN network outputs multiple candidate detection results for each duct inner wall image, including defect category probability, predicted bounding box, defect probability map, and physical consistency score. To ensure that these prediction results correspond one-to-one with the ground truth annotation results, matching is required first. Traditional DETR mainly performs Hungarian matching based on classification cost, bounding box cost, and geometric overlap cost. This patent further introduces physical residual cost and defect morphology cost, so that the matching process not only focuses on whether the predicted box coincides with the ground truth box, but also on whether the region has physical anomaly features of a real defect and whether the defect morphology is reasonable. The comprehensive matching cost can be expressed as:

[0031] S62. After matching is complete, train the model using the joint loss function. The total loss function can be expressed as:

[0032] S63. Jointly optimize the improved DETR visual detection branch and the image domain PINN physical branch through backpropagation; In one implementation, in step S7, the accuracy, precision, recall, and mean precision are calculated based on the number of true positives, false positives, true negatives, and false negatives in the confusion matrix; wherein, the calculation step of the mean precision includes: for each defect category, calculating the precision value at different recall thresholds, averaging the precision values ​​to obtain the mean precision of that category, and then averaging the mean precision of all defect categories to obtain the mean precision; the physical consistency score can be calculated from the mean physical residual within the prediction box area.

[0033] The beneficial effects of this application are as follows: 1. By introducing an improved detection converter network as the core detection model, an end-to-end detection paradigm is adopted, which avoids the complex process of relying on preset candidate boxes in traditional object detection algorithms, effectively reducing computational overhead and improving detection speed and overall efficiency.

[0034] 2. The improved DETR target detection network is combined with the PINN physical information branch in the image domain, so that the model not only relies on image visual features, but also utilizes physical constraints such as normal pipe wall appearance, illumination changes and defect anomalies, thereby improving the reliability of defect discrimination.

[0035] 3. Enhance the ability to express details of minute defects in the inner wall of the catheter by using a multi-scale feature fusion module, so that weak textures and small-scale defects such as shallow scratches and fine cracks can be more fully preserved in high-resolution features.

[0036] 4. By using visual saliency response maps and physical residual maps together to generate physical prior enhanced target queries, the Transformer decoder prioritizes regions with both visual and physical anomalies, thereby improving the recall rate of minor defects and reducing the risk of missed detections.

[0037] 4. The normalization, size-preserving scaling, and diverse data augmentation strategies employed in the data preprocessing stage suppressed the adverse effects of lighting differences and complex background textures on model training, thereby improving the model's generalization performance and robustness under different operating conditions.

[0038] 5. By separating the normal pipe wall apparent field, illumination field, and defect anomaly field through the PINN branch in the image domain, the interference of reflection, shadow, brightness unevenness, and normal texture abrupt changes on the detection results can be effectively reduced, thus reducing false defect detections. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the catheter micro-defect detection method based on artificial intelligence and vision technology according to an embodiment of this application; Figure 2 This is a schematic diagram of the internal sampling of the catheter according to an embodiment of this application; Figure 3 This is a schematic diagram of the improved detection converter network model structure used in the embodiments of this application. Detailed Implementation

[0040] The technical solution of this application will be clearly and completely described below with reference to specific embodiments. However, those skilled in the art will understand that the embodiments described below are only some embodiments of this application, not all embodiments, and are only used to illustrate this application, and should not be regarded as limiting the scope of this application. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] For the detection of micro-defects in the inner wall of catheters, this paper proposes an automated detection method based on an improved detection converter network. Since traditional target detection algorithms rely on the complex paradigm of pre-set candidate boxes and post-processing steps, this application adopts an end-to-end detection converter architecture. It utilizes a self-attention mechanism to model the global contextual relationships in the catheter inner wall image, directly outputting the defect category and bounding box prediction results to simplify the detection process and improve computational efficiency. Furthermore, the improved DETR visual branch extracts shallow texture features and deep semantic features of the catheter inner wall through a convolutional backbone and a multi-scale feature fusion module, enabling the detection of even small cracks. Minor defects such as shallow scratches and corrosion points are more fully represented. Simultaneously, an image-domain PINN physical branch is constructed, using pixel coordinates, image grayscale, and visual features as input to learn the normal pipe wall apparent field, illumination field, defect anomaly field, and defect potential function field, generating a physical residual map. This residual map is used to determine whether a region deviates from the normal pipe wall texture and illumination patterns, thus helping the model distinguish between real defects and reflections, shadows, or abrupt changes in normal texture. Furthermore, the visual saliency response map, physical residual map, and image gradient are fused to generate a physical prior enhancement query, which is then input into the Transformer decoder for defect prediction. These combined approaches improve the accuracy, recall, and robustness of micro-defect detection on the inner wall of the duct.

[0042] In one specific implementation, a method for detecting micro-defects in conduits based on artificial intelligence and vision technology, referring to... Figure 1 As shown, it includes the following steps: S1. Image Acquisition: Insert an endoscopic industrial camera into the pipe to be tested, control the endoscopic industrial camera to move and rotate along the pipe axis to capture images, obtaining a sequence of circumferential images of the inner wall of the pipe, as shown in the reference image. Figure 2 As shown.

[0043] The endoscopic industrial camera is set to a resolution of 1280×1080P and a frame rate of 30fps. The circumferential image sequence of the inner wall of the pipe includes images of scratch defects and crack defects.

[0044] S2. Image preprocessing and defect sample construction: The acquired duct inner wall images are denoised, normalized, sized, illuminated, augmented, and labeled with defects to construct a dataset containing defect category labels, bounding box labels, and optional defect mask labels.

[0045] Specifically, the data preprocessing process in step S2 is as follows: S21. Perform preliminary screening on the original images of the inner wall of the catheter acquired in step S1, and remove samples that are severely blurred, overexposed, underexposed, severely obscured, have unidentifiable defect areas, or have failed to acquire images, and retain effective images that can reflect the true surface condition of the inner wall of the catheter. S22. Due to the narrow imaging space inside the duct, the image is easily affected by low illumination, sensor noise, local reflection and texture interference. Therefore, Gaussian filtering is used to denoise the original image. S23. Perform uniform size adjustment and normalization on the denoised image; the normalization process is calculated as follows:

[0046] In the formula, Represents the original image. and These represent the minimum and maximum pixel values ​​in the current image, respectively. This represents the normalized image; Uniform resizing scales an image to a fixed input size. During scaling, the original aspect ratio is maintained and edge padding is used; S24. To address common issues in catheter inner wall images such as uneven brightness, localized reflections, shadows, and low contrast, Gamma correction is applied to the images after size standardization to enhance illumination. S25. Use a labeling format to annotate the location of defects on the inner wall of the conduit with bounding boxes, and record the corresponding defect category labels; the bounding box is represented as... ,in, and Indicates the coordinates of the center of the defect area. and These represent the width and height of the bounding box, respectively, and 'c' represents the defect category label. S26. For samples requiring defect boundary constraints or image-domain PINN physical residual supervision, further construct pixel-level defect masks:

[0047] in, Indicates the area where the defect is located. This represents a defect mask. Correspondingly, the mask for the normal pipe wall region is represented as: Defect masks are used to constrain the defect anomaly field and the defect potential function field, while normal region masks are used to constrain the normal pipe wall apparent field and illumination field to satisfy the smoothness and low-frequency variation law.

[0048] S27. In the data preprocessing stage, data augmentation strategies are introduced, including random flipping, random cropping, brightness perturbation, contrast adjustment, and scale transformation. S28. Divide the preprocessed image data into training set, validation set and test set according to a preset ratio.

[0049] S3. Constructing an improved DETR visual detection branch: The improved DETR visual detection branch includes a convolutional backbone feature extraction module, a multi-scale feature fusion module, a small target perception query initialization module, a Transformer encoder, a Transformer decoder, a classification head, and a bounding box regression head, used to extract multi-scale visual features of the duct inner wall and generate a visually salient response map. For example... Figure 3 As shown.

[0050] Specifically, the process of improving the DETR visual detection branch in step S3 is as follows: S31. Using a convolutional neural network as the backbone feature extraction network, multi-layer visual features are extracted from the duct image:

[0051] in Preserve edge and texture information of minor defects. Provides high-level semantic features, The enhanced image obtained in step S2; S32. Introduce a multi-scale feature pyramid network to fuse the different resolution features output by the backbone feature extraction network to obtain a high-resolution feature map containing details of minute defects:

[0052]

[0053] in, This represents the fused multi-scale feature map; S33, Utilizing high-resolution fusion feature maps Generate visually salient response maps: ,in, This represents the visual response map of suspected defective areas. Then, the area with the highest response value is selected from the visually salient response map. One position: in, This represents the initial set of candidate defect locations for visual analysis, and then for each candidate location... Extract visual features and location encoding to generate an initial visual query vector: ,in, This represents the initial visual query vector. This vector indicates the locations where visual defects may exist, and is used for subsequent physical prior augmentation. S34. Flatten the fused features, add positional encoding, and then input them into the Transformer encoder:

[0054] Self-attention is calculated as follows:

[0055] Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and d represents the feature dimension; S35, Output multi-scale visual features Encoder features Visual saliency response map and visual initial query vector This serves as input for subsequent PINN branches and the physical prior enhancement query module.

[0056] S4. Constructing an image domain physical information neural network branch: The image domain physical information neural network branch takes image pixel coordinates, image grayscale values, and visual features extracted by the improved DETR visual detection branch as input, and outputs the normal pipe wall apparent field, illumination field, defect anomaly field, defect potential function field, and physical residual map. For example... Figure 3 As shown.

[0057] The specific process of constructing the physical information neural network branch in step S4 is as follows: S41. For the preprocessed catheter inner wall image obtained in step S2 Establish a physical information neural network branch in the image pixel coordinate domain, assuming the position of any pixel in the image is: Meanwhile, to facilitate network learning, pixel coordinates are normalized to a fixed range. ,in:

[0058] Represents the original pixel coordinates ， These represent the image width and height, respectively, enabling the physical branch to learn the apparent changes in the inner wall of the duct, changes in illumination, and abnormal distribution of defects in a unified image coordinate space. S42. Perform multi-scale position encoding on the normalized pixel coordinates so that the network can simultaneously perceive large-scale illumination changes and local minor defects. S43. The image pixel coordinates, image grayscale values, and visual features extracted by the improved DETR visual branch are fused to form the input of the physical information neural network branch:

[0059] in Represents pixel coordinate encoding. This represents the grayscale value of the preprocessed catheter inner wall image. This indicates the corresponding positional visual features extracted by the improved DETR visual detection branch; S44. The image domain physical information neural network outputs four types of physical fields based on the above inputs and outputs:

[0060] in, This represents the normal apparent field of the pipe wall, used to describe the normal texture and grayscale distribution that the inner wall of the conduit should have under defect-free conditions; It represents the illumination field and is used to describe the low-frequency brightness changes caused by supplemental lighting, reflection, shadows, and uneven brightness during endoscopic imaging. It represents a defect anomaly field, used to indicate local anomaly regions that cannot be explained by normal pipe wall appearance and changes in illumination; This represents the defect potential function field, used to describe the defect boundary and its geometric continuity; S45. The image of the inner wall of the catheter is considered to be composed of the normal appearance of the catheter wall, changes in illumination, defects and anomalies, and noise. The basic relationship is as follows:

[0061] Based on the above relationships, the image reconstruction residuals are constructed as follows:

[0062] Used to measure whether the normal appearance field, illumination field, and defect / anomaly field output by the physical branch can reasonably explain the input image; S46. Construct apparent smoothing residuals and low-frequency illumination residuals to suppress the model from misclassifying normal textures, reflections, and shadows as defects. For defect boundaries, constrain their boundary continuity through a defect potential function field to make the boundary representation of slender defects such as cracks and scratches more stable. S47. The image reconstruction residual, apparent smoothing residual, low-frequency illumination residual, defect boundary residual, and defect anomaly field gradient are fused to obtain the physical residual map:

[0063] in, Represents the physical residual plot. This indicates the normal apparent residual of the pipe wall. Indicates illumination residual, This represents the residual of the defect potential function. to Learnable weights S5. Physical Prior Enhancement Target Query and Defect Prediction: Based on the visual saliency response map output by the improved DETR visual detection branch and the physical residual map output by the image domain physical information neural network branch, a physical prior enhancement response map is generated. A physical prior enhancement target query vector is then constructed from this physical prior enhancement response map, input into the Transformer decoder, and outputs the duct inner wall defect category, defect bounding box, defect probability map, and physical consistency score. (Example) Figure 3 As shown.

[0064] The specific process of step S5 is as follows: S51. Calculate the gradient magnitude of the enhanced image: It is used to reinforce areas of localized abrupt changes such as cracks, scratches, and corrosion boundaries; S52. Generate a physically prior enhanced response map by fusing visual salient response, physical residual, and image gradient:

[0065] in, To obtain a visually salient response map for S3, Obtain the physical residual map for S4. Then, obtain the physical prior augmentation response map. Select the top response values These locations were selected as the final candidate defect locations. These locations represent highly plausible candidate defect regions that are visually suspicious, physically abnormal, or exhibit abrupt changes at the edges. S53. Fuse the initial visual query from S3 with the physical information to generate the final query vector:

[0066] in, This represents the initial visual query obtained by S3. This represents the physical residual value at the candidate location. This represents the image gradient at the candidate location. This represents the grayscale value of the image at the candidate location; S54. Set the final target query set Global features output by the Transformer encoder Input the Transformer decoder together:

[0067] The decoder searches for corresponding defect regions in the global image features based on these query vectors and outputs target-level features; S55. The classification head outputs the defect category, the bounding box regression head outputs the defect location, and the optional defect probability map prediction head outputs the defect region probability map. Simultaneously, a physical consistency score is calculated based on the mean physical residual within the predicted bounding box region.

[0068] Indicates the first The physical consistency score of each predicted defect is calculated. If a region has a high detection confidence score and a high physical consistency score, it is more likely that the region is a real defect. If the detection confidence score is high but the physical consistency score is low, the region may be affected by reflection, shadow, or noise interference, and further verification is required.

[0069] S6. Physical quality enhancement matching and joint loss training: Hungarian matching is performed based on classification cost, bounding box cost, geometric overlap cost, physical residual cost and defect morphology cost, and the model is trained end-to-end using detection loss, scale-sensitive localization loss, image domain physical residual loss, defect boundary constraint loss and physical prior consistency loss.

[0070] The specific process of step S6 is as follows: S61. During model training, the improved DETR-PINN network outputs multiple candidate detection results for each duct inner wall image, including defect category probability, predicted bounding box, defect probability map, and physical consistency score. To ensure that these prediction results correspond one-to-one with the ground truth annotation results, matching is required first. Traditional DETR mainly performs Hungarian matching based on classification cost, bounding box cost, and geometric overlap cost. This patent further introduces physical residual cost and defect morphology cost, so that the matching process not only focuses on whether the predicted box coincides with the ground truth box, but also on whether the region has physical anomaly features of a real defect and whether the defect morphology is reasonable. The comprehensive matching cost can be expressed as:

[0071] S62. After matching is complete, train the model using the joint loss function. The total loss function can be expressed as:

[0072] S63. Jointly optimize the improved DETR visual detection branch and the image domain PINN physical branch through backpropagation. S7. Model Evaluation and Defect Detection Results Output: The trained model is evaluated using the test set, and mAP, accuracy, precision, recall and physical consistency score are calculated. The defect category, defect location, bounding box, confidence, defect probability map and defect detection report are output.

[0073] The calculation of each indicator is based on the number of true positives, false positives, true negatives, and false negatives in the confusion matrix. The specific correspondences in the confusion matrix are shown in Table 1 below: Table 1. Correspondence of Confusion Matrix

[0074] 1) Accuracy Accuracy is the ratio of correctly classified samples to the total number of samples, and is calculated using the following formula:

[0075] In the formula, Indicates the actual number of cases. Indicates the number of true negative examples. Indicates the number of false positives. This indicates the number of false negatives.

[0076] 2) Accuracy Precision represents the proportion of samples that the model predicts to be damaged, but which are actually damaged. The formula for calculation is:

[0077] In the formula, Indicates the actual number of cases. This indicates the number of false positives.

[0078] 3) Recall rate Recall rate represents the proportion of samples predicted as damaged by the model among all actual damaged samples, and is calculated using the following formula:

[0079] In the formula, Indicates the actual number of cases. This indicates the number of false negatives.

[0080] 4) The steps for calculating the mean precision are as follows: For each damage category (e.g., scratches and cracks), the corresponding precision value is calculated at different recall thresholds. The recall threshold ranges from 0 to 1, in increments of 0.1. The average precision for that category is obtained by summing the precision values ​​obtained at different recall thresholds and dividing by the number of precision values. The formula for calculating average accuracy is:

[0081] In the formula, This represents the precision value calculated at each recall threshold. This indicates the number of precision values ​​corresponding to the selected recall threshold.

[0082] The mean of the mean accuracy is obtained by averaging the average accuracy across all damage categories. When the damage category includes both scratches and cracks, the formula for calculating the average precision is:

[0083] Unlike conventional DETR testing methods, this patent also outputs a physical consistency score. For the first... For each predicted defect, its physical consistency score can be calculated from the mean of the physical residuals within the predicted bounding box:

[0084] in, Indicates the area covered by the predicted bounding box. This represents the physical residual map output by the PINN branch in the image domain. This score is used to measure whether the predicted region simultaneously conforms to both visual and physical anomaly characteristics.

Claims

1. A method for detecting micro-defects in catheters based on artificial intelligence and vision technology, characterized in that, Includes the following steps: S1. Image acquisition inside the conduit: Insert an endoscopic industrial camera into the inside of the pipe to be tested, control the endoscopic industrial camera to move and rotate along the pipe axis to take pictures, and obtain a circumferential image sequence of the inner wall of the pipe. S2. Image preprocessing and defect sample construction: The acquired duct inner wall images are denoised, normalized, sized, illuminated, augmented, and labeled with defects to construct a dataset containing defect category labels, bounding box labels, and optional defect mask labels. S3. Constructing an improved DETR visual detection branch: The improved DETR visual detection branch includes a convolutional backbone feature extraction module, a multi-scale feature fusion module, a small target perception query initialization module, a Transformer encoder, a Transformer decoder, a classification head, and a bounding box regression head, which are used to extract multi-scale visual features of the inner wall of the duct and generate a visually salient response map. S4. Constructing the image domain physical information neural network branch: The image domain physical information neural network branch takes the image pixel coordinates, image gray value and visual features extracted by the improved DETR visual detection branch as input, and outputs the normal pipe wall apparent field, illumination field, defect anomaly field, defect potential function field and physical residual map. S5. Physical Prior Enhancement Target Query and Defect Prediction: Based on the visual saliency response map output by the improved DETR visual detection branch and the physical residual map output by the image domain physical information neural network branch, a physical prior enhancement response map is generated. The physical prior enhancement target query vector is constructed from the physical prior enhancement response map, input into the Transformer decoder, and outputs the duct inner wall defect category, defect bounding box, defect probability map and physical consistency score. S6. Physical quality enhancement matching and joint loss training: Hungarian matching is performed based on classification cost, bounding box cost, geometric overlap cost, physical residual cost and defect morphology cost, and the model is trained end-to-end using detection loss, scale-sensitive localization loss, image domain physical residual loss, defect boundary constraint loss and physical prior consistency loss. S7. Model Evaluation and Defect Detection Results Output: The trained model is evaluated using the test set, and mAP, accuracy, precision, recall, and physical consistency score are calculated. The defect category, defect location, bounding box, confidence score, defect probability map, and defect detection report are output.

2. The catheter micro-defect detection method based on artificial intelligence and vision technology according to claim 1, characterized in that, In step S1, the endoscopic industrial camera has a resolution of 1280×1080P and a frame rate of 30fps; the circumferential image sequence of the inner wall of the pipe includes scratch defect images and crack defect images.

3. The catheter micro-defect detection method based on artificial intelligence and vision technology according to claim 1, characterized in that, Step S2 specifically includes: S21. Perform preliminary screening on the original images of the inner wall of the catheter acquired in step S1, and remove samples that are severely blurred, overexposed, underexposed, severely obscured, have unidentifiable defect areas, or have failed to acquire images, and retain effective images that can reflect the true surface condition of the inner wall of the catheter. S22. Due to the narrow imaging space inside the duct, the image is easily affected by low illumination, sensor noise, local reflection and texture interference. Therefore, Gaussian filtering is used to denoise the original image. S23. Perform uniform size adjustment and normalization on the denoised image; the normalization process is calculated as follows: In the formula, Represents the original image. and These represent the minimum and maximum pixel values ​​in the current image, respectively. This represents the normalized image; Uniform resizing scales an image to a fixed input size. During scaling, the original aspect ratio is maintained and edge padding is used; S24. To address common issues in catheter inner wall images such as uneven brightness, localized reflections, shadows, and low contrast, Gamma correction is applied to the images after size standardization to enhance illumination. S25. Use a labeling format to annotate the location of defects on the inner wall of the conduit with bounding boxes, and record the corresponding defect category labels; the bounding box is represented as... ,in, and Indicates the coordinates of the center of the defect area. and These represent the width and height of the bounding box, respectively, and 'c' represents the defect category label. S26. For samples requiring defect boundary constraints or image-domain PINN physical residual supervision, further construct pixel-level defect masks: in, Indicates the area where the defect is located. This represents a defect mask. Correspondingly, the mask for the normal pipe wall region is represented as: Defect masks are used to constrain the defect anomaly field and the defect potential function field, while normal region masks are used to constrain the normal pipe wall apparent field and illumination field to satisfy the smoothness and low-frequency variation law. 4.S27. In the data preprocessing stage, data augmentation strategies are introduced, including random flipping, random cropping, brightness perturbation, contrast adjustment, and scale transformation. S28. Divide the preprocessed image data into training set, validation set and test set according to a preset ratio.

5. The catheter micro-defect detection method based on artificial intelligence and vision technology according to claim 1, characterized in that, Step S3 specifically includes: S31. Using a convolutional neural network as the backbone feature extraction network, multi-layer visual features are extracted from the duct image: in Preserve edge and texture information of minor defects. Provides high-level semantic features, The enhanced image obtained in step S2; S32. Introduce a multi-scale feature pyramid network to fuse the different resolution features output by the backbone feature extraction network to obtain a high-resolution feature map containing details of minute defects: in, This represents the fused multi-scale feature map; S33, Utilizing high-resolution fusion feature maps Generate visually salient response maps: ,in, This represents the visual response map of suspected defective areas. Then, the area with the highest response value is selected from the visually salient response map. One position: in, This represents the initial set of candidate defect locations for visual analysis, and then for each candidate location... Extract visual features and location encoding to generate an initial visual query vector: ,in, This represents the initial visual query vector. This vector indicates the locations where visual defects may exist, and is used for subsequent physical prior augmentation. S34. Flatten the fused features, add positional encoding, and then input them into the Transformer encoder: Self-attention is calculated as follows: Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and d represents the feature dimension; S35, Output multi-scale visual features Encoder features Visual saliency response map and visual initial query vector This serves as input for subsequent PINN branches and the physical prior enhancement query module.

6. The catheter micro-defect detection method based on artificial intelligence and vision technology according to claim 1, characterized in that, Step S4 specifically includes: S41. For the preprocessed catheter inner wall image obtained in step S2 Establish a physical information neural network branch in the image pixel coordinate domain, assuming the position of any pixel in the image is: Meanwhile, to facilitate network learning, pixel coordinates are normalized to a fixed range. ,in: Represents the original pixel coordinates ， These represent the image width and height, respectively, enabling the physical branch to learn the apparent changes in the inner wall of the duct, changes in illumination, and abnormal distribution of defects in a unified image coordinate space. S42. Perform multi-scale position encoding on the normalized pixel coordinates so that the network can simultaneously perceive large-scale illumination changes and local minor defects. S43. The image pixel coordinates, image grayscale values, and visual features extracted by the improved DETR visual branch are fused to form the input of the physical information neural network branch: in Represents pixel coordinate encoding. This represents the grayscale value of the preprocessed catheter inner wall image. This indicates the corresponding positional visual features extracted by the improved DETR visual detection branch; S44. The image domain physical information neural network outputs four types of physical fields based on the above inputs and outputs: in, This represents the normal apparent field of the pipe wall, used to describe the normal texture and grayscale distribution that the inner wall of the conduit should have under defect-free conditions; It represents the illumination field and is used to describe the low-frequency brightness changes caused by supplemental lighting, reflection, shadows, and uneven brightness during endoscopic imaging. It represents a defect anomaly field, used to indicate local anomaly regions that cannot be explained by normal pipe wall appearance and changes in illumination; This represents the defect potential function field, used to describe the defect boundary and its geometric continuity; S45. The image of the inner wall of the catheter is considered to be composed of the normal appearance of the catheter wall, changes in illumination, defects and anomalies, and noise. The basic relationship is as follows: Based on the above relationships, the image reconstruction residuals are constructed as follows: Used to measure whether the normal appearance field, illumination field, and defect / anomaly field output by the physical branch can reasonably explain the input image; S46. Construct apparent smoothing residuals and low-frequency illumination residuals to suppress the model from misclassifying normal textures, reflections, and shadows as defects. For defect boundaries, constrain their boundary continuity through a defect potential function field to make the boundary representation of slender defects such as cracks and scratches more stable. S47. The image reconstruction residual, apparent smoothing residual, low-frequency illumination residual, defect boundary residual, and defect anomaly field gradient are fused to obtain the physical residual map: in, Represents the physical residual plot. This indicates the normal apparent residual of the pipe wall. Indicates illumination residual, This represents the residual of the defect potential function. to These are learnable weights.

7. The method for detecting micro-defects in catheters based on artificial intelligence and vision technology according to claim 1, characterized in that, Step S5 specifically includes: S51. Calculate the gradient magnitude of the enhanced image: It is used to reinforce areas of localized abrupt changes such as cracks, scratches, and corrosion boundaries; S52. Generate a physically prior enhanced response map by fusing visual salient response, physical residual, and image gradient: in, To obtain a visually salient response map for S3, Obtain the physical residual map for S4. Then, obtain the physical prior augmentation response map. Select the top response values These locations were selected as the final candidate defect locations. These locations represent highly plausible candidate defect regions that are visually suspicious, physically abnormal, or exhibit abrupt changes at the edges. S53. Fuse the initial visual query from S3 with the physical information to generate the final query vector: in, This represents the initial visual query obtained by S3. This represents the physical residual value at the candidate location. This represents the image gradient at the candidate location. This represents the grayscale value of the image at the candidate location; S54. Set the final target query set Global features output by the Transformer encoder Input the Transformer decoder together: The decoder searches for corresponding defect regions in the global image features based on these query vectors and outputs target-level features; S55. The classification head outputs the defect category, the bounding box regression head outputs the defect location, and the optional defect probability map prediction head outputs the defect region probability map. Simultaneously, a physical consistency score is calculated based on the mean physical residual within the predicted bounding box region. Indicates the first The physical consistency score of each predicted defect is calculated. If a region has a high detection confidence score and a high physical consistency score, it is more likely that the region is a real defect. If the detection confidence score is high but the physical consistency score is low, the region may be affected by reflection, shadow, or noise interference, and further verification is required.

8. The method for detecting micro-defects in catheters based on artificial intelligence and vision technology according to claim 1, characterized in that, Step S6 specifically includes: S61. During model training, the improved DETR-PINN network outputs multiple candidate detection results for each duct inner wall image, including defect category probability, predicted bounding box, defect probability map, and physical consistency score. To ensure that these prediction results correspond one-to-one with the ground truth annotation results, matching is required first. Traditional DETR mainly performs Hungarian matching based on classification cost, bounding box cost, and geometric overlap cost. This patent further introduces physical residual cost and defect morphology cost, so that the matching process not only focuses on whether the predicted box coincides with the ground truth box, but also on whether the region has physical anomaly features of a real defect and whether the defect morphology is reasonable. The comprehensive matching cost can be expressed as: S62. After matching is complete, train the model using the joint loss function. The total loss function can be expressed as: S63. Perform joint optimization of the improved DETR visual detection branch and the image domain PINN physical branch through backpropagation.

9. The method for detecting micro-defects in catheters based on artificial intelligence and vision technology according to claim 1, characterized in that, In step S7, the improved DETR-PINN fusion model after training is evaluated using the test set. mAP, accuracy, precision, recall and physical consistency score are calculated, and the defect category, defect location, bounding box, confidence, defect probability map and defect detection report of the inner wall of the catheter are output.

Images