A method, device and equipment for detecting pipe defects based on machine vision

Abstract

The present invention discloses a pipe defect detection method, apparatus, and device based on machine vision. The method comprises: acquiring an image of a pipe to be inspected to obtain the image to be inspected; generating an image embedding of the image to be inspected using a pretrained image encoder; generating a hint embedding of the image to be inspected using a trained hint generation network; the hint embedding is used to indicate the pipe area in the image to be inspected; generating a mask matrix corresponding to the image to be inspected using a pretrained mask decoder based on the image embedding and the hint embedding; determining multiple sets of pipe boundary coordinates based on the mask matrix corresponding to the image to be inspected; and determining defect detection results for the pipe to be inspected based on the multiple sets of pipe boundary coordinates. The present invention does not require a large amount of training data, can improve defect detection capabilities, and significantly saves labor costs.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning technology, and specifically relates to a pipe defect detection method, device and equipment based on machine vision. Background Art

[0002] In the current evolution of healthcare systems, accurate drug delivery not only improves treatment outcomes but also significantly enhances overall system efficiency. Infusion accuracy is essential for effective treatment. For peristaltic pump-driven infusions, infusion accuracy is determined by the quality of the peristaltic pump rubber tubing. The peristaltic pump rubber tubing directly impacts infusion accuracy, infusion pressure, and service life, making it a critical control factor. During the infusion process, liquid is squeezed through the tubing. To meet infusion accuracy requirements, the volume of liquid delivered must be consistent across each segment. The determination of each segment's volume depends on the length and diameter of the tubing, which is determined by the pitch of the peristaltic pump impellers. The pitch of the impellers is fixed for each device, allowing accurate determination of the delivered volume simply by measuring the tubing diameter. Therefore, accurate measurement of the tubing diameter is crucial for ensuring infusion accuracy. Due to the soft nature of peristaltic pump tubing, traditional caliper or contact-based measurement methods are difficult and prone to tubing deformation, compromising measurement accuracy. Furthermore, traditional tools like calipers cannot directly measure the inner diameter and often require cutting the tubing, compromising sample integrity.

[0003] Currently, several non-contact defect detection methods exist, such as ultrasonic defect detection, eddy current defect detection technology, and some deep learning-based defect detection methods. While widely used in industrial inspection, ultrasonic defect detection methods have several limitations. First, they require high surface quality for the inspection object. Rough or contaminated surfaces can affect sound wave propagation, leading to signal distortion. Second, the detection depth of ultrasonic waves is limited by material properties and frequency. Higher-frequency ultrasound waves offer higher resolution but shallower penetration depths, while lower-frequency ultrasound waves have greater penetration but lower resolution. Furthermore, ultrasonic testing is primarily used to identify defects such as cracks and pores, and its sensitivity to certain small or complex defects is low. Furthermore, this method requires high operator expertise, and inexperienced operators may misjudge or miss certain defects. Furthermore, ultrasonic testing equipment is expensive and complex to maintain, and its portability may limit its application in complex environments. Finally, the reflection and refraction of ultrasonic signals are affected by the location and morphology of the defect, and data interpretation requires advanced technical expertise, which increases labor costs and inspection time. While eddy current defect detection technology offers the advantages of being non-destructive, rapid, and efficient, it also has some limitations. First, this technology also has high requirements for the material surface, otherwise the signal accuracy will be affected. Second, eddy current testing is mainly suitable for detecting surface or near-surface defects. It has low sensitivity to deep defects and cannot effectively detect problems within thick materials. It also has certain requirements for users. In addition, the equipment is expensive and requires regular calibration and maintenance, which further increases the cost of use. Deep learning-based defect detection methods, while offering the advantages of efficiency and automation, also have some significant drawbacks. First, deep learning models require a large amount of high-quality annotated data, while defect samples are often scarce and difficult to obtain, resulting in high annotation costs. Second, model training takes a long time and requires high computational resources, especially in the case of large-scale datasets and complex models. Poor data quality or incorrect annotations can affect model accuracy, and overfitting can also reduce the model's generalization ability. Finally, the model's ability to identify complex or rare defects is insufficient, especially in some special working conditions, and its adaptability is poor. Summary of the Invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a pipe defect detection method, device and equipment based on machine vision.

[0005] The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0006] The present invention provides a method for detecting pipe defects based on machine vision, comprising:

[0007] Acquire an image of the tube to be inspected to obtain an image to be inspected;

[0008] generating an image embedding of the image to be inspected using a pre-trained image encoder;

[0009] Using a trained hint generation network to generate a hint embedding of the image to be inspected; the hint embedding is used to indicate the tube body area in the image to be inspected;

[0010] Using a pre-trained mask decoder to generate a mask matrix corresponding to the image to be inspected based on the image embedding and the prompt embedding;

[0011] Determining multiple sets of tube body boundary coordinates according to a mask matrix corresponding to the image to be inspected;

[0012] Based on the multiple sets of pipe body boundary coordinates, a defect detection result of the pipe body to be inspected is determined.

[0013] The present invention also provides a pipe defect detection device based on machine vision, comprising:

[0014] An acquisition module is used to acquire an image of the tube to be inspected and obtain an image to be inspected;

[0015] A boundary coordinate determination module is configured to generate an image embedding of the image to be inspected using a pre-trained image encoder; generate a hint embedding of the image to be inspected using a trained hint generation network; the hint embedding is used to indicate the tube body area in the image to be inspected; generate a mask matrix corresponding to the image to be inspected using a pre-trained mask decoder based on the image embedding and the hint embedding; and determine multiple sets of tube body boundary coordinates based on the mask matrix corresponding to the image to be inspected;

[0016] The defect detection module is used to determine the defect detection result of the pipe to be inspected based on the multiple sets of pipe boundary coordinates.

[0017] The present invention also provides a pipe defect detection device based on machine vision, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0018] The memory is used to store computer programs;

[0019] The processor is used to implement the steps of the above-mentioned pipe defect detection method based on machine vision when executing the program stored in the memory.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The pipe defect detection method of the present invention uses trained and pretrained networks to perform pipe defect detection based on images of the pipe to be inspected. Therefore, it is not affected by the surface quality of the material of the object being inspected, and a rough or stained surface will not affect the final detection results. Defect detection can be performed without the need for professional operation, and the trained model does not require regular maintenance, thereby significantly reducing costs, overcoming the limitation of being unable to detect deep-seated defects, and providing more accurate detection results. Furthermore, when performing pipe defect detection, the present invention uses pretrained image encoders and pretrained mask decoders, eliminating the need for large amounts of training data and addressing the data shortage issue. Furthermore, because the pretrained models have been trained on a large dataset, they have good generalization capabilities, thereby improving defect detection capabilities. Furthermore, subsequent training does not require the retraining of the pretrained models, thereby reducing training time and required computing resources. In addition, when performing pipe body defect detection, the present invention uses a trained prompt generation network to automatically generate prompt embeddings for the image to be inspected. It can automatically generate high-quality prompts for the region of interest to assist the pre-trained mask decoder in accurately generating the mask of the target area. This not only eliminates the need for manual intervention and significantly saves labor costs, but also improves the accuracy of the generated mask matrix, thereby improving the defect detection capability.

[0022] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a pipe defect detection method based on machine vision provided by an embodiment of the present invention;

[0024] Figure 2 is a diagram showing the input and output connections between the image encoder, the hint generation network, and the mask decoder provided by an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of the structure of a prompt generation network provided by an embodiment of the present invention;

[0026] Figure 4 1 is a structural diagram of a mask decoder provided by an embodiment of the present invention;

[0027] Figure 5 1 is a schematic diagram of a set of tube boundary coordinates of a tube to be detected provided by an embodiment of the present invention, and a principle of calculating a corresponding set of index values based on the set of tube boundary coordinates. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below with reference to specific examples, but the embodiments of the present invention are not limited thereto.

[0029] Figure 1 FIG. 1 is a flow chart of a method for detecting tube defects based on machine vision provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0030] S101, acquiring an image of a tube to be inspected, and obtaining an image to be inspected.

[0031] Here, the tube to be tested can be any tube, such as a soft tube, a hard tube, etc., which is not limited in the present invention. For example, the tube to be tested can be a soft tube on a peristaltic pump.

[0032] In some embodiments, the image to be inspected can be a global image of the tube. In some embodiments, the image to be inspected can also be each of multiple local images of the tube, and these multiple local images are spliced along the axial direction of the tube to form a global image of the tube. When the image to be inspected is each of the multiple local images of the tube, the following steps S102 to S106 are the processing steps for each of these multiple local images.

[0033] S102: Generate an image embedding of the image to be inspected using a pre-trained image encoder.

[0034] Here, the image embedding of the image to be inspected is the image feature of the image to be inspected. Exemplarily, the pre-trained image encoder can be a MAE pre-trained visual transformer (ViT), where MAE stands for Masked Autoencoder and ViT stands for Vision Transformer.

[0035] S103, using the trained hint generation network to generate a hint embedding of the image to be inspected; the hint embedding is used to hint the tube body area in the image to be inspected.

[0036] S104 , using a pre-trained mask decoder to generate a mask matrix corresponding to the image to be inspected according to the image embedding and the prompt embedding.

[0037] For example, Figure 2 is a graph of the input and output connections between the image encoder, hint generation network, and mask decoder.

[0038] Here, the number of rows and columns of the mask matrix corresponding to the image to be inspected is the same as the resolution of the image to be inspected, and each element in the mask matrix represents whether a pixel at the same position as the element in the image to be inspected belongs to the tube body or the background. Specifically, the mask matrix is a matrix composed of 0 and 1. In some embodiments, 0 represents that a pixel at the same position as itself in the image to be inspected belongs to the tube body, and 1 represents that a pixel at the same position as itself in the image to be inspected belongs to the background. In other embodiments, 1 represents that a pixel at the same position as itself in the image to be inspected belongs to the tube body, and 0 represents that a pixel at the same position as itself in the image to be inspected belongs to the background.

[0039] S105 , determining multiple sets of tube body boundary coordinates according to the mask matrix corresponding to the image to be inspected.

[0040] S106 : Determine defect detection results of the pipe to be inspected based on the multiple sets of pipe boundary coordinates.

[0041] In some embodiments, the prompt generation network includes: an encoder and a decoder; the input of the encoder serves as the input of the prompt generation network, and the output of the decoder serves as the output of the prompt generation network; the decoder includes at least a first upsampling module and a second upsampling module, the output of the first upsampling module and the output of the encoder are both connected to the input of the second upsampling module, and the output of the second upsampling module serves as the output of the decoder. Exemplarily, the encoder can be a HarmonicDenseNet; the first upsampling module includes, in sequence, a first convolutional layer, a second convolutional layer, a first batch of normalization layers, and a first activation layer; the second upsampling module includes, in sequence, a third convolutional layer, a fourth convolutional layer, a second batch of normalization layers, and a second activation layer, and the parameters of the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer are all the same, the parameters of the first batch of normalization layers and the second batch of normalization layers are the same, and the activation functions used by the first activation layer and the second activation layer are different. Figure 3 It is a schematic diagram of the structure of the prompt generation network. Figure 3The prompt generation network is composed of an encoder and a decoder, and the decoder is composed of a first upsampling module and a second upsampling module connected in series. The first upsampling module is composed of two convolutional layers Cov, each with a kernel size of 3×3 and a zero padding of 1, a batch normalization layer Batch Norm, and a Relu activation layer. The second upsampling module is composed of two convolutional layers Cov, each with a kernel size of 3×3 and a zero padding of 1, a batch normalization layer Batch Norm, and a Tanh activation layer. The output of the encoder and the output of the ReLu activation layer are both connected to the input of the first convolutional layer Cov in the second upsampling module. Both the first upsampling module and the second upsampling module can generate features with a resolution of 64×64 and 256 output channels. It should be noted that in some embodiments, the decoder in the prompt generation network may also include more than two of the above-mentioned upsampling modules, and the structures of the upsampling modules are the same.

[0042] In the present invention, the trained cue generation network is obtained using a binary cross-entropy loss function and a Dice (differentiable image entropy compression) loss function, using a training set containing multiple labeled training samples. Each label is the true mask matrix of the training sample, which represents the tube body region in the training sample. The Dice loss measures the overlap between the predicted mask matrix and the true mask matrix.

[0043] Here, the expressions of the binary cross entropy loss function and the Dice loss function are as follows:

[0044] ;

[0045] ;

[0046] in, represents the binary cross entropy loss for each training sample, represents the Dice loss of each training sample, represents each training sample, represents the hint embedding for each training sample, represents the true mask matrix of each training sample, The mask matrix representing the prediction for each training example, represents the true positive difference between the real mask matrix and the predicted mask matrix for each training sample, represents the false positives between the true mask matrix and the predicted mask matrix for each training sample, Represents the false negatives between the true mask matrix and the predicted mask matrix for each training sample.

[0047] Specifically, the trained prompt generation network is trained using the method described in the following steps S001 to S004:

[0048] S001. Obtain a training set and an initial model respectively; each training sample in the training set is a tube body image; the initial model includes a pre-trained image encoder, an initial prompt generation network, and a pre-trained mask decoder.

[0049] For example, a training set can be created using the following method: a camera is mounted on a stepper motor, which is fixed on a straight axis. A tube sample is placed below the camera, with the extension direction of both ends of the tube sample parallel to the axis. The camera captures a partial image of the tube sample starting from one end, with each captured image being used as an original sample. Each time an original sample is captured, the camera moves forward a certain distance, such as 2 cm, until all positions of the tube sample are captured. This method is used to capture images of one or more tube samples to obtain multiple original samples. After obtaining multiple original samples, each original sample is preprocessed, for example, by sequentially performing denoising, deblurring, and resizing and formatting, thereby obtaining multiple training samples. A mask matrix is then generated for each training sample, and this mask matrix is used as the true mask matrix for the training sample.

[0050] Here, since the prompt generation network consists of an encoder and a decoder, the network parameters of the initial prompt generation network are composed of two parts: the initial parameters of the encoder and the initial parameters of the decoder. For example, the initial parameters of the encoder can be generated using a pre-trained ImageNet network, and the initial parameters of the decoder can be obtained by random generation.

[0051] S002. During each training, select the current training sample from the training set, and input the current training sample into the model obtained in the previous training to obtain the predicted mask matrix of each training sample in the current training sample; the model obtained in the previous training includes the pre-trained image encoder, the hint generation network obtained in the previous training, and the pre-trained mask decoder.

[0052] Here, the number of training samples selected during each training can be set according to actual needs, and the present invention does not limit this.

[0053] S003. Using the binary cross entropy loss function and the Dice loss function, according to the predicted mask matrix of each training sample in this training sample and the true mask matrix of each training sample in this training sample, calculate the binary cross entropy loss and the Dice loss respectively.

[0054] S004. According to the binary cross entropy loss and the Dice loss, the network parameters of the prompt generation network obtained in the previous training are adjusted to obtain the prompt generation network obtained in this training. According to the prompt generation network obtained in this training, the pre-trained image encoder and the pre-trained mask decoder, the model obtained in this training is obtained. The training is iterated until the training termination condition is reached to obtain the trained prompt generation network.

[0055] Here, after adding the calculated binary cross entropy loss and the Dice loss, the total loss of this training can be obtained. According to the total loss of this training, the network parameters of the prompt generation network obtained in the previous training are updated through reverse gradient propagation to obtain the prompt generation network obtained in this training. According to the prompt generation network obtained in this training, the pre-trained image encoder and the pre-trained mask decoder, the model obtained in this training is obtained. According to the above principle, iterative training is performed until the loss obtained is less than or equal to the preset loss or the number of training times reaches the preset number, which indicates that the training termination condition is met, so that the prompt generation network obtained in the last training can be used as the trained prompt generation network. It should be noted that the principle of updating the network parameters through reverse gradient propagation according to the total loss is an existing principle, and the present invention will not elaborate on it.

[0056] In some embodiments, the mask decoder is a decoder designed using a pre-trained Transformer decoder. The main task of the mask decoder is to map the image embedding, the hint embedding, and the output token to the mask space, where the output token is a learnable token specifically used for mask prediction and is used to dynamically generate parameters related to the mask. Specifically, Figure 4As shown in the figure, the mask decoder consists of two decoding layers (Decoding Layer 1 and Decoding Layer 2), an image embedding upsampling module (Upsampling), a multilayer perceptron (MLP), a dynamic linear classifier, and a dot product unit (Dot). The decoding layer is a Transformer decoder. The decoding layer updates the interaction between the prompt embedding (Prompt Encoder Input) through prompt self-attention (Prompt Self-Attention). It also uses cross-attention in two directions (Cross-Attention in Two Directions) to perform bidirectional information exchange between the image embedding (Image Encoder Input) and the prompt embedding to generate a matrix feature (Mask Feature). In this way, the model can maintain a comprehensive understanding of image and prompt information during the decoding process, thereby improving the accuracy of mask prediction. Decoding Layers 1 and 2 update all embeddings and output the updated image embedding and prompt embedding, as well as the output token. After running decoding layer 1 and decoding layer 2, the matrix features output by decoding layer 2 are spatially upsampled through the image embedding upsampling module to restore it to the resolution of the input image. At the same time, the output tokens are processed by the multi-layer perceptron, and the multi-layer perceptron inputs the output to the dynamic linear classifier. The dynamic linear classifier calculates the mask foreground probability of each pixel of the image and outputs the mask foreground probability of each pixel to the dot product operation unit. The dot product operation unit performs a dot product operation on the upsampled matrix features and the mask foreground probability of each pixel of the image, and finally generates a segmentation mask (i.e., a mask matrix Mask).

[0057] In some embodiments, the above S105 is implemented by the following steps:

[0058] S1051 , from the mask matrix corresponding to the image to be inspected, select a row of mask values every Q rows, where Q is a preset positive integer, for example, Q may be 200.

[0059] S1052. For each selected row of mask values, analyze whether every two adjacent mask values in a row of mask values are the same based on the arrangement order of the mask values contained in the row of mask values. If the two adjacent mask values are different, record the position of the first mask value of the two adjacent mask values in the mask matrix corresponding to the image to be inspected as a changed position, thereby obtaining a plurality of changed positions arranged in sequence.

[0060] Here, when the number of rows of the mask matrix is X and the number of columns is Y, the Y mask values of the first row can be selected for the first time, the Y mask values of the 1+Q rows can be selected for the second time, and so on, until the last Y mask values are taken out.

[0061] Specifically, when the mask value changes from 0 to 1 or from 1 to 0, the position corresponding to this 0 or the position corresponding to this 1 is recorded. The recorded position is a change position, and these change points are boundary positions, marking the dividing line between the target area and the background area. Multiple change positions arranged in sequence can be obtained from each row taken out.

[0062] S1053. Obtain a set of tube body boundary coordinates according to the multiple sequentially arranged change positions.

[0063] The multiple sequentially arranged change positions obtained from each row are the important boundary positions of the tube body. Theoretically, four sequentially arranged change positions can be obtained from each row, and these four sequentially arranged change positions are the four boundary positions of the tube body. Therefore, in some embodiments, the first change position among the multiple sequentially arranged change positions can be used as the outer boundary point coordinates of the left tube wall of the tube body; the second change position among the multiple sequentially arranged change positions can be used as the inner boundary point coordinates of the left tube wall of the tube body; the third change position among the multiple sequentially arranged change positions can be used as the inner boundary point coordinates of the right tube wall of the tube body; the fourth change position among the multiple sequentially arranged change positions can be used as the outer boundary point coordinates of the right tube wall of the tube body; the outer boundary point coordinates and inner boundary point coordinates of the left tube wall of the tube body, as well as the outer boundary point coordinates and inner boundary point coordinates of the right tube wall of the tube body, can be used as a set of tube body boundary coordinates. In this way, a set of tube body boundary coordinates can be obtained based on the mask value of a row selected each time.

[0064] S1054: After traversing the mask matrix corresponding to the image to be inspected, multiple sets of tube body boundary coordinates are obtained.

[0065] In some embodiments, the above S106 is implemented by the following steps:

[0066] S1061. Determine a set of index values based on each set of tube body boundary coordinates; wherein the set of index values includes: the length of the outer side of the left tube wall from the boundary, the length of the inner side of the left tube wall from the boundary, the length of the outer side of the right tube wall from the boundary, the length of the inner side of the right tube wall from the boundary, the thickness of the left tube wall, the thickness of the right tube wall, the inner diameter length, and the outer diameter length.

[0067] In some embodiments, a set of index values may also only include: inner diameter length and outer diameter length.

[0068] Specifically, for a set of pipe boundary coordinates, the horizontal coordinate of the outer boundary point of the left pipe wall is used as the length of the outer side of the left pipe wall from the boundary; the horizontal coordinate of the inner boundary point of the left pipe wall is used as the length of the inner side of the left pipe wall from the boundary; the horizontal coordinate of the outer boundary point of the right pipe wall is used as the length of the outer side of the right pipe wall from the boundary; the horizontal coordinate of the inner boundary point of the right pipe wall is used as the length of the inner side of the right pipe wall from the boundary; the horizontal coordinate of the inner boundary point of the left pipe wall is subtracted from the horizontal coordinate of the inner boundary point of the right pipe wall to obtain an inner diameter length; the horizontal coordinate of the outer boundary point of the left pipe wall is subtracted from the horizontal coordinate of the outer boundary point of the right pipe wall to obtain an outer diameter length. The outer diameter length reflects the outer width of the entire pipe body, while the inner diameter length reflects the width of the internal space of the pipe body, that is, the width of the area where liquid or material flows.

[0069] For example, Figure 5 This is a schematic diagram of a set of tube boundary coordinates of a tube to be inspected, and the principle of calculating the length of the outer side of the left tube wall from the boundary, the length of the inner side of the left tube wall from the boundary, the length of the outer side of the right tube wall from the boundary, the length of the inner side of the right tube wall from the boundary, the left tube wall thickness, the right tube wall thickness, the inner diameter length, and the outer diameter length based on this set of tube boundary coordinates, wherein point A is the outer boundary point of the left tube wall of the tube body, point B is the inner boundary point of the left tube wall of the tube body, point C is the inner boundary point of the right tube wall of the tube body, and point D is the outer boundary point of the right tube wall of the tube body.

[0070] S1062. Determine defect detection results of the pipe to be inspected based on multiple groups of index values corresponding to multiple groups of pipe boundary coordinates.

[0071] Specifically, S1062 can be implemented through steps S1 to S5:

[0072] S1. Calculate the mean and standard deviation of the inner diameter lengths in multiple groups of index values corresponding to multiple groups of tube boundary coordinates, and obtain a first mean and a first standard deviation.

[0073] Here, the mean of the inner diameter length among the multiple groups of index values corresponding to the multiple groups of tube body boundary coordinates is calculated, and the obtained mean is recorded as the first mean, and the standard deviation of the inner diameter length among the multiple groups of index values corresponding to the multiple groups of tube body boundary coordinates is calculated, and the obtained standard deviation is recorded as the first standard deviation.

[0074] S2. Calculate the mean and standard deviation of the outer diameter lengths in multiple groups of index values to obtain a second mean and a second standard deviation.

[0075] Here, the mean of the outer diameter length among the multiple groups of index values corresponding one-to-one to the multiple groups of tube body boundary coordinates is calculated, and the obtained mean is recorded as the second mean, and the standard deviation of the outer diameter length among the multiple groups of index values corresponding one-to-one to the multiple groups of tube body boundary coordinates is calculated, and the obtained standard deviation is recorded as the second standard deviation.

[0076] S3. Calculate a first standard deviation percentage based on the first mean and the first standard deviation, and calculate a second standard deviation percentage based on the second mean and the second standard deviation.

[0077] S4. When the first mean satisfies the first preset condition, the second mean satisfies the second preset condition, the first standard deviation satisfies the third preset condition, the second standard deviation satisfies the fourth preset condition, the first standard deviation percentage is within the first preset standard deviation percentage range, the second standard deviation percentage is within the second preset standard deviation percentage range, the length of the left tube wall outer side from the boundary in multiple groups of indicators are all within the first preset boundary value range, the length of the left tube wall inner side from the boundary in multiple groups of indicators are all within the second preset boundary value range, the length of the right tube wall outer side from the boundary in multiple groups of indicators are all within the third preset boundary value range, and the length of the right tube wall inner side from the boundary in multiple groups of indicators are all within the fourth preset boundary value range, a defect detection result is obtained, which indicates that the tube diameter size of the tube body to be inspected is free of defects.

[0078] S5. When at least one of the first mean, the first standard deviation, the second mean, and the second standard deviation does not meet the corresponding preset conditions, or at least one of the first standard deviation percentage and the second standard deviation percentage is not within the corresponding preset standard deviation percentage range, or the lengths of the outer side of the left tube wall from the boundary in multiple groups of indicators are not all within the first preset boundary value range, or the lengths of the inner side of the left tube wall from the boundary in multiple groups of indicators are not all within the second preset boundary value range, or the lengths of the outer side of the right tube wall from the boundary in multiple groups of indicators are not all within the third preset boundary value range, or the lengths of the inner side of the right tube wall from the boundary in multiple groups of indicators are not all within the fourth preset boundary value range, a defect detection result is obtained, indicating that the diameter size of the tube body to be inspected is defective.

[0079] Here, these indicators can be used to evaluate the dimensional consistency and stability of the pipe to be inspected at different positions, thereby providing a more accurate basis for defect detection.

[0080] Here, the first preset condition, the second preset condition, the third preset condition, the fourth preset condition, the first preset standard deviation percentage range, the second preset standard deviation percentage range, the first preset boundary value range, the second preset boundary value range, the third preset boundary value range and the fourth preset boundary value range can all be set according to actual needs, and the present invention is not limited to this.

[0081] In some embodiments, the first preset condition is a preset first mean threshold, the second preset condition is a preset second mean threshold, the third preset condition is a preset first standard deviation threshold, and the fourth preset condition is a preset second standard deviation threshold. Accordingly, when the first mean is less than or equal to the preset first mean threshold, the second mean is less than or equal to the preset second mean threshold, the first standard deviation is less than or equal to the preset first standard deviation threshold, and the second standard deviation is less than or equal to the preset second standard deviation threshold, a defect detection result is obtained, indicating that the tube diameter size of the tube body to be inspected is free of defects; otherwise, a defect detection result is obtained, indicating that the tube diameter size of the tube body to be inspected is defective.

[0082] In some embodiments, the first preset condition is a preset first numerical range, the second preset condition is a preset second numerical range, the third preset condition is a preset third numerical range, and the fourth preset condition is a preset fourth numerical range. Accordingly, when the first mean belongs to the preset first numerical range, the second mean belongs to the preset second numerical range, the first standard deviation belongs to the preset third numerical range, and the second standard deviation belongs to the preset fourth numerical range, it indicates that the inner diameter and outer diameter of the tube body to be tested are qualified.

[0083] In some embodiments, after obtaining the defect detection results of each image to be inspected, multiple groups of indicators of the image to be inspected, as well as the first mean, first standard deviation, second mean and second standard deviation can be organized into a table, and the defect detection results of the image to be inspected and the table can be output together.

[0084] The pipe defect detection method of the present invention uses trained and pretrained networks to perform pipe defect detection based on images of the pipe to be inspected. Therefore, it is not affected by the surface quality of the material of the object being inspected, and a rough or stained surface will not affect the final detection results. Defect detection can be performed without the need for professional operation, and the trained model does not require regular maintenance, thereby significantly reducing costs, overcoming the limitation of being unable to detect deep-seated defects, and providing more accurate detection results. Furthermore, when performing pipe defect detection, the present invention uses pretrained image encoders and pretrained mask decoders, eliminating the need for large amounts of training data and addressing the data shortage issue. Furthermore, because the pretrained models have been trained on a large dataset, they have good generalization capabilities, thereby improving defect detection capabilities. Furthermore, subsequent training does not require the retraining of the pretrained models, thereby reducing training time and required computing resources. In addition, when performing pipe body defect detection, the present invention uses a trained prompt generation network to automatically generate prompt embeddings for the image to be inspected. It can automatically generate high-quality prompts for the region of interest to assist the pre-trained mask decoder in accurately generating the mask of the target area. This not only eliminates the need for manual prompt input, significantly saving labor costs, but also improves the accuracy of the generated mask matrix, thereby improving defect detection capabilities.

[0085] The present invention also provides a pipe defect detection device based on machine vision, comprising:

[0086] An acquisition module is used to acquire an image of the tube to be inspected and obtain an image to be inspected;

[0087] The boundary coordinate determination module is used to generate an image embedding of the image to be inspected using a pre-trained image encoder; generate a hint embedding of the image to be inspected using a trained hint generation network; the hint embedding is used to indicate the tube body area in the image to be inspected; use a pre-trained mask decoder to generate a mask matrix corresponding to the image to be inspected based on the image embedding and the hint embedding; and determine multiple sets of tube body boundary coordinates based on the mask matrix corresponding to the image to be inspected;

[0088] The defect detection module is used to determine the defect detection results of the pipe to be inspected based on multiple sets of pipe boundary coordinates.

[0089] It should be noted that the specific execution principle of each of the above modules has been described in detail in the above-mentioned tube defect detection method based on machine vision, and will not be repeated here.

[0090] The present invention also provides a pipe defect detection device based on machine vision, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store computer programs; the processor is used to implement the steps of the above-mentioned pipe defect detection method based on machine vision when executing the program stored in the memory.

[0091] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, features defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0092] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification.

[0093] In the specification, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple situations. Certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0094] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A method for detecting tube defects based on machine vision, characterized in that: include: Acquire an image of the tube to be inspected to obtain an image to be inspected; generating an image embedding of the image to be inspected using a pre-trained image encoder; Using a trained hint generation network to generate a hint embedding of the image to be inspected; the hint embedding is used to indicate the tube body area in the image to be inspected; A pre-trained mask decoder is used to generate a mask matrix corresponding to the image to be inspected based on the image embedding and the prompt embedding; the number of rows and columns of the mask matrix corresponding to the image to be inspected is the same as the resolution of the image to be inspected, and each element in the mask matrix corresponding to the image to be inspected is used to represent whether a pixel in the image to be inspected that is located at the same position as the element belongs to the tube body or the background; From the mask matrix corresponding to the image to be inspected, a row of mask values is selected every Q rows, where Q is a preset positive integer; For each selected row of mask values, analyzing whether every two adjacent mask values in the row of mask values are the same based on the arrangement order of the mask values contained in the row of mask values, and if two adjacent mask values are different, recording the position of the first mask value of the two adjacent mask values in the mask matrix corresponding to the image to be inspected as a changed position, thereby obtaining a plurality of sequentially arranged changed positions; Obtaining a set of tube body boundary coordinates according to the plurality of sequentially arranged change positions; After traversing the mask matrix corresponding to the image to be inspected, multiple sets of tube body boundary coordinates are obtained; Based on the multiple sets of pipe body boundary coordinates, a defect detection result of the pipe body to be inspected is determined.

2. The method for detecting tube defects based on machine vision according to claim 1, characterized in that: The trained prompt generation network is obtained by training using a binary cross entropy loss function, a Dice loss function and a training set, wherein the training set contains multiple labeled training samples, the label of each training sample is the true mask matrix of the training sample, and the true mask matrix represents the tube body area in the training sample.

3. The method for detecting tube defects based on machine vision according to claim 2, characterized in that: The trained prompt generation network is trained using the following method: The training set and the initial model are obtained respectively; each training sample in the training set is a tube body image; the initial model includes the pre-trained image encoder, the initial hint generation network and the pre-trained mask decoder; During each training session, a training sample is selected from the training set and input into the model obtained from the previous training session to obtain a predicted mask matrix for each training sample in the training session; the model obtained from the previous training session includes the pre-trained image encoder, the cue generation network obtained from the previous training session, and the pre-trained mask decoder; Using the binary cross entropy loss function and the Dice loss function, according to the predicted mask matrix of each training sample in the current training sample and the true mask matrix of each training sample in the current training sample, respectively calculate the current binary cross entropy loss and the current Dice loss; According to the binary cross entropy loss and the Dice loss, the network parameters of the prompt generation network obtained in the previous training are adjusted to obtain the prompt generation network obtained in this training. According to the prompt generation network obtained in this training, the pre-trained image encoder and the pre-trained mask decoder, the model obtained in this training is obtained. The training is iterated in this way until the training termination condition is reached, and the trained prompt generation network is obtained.

4. The method for detecting tube defects based on machine vision according to claim 2, characterized in that: The expressions of the binary cross entropy loss function and the Dice loss function are as follows: ； ； in, represents the binary cross entropy loss for each training sample, represents the Dice loss of each training sample, represents each training sample, represents the hint embedding for each training sample, represents the true mask matrix of each training sample, The mask matrix representing the prediction for each training example, represents the true positive difference between the real mask matrix and the predicted mask matrix for each training sample, represents the false positives between the true mask matrix and the predicted mask matrix for each training sample, Represents the false negatives between the true mask matrix and the predicted mask matrix for each training sample.

5. The method for detecting tube defects based on machine vision according to claim 1, characterized in that: The prompt generation network includes: an encoder and a decoder; the input of the encoder serves as the input of the prompt generation network, and the output of the decoder serves as the output of the prompt generation network; the decoder includes a first upsampling module and a second upsampling module, the output of the first upsampling module and the output of the encoder are both connected to the input of the second upsampling module, and the output of the second upsampling module serves as the output of the decoder.

6. The method for detecting tube defects based on machine vision according to claim 5, characterized in that: The encoder is Harmonic DenseNet; The first upsampling module includes, in sequence: a first convolutional layer, a second convolutional layer, a first batch of normalization layers, and a first activation layer; the second upsampling module includes, in sequence: a third convolutional layer, a fourth convolutional layer, a second batch of normalization layers, and a second activation layer, and the parameters of the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer are the same, the parameters of the first batch of normalization layers and the second batch of normalization layers are the same, and the activation functions used by the first activation layer and the second activation layer are different.

7. The method for detecting tube defects based on machine vision according to claim 1, characterized in that: The step of determining the defect detection result of the pipe to be inspected based on the multiple sets of pipe boundary coordinates includes: Determine a set of index values based on each set of tube body boundary coordinates; wherein the set of index values includes: the length of the outer side of the left tube wall from the boundary, the length of the inner side of the left tube wall from the boundary, the length of the outer side of the right tube wall from the boundary, the length of the inner side of the right tube wall from the boundary, the thickness of the left tube wall, the thickness of the right tube wall, the inner diameter, and the outer diameter; Based on the multiple groups of index values corresponding one-to-one to the multiple groups of pipe body boundary coordinates, the defect detection result of the pipe body to be inspected is determined.

8. A pipe defect detection device based on machine vision, characterized in that: include: An acquisition module is used to acquire an image of the tube to be inspected and obtain an image to be inspected; The boundary coordinate determination module is used to generate the image embedding of the image to be inspected by using a pre-trained image encoder; generate the hint embedding of the image to be inspected by using a trained hint generation network; the hint embedding is used to indicate the tube body area in the image to be inspected; use the pre-trained mask decoder to generate the mask matrix corresponding to the image to be inspected according to the image embedding and the hint embedding; the number of rows and columns of the mask matrix corresponding to the image to be inspected is the same as the resolution of the image to be inspected, and each element in the mask matrix corresponding to the image to be inspected is used to represent whether a pixel in the image to be inspected that is at the same position as the element belongs to the tube body or the background; In the corresponding mask matrix, a row of mask values is selected every Q rows, where Q is a preset positive integer; for each selected row of mask values, whether every two adjacent mask values in the row of mask values are the same is analyzed based on the arrangement order of the mask values contained in the row of mask values, and if the two adjacent mask values are different, the position of the first mask value of the two adjacent mask values in the mask matrix corresponding to the image to be inspected is recorded as a changed position, thereby obtaining a plurality of sequentially arranged changed positions; a set of tube body boundary coordinates is obtained based on the plurality of sequentially arranged changed positions; after traversing the mask matrix corresponding to the image to be inspected, a plurality of sets of tube body boundary coordinates are obtained; The defect detection module is used to determine the defect detection result of the pipe to be inspected based on the multiple sets of pipe boundary coordinates.

9. A pipe defect detection device based on machine vision, comprising a processor, a communication interface, a memory and a communication bus, characterized in that: The processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is configured to implement the method steps described in any one of claims 1 to 7 when executing a program stored in the memory.

Images