Pipe defect detection method and device based on grayscale image and depth data fusion
By combining grayscale images and depth data, fitting the pipe surface profile equation and identifying arc surface defects, the problems of inaccurate and unclear detection in the prior art are solved, and more accurate and intelligent pipe defect detection is achieved.
Patent Information
- Application Number
- CN202210203782.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-02
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-03-02
AI Technical Summary
The existing pipe surface defect detection methods rely on two-dimensional grayscale images, and cannot accurately identify the depth and severity of defects. The arc-shaped surface increases the detection difficulty, resulting in inaccurate and unclear detection.
Using a fusion method based on grayscale image and depth data, surface depth data and grayscale images are obtained through a 3D camera, outline equations are fitted, and defect candidate areas are matched with depth and grayscale features, and defect location, size, category and depth.
It improves the accuracy and reliability of defect detection, can quantify the severity of defects, improves the level of intelligent detection, reduces labor costs, and provides guarantees for intelligent product grading.
Smart Images

Figure CN114677331B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of machine vision detection, and in particular to a method and device for detecting pipe defects based on grayscale image and depth data fusion. Background Art
[0002] Surface defects of pipe products are key factors in determining their quality characteristics. Effective defect detection is crucial to achieving product grading and reducing customer complaints. At present, the commonly used surface defect detection methods mostly rely on industrial cameras to collect two-dimensional grayscale images of the surface and use specific detection algorithms to locate and identify defects. This method only uses the reflection characteristics of light to conduct qualitative analysis of defects, and lacks consideration of the depth dimension of the defects. Therefore, the identified defects will have practical problems such as inaccurate detection and unclear severity.
[0003] The surface profile of the pipe is an arc shape, which is different from traditional hot-rolled strip steel, medium and thick plates and other products. The irregular surface profile makes defect detection more difficult, resulting in the existing detection equipment being insufficient to support intelligent inspection tasks. In addition, although the depth information can be detected by the distance sensor, due to the existence of the arc surface, it is impossible to directly confirm the depth of the defect on the pipe surface and cannot be directly used for the depth measurement of the defect. Summary of the invention
[0004] The present invention is proposed to address the problems that defects identified by the prior art may be inaccurately detected, unclear in severity, and irregular surface contours make defect detection more difficult, resulting in existing detection equipment being insufficient to support intelligent inspection tasks.
[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0006] On the one hand, the present invention provides a pipe defect detection method based on grayscale image and depth data fusion, the method is implemented by an electronic device, and the method includes:
[0007] S1. Obtaining surface feature information of a sample to be detected; wherein the surface feature information includes depth data and a grayscale image.
[0008] S2. Based on the depth data, the defect candidate area with depth information is obtained
[0009] S3. Based on the grayscale image and the trained target detection model, the defect candidate area with grayscale features is obtained.
[0010] S4: Defect candidate area with depth information And defect candidate areas with grayscale features Fusion matching is performed to obtain defect detection results; wherein the defect detection results include the location coordinates, size, defect category and defect depth of the defect; the defect category includes primary defect category and secondary defect category.
[0011] Optionally, obtaining surface feature information of the sample to be detected in S1 includes:
[0012] The 3D camera is used to collect the depth data and grayscale image of the sample to be detected.
[0013] Depth data includes the three-dimensional characteristics of the surface of the sample to be inspected.
[0014] Grayscale images include the color, texture, and morphological characteristics of the sample to be detected.
[0015] Optionally, obtaining the defect candidate area with depth information according to the depth data in S2 includes:
[0016] According to the depth data, the contour equation of the surface of the sample to be inspected is fitted, and the depth data is projected with the fitted surface contour equation as the reference plane to obtain the surface relative depth feature. The surface relative depth feature is subjected to threshold segmentation processing to obtain the defect candidate area with depth information.
[0017] The defect candidate area with depth information is an area exceeding a specified depth size.
[0018] Optionally, the profile equation of the surface of the sample to be detected is as shown in the following formula (1):
[0019] Ax 2 +Bz 2 +Cxz+Dx+Ez+F=0 (1)
[0020] Where A, B, C, D, E, and F are the parameters of the profile equation; x is the lateral position distribution coordinate in the depth data; and z is the surface depth value of the sample to be detected in the depth data.
[0021] Optionally, the fitting process of the profile equation parameters includes:
[0022] The depth data is processed line by line, and the valid data in each line of depth data is used as the sampling point data of the surface contour. The sampling point data is fitted by the RANSAC algorithm to obtain the contour equation parameters.
[0023] The valid data is the depth value from the surface of the sample to be detected, which is data greater than 0.
[0024] Optionally, the depth data is projected using the contour equation of the fitted surface as a reference plane to obtain the surface relative depth features including:
[0025] The x-direction sampling point X(x 1 ,x 2 ,...x w ), and then input them into the surface profile equation to obtain the reference surface depth value Z′(z′ 1 ,z′ 2 ,...z′ w ), the relative depth feature of the projected surface is calculated by the following formula (2):
[0026]
[0027] in, is the relative depth feature of the projected surface; Z is the surface depth value of the pipe at the sampling point x; w is the number of valid sampling point data.
[0028] Optionally, threshold segmentation processing is performed on the surface relative depth feature to obtain defect candidate areas with depth information including:
[0029] Independent connected areas whose absolute values of surface relative depth features are greater than a preset detection threshold are selected as defect candidate areas with depth information.
[0030] The defect candidate region with depth information includes the position coordinates, size and defect depth of the defect candidate region.
[0031] Optionally, in S3, defect candidate regions with grayscale features are obtained according to the grayscale image and the trained target detection model, including:
[0032] The grayscale image is input into the trained object detection model to obtain the defect candidate area with grayscale features.
[0033] The training data for the object detection model are real defect images.
[0034] The defect candidate region with grayscale features includes the position coordinates, size, and defect category of the defect candidate region.
[0035] Optionally, the defect candidate region with depth information in S4 And defect candidate areas with grayscale features After fusion matching, the defect detection results include:
[0036] Defect candidate regions with depth information The first-level defect category is deep defect.
[0037] Go through them one by one If the defect candidate area in the defect candidate area with grayscale features In, there is The defect candidate area corresponding to the defect candidate area position in The corresponding defect candidate area category identified in is Secondary defect categories of defect candidate areas in .
[0038] If in the defect candidate area with grayscale features There is no The defect candidate area corresponding to the defect candidate area position in The secondary defect category of the defect candidate area in is to be classified.
[0039] Not with The first-level defect category of the matched defect candidate area is color difference defect, the defect depth value is 0, and the second-level defect category is The defect categories identified in .
[0040] On the other hand, the present invention provides a pipe defect detection device based on grayscale image and depth data fusion, which is used to implement a pipe defect detection method based on grayscale image and depth data fusion, and the device includes:
[0041] The acquisition module is used to acquire the surface feature information of the sample to be detected; wherein the surface feature information includes depth data and grayscale image.
[0042] The deep defect candidate area module is used to obtain defect candidate areas with depth information based on depth data.
[0043] Grayscale defect candidate region module, used to obtain defect candidate regions with grayscale features based on grayscale images and trained target detection models
[0044] Output module, used to output defect candidate areas with depth information And defect candidate areas with grayscale features Fusion matching is performed to obtain defect detection results; wherein the defect detection results include the location coordinates, size, defect category and defect depth of the defect; the defect category includes primary defect category and secondary defect category.
[0045] Optionally, the acquisition module is further used to:
[0046] The 3D camera is used to collect the depth data and grayscale image of the sample to be detected.
[0047] Depth data includes the three-dimensional characteristics of the surface of the sample to be inspected.
[0048] Grayscale images include the color, texture, and morphological characteristics of the sample to be detected.
[0049] Optionally, the deep defect candidate region module is further used to:
[0050] According to the depth data, the contour equation of the surface of the sample to be inspected is fitted, and the depth data is projected with the fitted surface contour equation as the reference plane to obtain the surface relative depth feature. The surface relative depth feature is subjected to threshold segmentation processing to obtain the defect candidate area with depth information.
[0051] The defect candidate area with depth information is an area exceeding a specified depth size.
[0052] Optionally, the profile equation of the surface of the sample to be detected is as shown in the following formula (1):
[0053] Ax 2 +Bz 2 +Cxz+Dx+Ez+F=0 (1)
[0054] Where A, B, C, D, E, and F are the parameters of the profile equation; x is the lateral position distribution coordinate in the depth data; and z is the surface depth value of the sample to be detected in the depth data.
[0055] Optionally, the deep defect candidate region module is further used to:
[0056] The depth data is processed line by line, and the valid data in each line of depth data is used as the sampling point data of the surface contour. The sampling point data is fitted by the RANSAC algorithm to obtain the contour equation parameters.
[0057] The valid data is the depth value from the surface of the sample to be detected, which is data greater than 0.
[0058] Optionally, the deep defect candidate region module is further used to:
[0059] The x-direction sampling point X(x 1 ,x 2 ,...x w ), and then input them into the surface profile equation to obtain the reference surface depth value Z′(z′ 1 ,z′ 2 ,...z′ w ), the relative depth feature of the projected surface is calculated by the following formula (2):
[0060]
[0061] in, is the relative depth feature of the projected surface; Z is the surface depth value of the pipe at the sampling point x; w is the number of valid sampling point data.
[0062] Optionally, the deep defect candidate region module is further used to:
[0063] Independent connected areas whose absolute values of surface relative depth features are greater than a preset detection threshold are selected as defect candidate areas with depth information.
[0064] The defect candidate region with depth information includes the position coordinates, size and defect depth of the defect candidate region.
[0065] Optionally, the grayscale defect candidate region module is further used to:
[0066] The grayscale image is input into the trained object detection model to obtain the defect candidate area with grayscale features.
[0067] The training data for the object detection model are real defect images.
[0068] The defect candidate region with grayscale features includes the position coordinates, size, and defect category of the defect candidate region.
[0069] Optionally, the output module is further configured to:
[0070] Defect candidate regions with depth information The first-level defect category is deep defect.
[0071] Go through them one by one If the defect candidate area in the defect candidate area with grayscale features In, there is The defect candidate area corresponding to the defect candidate area position in The corresponding defect candidate area category identified in is Secondary defect categories of defect candidate areas in .
[0072] If in the defect candidate area with grayscale features There is no The defect candidate area corresponding to the defect candidate area position in The secondary defect category of the defect candidate area in is to be classified.
[0073] Not with The first-level defect category of the matched defect candidate area is color difference defect, the defect depth value is 0, and the second-level defect category is The defect categories identified in .
[0074] On the one hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned pipe defect detection method based on grayscale image and depth data fusion.
[0075] On the one hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned pipe defect detection method based on grayscale image and depth data fusion.
[0076] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0077] In the above scheme, in view of the existing problems in the current surface defect detection of pipes, on the basis of the general grayscale image defect detection method, the grayscale image obtained by the 3D camera is fused with the depth data for detection, which can not only obtain a two-dimensional grayscale image for defect target detection, but also output the three-dimensional depth features of the surface synchronously. The surface contour equation of the pipe is obtained by fitting the depth data, and the depth data is projected onto the contour reference plane to obtain the relative depth information of its surface. Finally, the defect category is defined by multi-level labels by combining the candidate areas extracted from the depth data and the candidate areas identified on the grayscale image, and finally the quantified defect detection results are output. The problem of the inability to quantify the severity of defects caused by the simple use of grayscale images is avoided. The added depth dimension features significantly improve the accuracy of defect judgment, making defect detection more intelligent, and providing strong support for reducing labor costs and intelligent product grading. This method effectively improves the accuracy of defect detection, and the detected defect data is more reliable, providing the greatest guarantee for intelligent product grading. BRIEF DESCRIPTION OF THE DRAWINGS
[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0079] Figure 1 It is a schematic flow chart of a pipe defect detection method based on grayscale image and depth data fusion provided by an embodiment of the present invention;
[0080] Figure 2 It is a schematic diagram of obtaining the relative depth of the pipe surface provided by an embodiment of the present invention;
[0081] Figure 3is a diagram of the defect candidate region fusion process under two types of data detection provided by an embodiment of the present invention;
[0082] Figure 4 It is a block diagram of a pipe defect detection device based on grayscale image and depth data fusion provided by an embodiment of the present invention;
[0083] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0084] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.
[0085] like Figure 1 As shown, an embodiment of the present invention provides a pipe defect detection method based on grayscale image and depth data fusion, which can be implemented by an electronic device. Figure 1 The process flow chart of the pipe defect detection method based on grayscale image and depth data fusion is shown. The process flow of the method may include the following steps:
[0086] S1. Obtaining surface feature information of the sample to be tested.
[0087] The surface feature information includes depth data and grayscale images.
[0088] Optionally, obtaining surface feature information of the sample to be detected in S1 includes:
[0089] The 3D camera is used to collect the depth data and grayscale image of the sample to be detected.
[0090] Depth data includes the three-dimensional characteristics of the surface of the sample to be inspected.
[0091] Grayscale images include the color, texture, and morphological characteristics of the sample to be detected.
[0092] In a feasible implementation, six 3D cameras can be used to collect surface feature information of the pipe in a circular distribution. The surface feature information of the pipe can include grayscale images with characteristics such as color, texture, and shape, as well as depth data with three-dimensional surface characteristics.
[0093] Optionally, a color filter may be added to the camera to shield interference from ambient light.
[0094] The 3D camera can simultaneously collect grayscale images and depth data, and the two data sizes and shooting positions are consistent. The grayscale image size is 2560*1080, and the data range is [0,255]; the depth data matrix size is 2560*1080, the data is based on the camera target surface as the zero point, and the distance from the surface of the photographed pipe to the camera target surface is the depth value. The measured depth accuracy is 0.1mm.
[0095] S2. Based on the depth data, the defect candidate area with depth information is obtained
[0096] Optionally, obtaining the defect candidate area with depth information according to the depth data in S2 includes:
[0097] S21. According to the depth data, a profile equation of the surface of the sample to be detected is obtained by fitting.
[0098] Optionally, the profile equation of the surface of the sample to be detected is as shown in the following formula (1):
[0099] Ax 2 +Bz 2 +Cxz+Dx+Ez+F=0 (1)
[0100] Where A, B, C, D, E, and F are the parameters of the profile equation; x is the lateral position distribution coordinate in the depth data; and z is the surface depth value of the sample to be detected in the depth data.
[0101] Optionally, the fitting process of the profile equation parameters includes:
[0102] The depth data is processed line by line, and the valid data in each line of depth data is used as the sampling point data of the surface contour. The sampling point data is fitted by the RANSAC algorithm to obtain the contour equation parameters.
[0103] The valid data is the depth value from the surface of the sample to be detected, which is data greater than 0.
[0104] In a feasible implementation, Figure 2 As shown, the fitting process of the equation can be to process the depth data line by line, and each line of depth data can be used as the sampling point data of the surface profile. The number of sampling points is 2560. The depth value of the area where the pipe is not photographed is -1000000, and the depth value of the area where the pipe is photographed is greater than 0. Take the valid data of the pipe area as the sampling point, where X(x 1 ,x 2 ,...x w ) is the sampling point of the transverse position distribution coordinates on the pipe surface, Z(z 1 ,z 2 ,...zw ) is the surface depth value of the pipe at the corresponding point x, w is the number of valid data points, and the sampling point data is fitted by the RANSAC (Random Sample Consensus) algorithm to obtain the profile equation parameters A, B, C, D, E, and F.
[0105] During the RANSAC algorithm fitting process, 16 sampling points are randomly selected from w valid data points for fitting each time, and the best fitting parameters are obtained by iterating 100 rounds in sequence.
[0106] S22. Projecting the depth data using the contour equation of the fitted surface as a reference plane to obtain a relative depth feature of the surface.
[0107] Optionally, the depth data is projected using the contour equation of the fitted surface as a reference plane to obtain the surface relative depth features including:
[0108] The x-direction sampling point X(x 1 ,x 2 ,...x w ), and then input them into the surface profile equation to obtain the reference surface depth value Z′(z′ 1 ,z′ 2 ,...z′ w ), the relative depth feature of the projected surface is calculated by the following formula (2):
[0109]
[0110] in, is the relative depth feature of the projected surface; Z is the surface depth value of the pipe at the sampling point x; w is the number of valid sampling point data.
[0111] In a feasible implementation manner, the invalid data is a specific negative number set by the camera.
[0112] S23, performing threshold segmentation processing on the surface relative depth feature to obtain defect candidate areas with depth information.
[0113] The defect candidate area with depth information is an area exceeding a specified depth size.
[0114] Optionally, threshold segmentation processing is performed on the surface relative depth feature to obtain defect candidate areas with depth information including:
[0115] Independent connected areas whose absolute values of surface relative depth features are greater than a preset detection threshold are selected as defect candidate areas with depth information.
[0116] The defect candidate region with depth information includes the position coordinates, size and defect depth of the defect candidate region.
[0117] In one possible implementation, the surface relative depth information The detection threshold is set according to actual needs. For example, the detection threshold is set to 0.25 mm, and the relative depth value is The independent connected regions whose absolute value is greater than the detection threshold are defined as defect candidate regions. It contains the location coordinates, size and defect depth of the candidate area.
[0118] S3. Based on the grayscale image and the trained target detection model, the defect candidate area with grayscale features is obtained.
[0119] Optionally, in S3, defect candidate regions with grayscale features are obtained according to the grayscale image and the trained target detection model, including:
[0120] The grayscale image is input into the trained object detection model to obtain the defect candidate area with grayscale features.
[0121] The training data for the object detection model are real defect images.
[0122] The defect candidate region with grayscale features includes the position coordinates, size, and defect category of the defect candidate region.
[0123] In a feasible implementation, a target detection model is used on a grayscale image to identify defects. The model may be selected but is not limited to a Yolov5 model, a RetinaNet model, or an EfficientDet model. The model is trained and learned using pre-collected real defect image samples.
[0124] The sample data includes 9 categories: folding, scarring, tearing, scratching, denting, delamination, dirt, iron oxide, and mimeographing. The number of samples in each category is more than 1,000. The sample set is divided into training set, validation set, and test set in a ratio of 8:1:1 to train the model and verify the model indicators. The training process can use flipping, cropping, scaling, mixing, adding noise and other methods to amplify the samples.
[0125] After the grayscale image is input into the trained model, the identified defect candidate areas can be output It not only contains the location coordinates and size of the candidate area, but also the corresponding defect category.
[0126] S4: Defect candidate area with depth information And defect candidate areas with grayscale features Fusion matching is performed to obtain defect detection results; wherein the defect detection results include the location coordinates, size, defect category and defect depth of the defect; the defect category includes primary defect category and secondary defect category.
[0127] Optionally, the defect candidate region with depth information in S4 And defect candidate areas with grayscale features After fusion matching, the defect detection results include:
[0128] Defect candidate regions with depth information The first-level defect category is deep defect.
[0129] Go through them one by one If the defect candidate area in the defect candidate area with grayscale features In, there is The defect candidate area corresponding to the defect candidate area position in The corresponding defect candidate area category identified in is Secondary defect categories of defect candidate areas in .
[0130] If in the defect candidate area with grayscale features There is no The defect candidate area corresponding to the defect candidate area position in The secondary defect category of the defect candidate area in is to be classified.
[0131] Not with The first-level defect category of the matched defect candidate area is color difference defect, the defect depth value is 0, and the second-level defect category is The defect categories identified in .
[0132] In a feasible implementation, Figure 3 As shown in the figure, the categories corresponding to the defect candidate boxes are represented by multi-level labels. The final category definition rules are based on the following steps: traverse one by one The defect candidate area in the first level is defined as a deep defect. Find the defect candidate area corresponding to its position, then The categories identified in are used as the secondary categories of the defect candidate area (including folding, scarring, tearing, scratching, denting, and delamination). If no defect candidate area corresponding to its position is found in the , the secondary category of the defect candidate area is defined as to be classified; then traverse Not with The matched defect candidate area has its primary category defined as color difference defect, defect depth value is 0, and its secondary category (including dirt, iron oxide, and oil print) is The category identified in the above analysis is used as the basis; finally, the complete information of the surface defects of the pipe (position coordinates, size, defect category, defect depth) is obtained.
[0133] In the embodiment of the present invention, in view of the problems existing in the current surface defect detection of pipes, on the basis of the general grayscale image defect detection means, the grayscale image obtained by the 3D camera is fused with the depth data for detection, which can not only obtain a two-dimensional grayscale image for defect target detection, but also synchronously output the three-dimensional depth features of the surface. The pipe surface contour equation is obtained by fitting the depth data, and the depth data is projected onto the contour reference plane to obtain the relative depth information of its surface. Finally, the defect category is defined by multi-level labels in combination with the candidate area extracted from the depth data and the candidate area identified on the grayscale image, and the quantified defect detection results are finally output. The problem of the inability to quantify the severity of defects caused by the simple use of grayscale images is avoided. The added depth dimension features significantly improve the accuracy of defect judgment, so that defect detection is promoted to a more intelligent level, which provides strong support for reducing labor costs and intelligent product grading. This method effectively improves the accuracy of defect detection, and the detected defect data is more reliable, providing the greatest guarantee for intelligent product grading.
[0134] like Figure 4 As shown, an embodiment of the present invention provides a pipe defect detection device 400 based on grayscale image and depth data fusion. The device 400 is applied to implement a pipe defect detection method based on grayscale image and depth data fusion. The device 400 includes:
[0135] The acquisition module 410 is used to acquire surface feature information of the sample to be detected; wherein the surface feature information includes depth data and grayscale image.
[0136] The depth defect candidate region module 420 is used to obtain a defect candidate region with depth information according to the depth data.
[0137] Grayscale defect candidate region module 430 is used to obtain defect candidate regions with grayscale features based on the grayscale image and the trained target detection model.
[0138] Output module 440 is used to output defect candidate areas with depth information And defect candidate areas with grayscale features Fusion matching is performed to obtain defect detection results; wherein the defect detection results include the location coordinates, size, defect category and defect depth of the defect; the defect category includes primary defect category and secondary defect category.
[0139] Optionally, the acquisition module 410 is further configured to:
[0140] The 3D camera is used to collect the depth data and grayscale image of the sample to be detected.
[0141] Depth data includes the three-dimensional characteristics of the surface of the sample to be inspected.
[0142] Grayscale images include the color, texture, and morphological characteristics of the sample to be detected.
[0143] Optionally, the depth defect candidate region module 420 is further configured to:
[0144] According to the depth data, the contour equation of the surface of the sample to be inspected is fitted, and the depth data is projected with the fitted surface contour equation as the reference plane to obtain the surface relative depth feature. The surface relative depth feature is subjected to threshold segmentation processing to obtain the defect candidate area with depth information.
[0145] The defect candidate area with depth information is an area exceeding a specified depth size.
[0146] Optionally, the profile equation of the surface of the sample to be detected is as shown in the following formula (1):
[0147] Ax 2 +Bz 2 +Cxz+Dx+Ez+F=0 (1)
[0148] Where A, B, C, D, E, and F are the parameters of the profile equation; x is the lateral position distribution coordinate in the depth data; and z is the surface depth value of the sample to be detected in the depth data.
[0149] Optionally, the depth defect candidate region module 420 is further configured to:
[0150] The depth data is processed line by line, and the valid data in each line of depth data is used as the sampling point data of the surface contour. The sampling point data is fitted by the RANSAC algorithm to obtain the contour equation parameters.
[0151] The valid data is the depth value from the surface of the sample to be detected, which is data greater than 0.
[0152] Optionally, the depth defect candidate region module 420 is further configured to:
[0153] The x-direction sampling point X(x 1,x 2 ,...x w ), and then input them into the surface profile equation to obtain the reference surface depth value Z′(z′ 1 ,z′ 2 ,...z′ w ), the relative depth feature of the projected surface is calculated by the following formula (2):
[0154]
[0155] in, is the relative depth feature of the projected surface; Z is the surface depth value of the pipe at the sampling point x; w is the number of valid sampling point data.
[0156] Optionally, the depth defect candidate region module 420 is further configured to:
[0157] Independent connected areas whose absolute values of surface relative depth features are greater than a preset detection threshold are selected as defect candidate areas with depth information.
[0158] The defect candidate region with depth information includes the position coordinates, size and defect depth of the defect candidate region.
[0159] Optionally, the grayscale defect candidate region module 430 is further configured to:
[0160] The grayscale image is input into the trained object detection model to obtain the defect candidate area with grayscale features.
[0161] The training data for the object detection model are real defect images.
[0162] The defect candidate region with grayscale features includes the position coordinates, size, and defect category of the defect candidate region.
[0163] Optionally, the output module 440 is further configured to:
[0164] Defect candidate regions with depth information The first-level defect category is deep defect.
[0165] Go through them one by one If the defect candidate area in the defect candidate area with grayscale features In, there is The defect candidate area corresponding to the defect candidate area position in The corresponding defect candidate area category identified in is Secondary defect categories of defect candidate areas in .
[0166] If in the defect candidate area with grayscale features There is no The defect candidate area corresponding to the defect candidate area position in The secondary defect category of the defect candidate area in is to be classified.
[0167] Not with The first-level defect category of the matched defect candidate area is color difference defect, the defect depth value is 0, and the second-level defect category is The defect categories identified in .
[0168] In the embodiment of the present invention, in view of the problems existing in the current surface defect detection of pipes, on the basis of the general grayscale image defect detection means, the grayscale image obtained by the 3D camera is fused with the depth data for detection, which can not only obtain a two-dimensional grayscale image for defect target detection, but also synchronously output the three-dimensional depth features of the surface. The pipe surface contour equation is obtained by fitting the depth data, and the depth data is projected onto the contour reference plane to obtain the relative depth information of its surface. Finally, the defect category is defined by multi-level labels in combination with the candidate area extracted from the depth data and the candidate area identified on the grayscale image, and the quantified defect detection results are finally output. The problem of the inability to quantify the severity of defects caused by the simple use of grayscale images is avoided. The added depth dimension features significantly improve the accuracy of defect judgment, so that defect detection is promoted to a more intelligent level, which provides strong support for reducing labor costs and intelligent product grading. This method effectively improves the accuracy of defect detection, and the detected defect data is more reliable, providing the greatest guarantee for intelligent product grading.
[0169] Figure 5 1 is a schematic diagram of the structure of an electronic device 500 provided by an embodiment of the present invention. The electronic device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 501 and one or more memories 502, wherein the memory 502 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 501 to implement the following method:
[0170] S1. Obtaining surface feature information of a sample to be detected; wherein the surface feature information includes depth data and a grayscale image.
[0171] S2. Based on the depth data, the defect candidate area with depth information is obtained
[0172] S3. Based on the grayscale image and the trained target detection model, the defect candidate area with grayscale features is obtained.
[0173] S4: Defect candidate area with depth information And defect candidate areas with grayscale features Fusion matching is performed to obtain defect detection results; wherein the defect detection results include the location coordinates, size, defect category and defect depth of the defect; the defect category includes primary defect category and secondary defect category.
[0174] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, which can be executed by a processor in a terminal to complete the above-mentioned pipe defect detection method based on grayscale image and depth data fusion. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0175] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0176] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A pipe defect detection method based on grayscale image and depth data fusion, characterized in that: The method comprises: S1. Obtaining surface feature information of a sample to be detected; wherein the surface feature information includes depth data and a grayscale image; S2. Obtaining a defect candidate region with depth information according to the depth data S3: Obtain defect candidate areas with grayscale features based on the grayscale image and the trained target detection model S4, the defect candidate area with depth information And defect candidate areas with grayscale features Perform fusion matching to obtain defect detection results; wherein the defect detection results include the location coordinates, size, defect category and defect depth of the defect; the defect category includes primary defect category and secondary defect category; The defect candidate area with depth information obtained according to the depth data in S2 includes: According to the depth data, a contour equation of the surface of the sample to be inspected is fitted, the depth data is projected with the fitted surface contour equation as a reference plane to obtain a surface relative depth feature, and a threshold segmentation process is performed on the surface relative depth feature to obtain a defect candidate area with depth information; Wherein, the defect candidate area with depth information is an area exceeding a specified depth size; The profile equation of the surface of the sample to be detected is shown in the following formula (1): Ax 2 +Bz 2 +Cxz+Dx+Ez+F=0 (1) Where A, B, C, D, E, and F are the parameters of the profile equation; x is the lateral position distribution coordinate in the depth data; z is the surface depth value of the sample to be detected in the depth data; The fitting process of the profile equation parameters includes: The depth data is processed line by line, and valid data in each line of depth data is used as sampling point data of the surface profile, and the sampling point data is fitted by a RANSAC algorithm to obtain profile equation parameters; Wherein, the valid data is data whose depth value from the surface of the sample to be detected is greater than 0; The projecting of the depth data with the contour equation of the fitted surface as a reference plane to obtain the surface relative depth feature comprises: The x-direction sampling points X(x1,x2,...x w ), and then input them into the profile equation of the surface in turn to obtain the reference surface depth value Z′(z′1,z′2,...z′ in the z direction) w ), the relative depth feature of the projected surface is calculated by the following formula (2): in, is the relative depth feature of the projected surface; Z is the surface depth value of the pipe at the sampling point; w is the number of valid sampling point data.
2. The method according to claim 1, characterized in that The obtaining of surface characteristic information of the sample to be detected in S1 includes: Use a 3D camera to collect depth data and grayscale images of the sample to be tested; The depth data includes the three-dimensional characteristics of the surface of the sample to be detected; The grayscale image includes the color, texture and morphological characteristics of the sample to be detected.
3. The method according to claim 1, characterized in that: The threshold segmentation process is performed on the surface relative depth feature to obtain the defect candidate area with depth information, including: Selecting independent connected areas whose absolute values of the surface relative depth features are greater than a preset detection threshold as defect candidate areas with depth information; The defect candidate region with depth information includes the position coordinates, size and defect depth of the defect candidate region.
4. The method according to claim 1, characterized in that: The defect candidate regions with grayscale features obtained in S3 according to the grayscale image and the trained target detection model include: Inputting the grayscale image into a trained target detection model to obtain a defect candidate area with grayscale features; The training data of the target detection model is real defect pictures; The defect candidate region with grayscale features includes the position coordinates, size and defect category of the defect candidate region.
5. The method according to claim 1, characterized in that In S4, the defect candidate area having depth information is And defect candidate areas with grayscale features After fusion matching, the defect detection results include: The defect candidate region having depth information The first-level defect category is deep defect; Go through them one by one In the defect candidate area with grayscale features, if In, there is The defect candidate area corresponding to the defect candidate area position in The corresponding defect candidate area category identified in is Secondary defect categories of defect candidate areas in; If in the defect candidate area having grayscale features There is no The defect candidate area corresponding to the defect candidate area position in The secondary defect category of the defect candidate area in is to be classified; Not with The first-level defect category of the matched defect candidate area is color difference defect, the defect depth value is 0, and the second-level defect category is The defect categories identified in .
6. A pipe defect detection device based on grayscale image and depth data fusion, characterized in that: The device comprises: An acquisition module, used to acquire surface feature information of a sample to be detected; wherein the surface feature information includes depth data and a grayscale image; A depth defect candidate region module is used to obtain a defect candidate region with depth information based on the depth data. Grayscale defect candidate region module, used to obtain defect candidate regions with grayscale features based on the grayscale image and the trained target detection model An output module is used to output the defect candidate area with depth information And defect candidate areas with grayscale features Perform fusion matching to obtain defect detection results; wherein the defect detection results include the location coordinates, size, defect category and defect depth of the defect; the defect category includes primary defect category and secondary defect category; The step of obtaining a defect candidate area having depth information according to the depth data includes: According to the depth data, a contour equation of the surface of the sample to be inspected is fitted, the depth data is projected with the fitted surface contour equation as a reference plane to obtain a surface relative depth feature, and a threshold segmentation process is performed on the surface relative depth feature to obtain a defect candidate area with depth information; Wherein, the defect candidate area with depth information is an area exceeding a specified depth size; The profile equation of the surface of the sample to be detected is shown in the following formula (1): Ax 2 +Bz 2 +Cxz+Dx+Ez+F=0 (1) Where A, B, C, D, E, and F are the parameters of the profile equation; x is the lateral position distribution coordinate in the depth data; z is the surface depth value of the sample to be detected in the depth data; The fitting process of the profile equation parameters includes: The depth data is processed line by line, and valid data in each line of depth data is used as sampling point data of the surface profile, and the sampling point data is fitted by a RANSAC algorithm to obtain profile equation parameters; Wherein, the valid data is data whose depth value from the surface of the sample to be detected is greater than 0; The projecting of the depth data with the contour equation of the fitted surface as a reference plane to obtain the surface relative depth feature comprises: The x-direction sampling points X(x1,x2,...x w ), and then input them into the profile equation of the surface in turn to obtain the reference surface depth value Z′(z′1,z′2,...z′ in the z direction) w ), the relative depth feature of the projected surface is calculated by the following formula (2): in, is the relative depth feature of the projected surface; Z is the surface depth value of the pipe at the sampling point; w is the number of valid sampling point data.
Citation Information
Cited By
Self-cleaning air conditioner air duct purifier
CN122974645A
