A defect detection method, apparatus, electronic device, and storage medium

Abstract

This application provides a defect detection method, apparatus, electronic device, and storage medium. The method includes: acquiring a depth map of a workpiece to be inspected; using a preset contour shape, determining initial positioning points in the depth map belonging to the object to be inspected through shape fitting; for each initial positioning point, determining a region to be positioned based on the size of the object to be inspected, and performing preset contour shape fitting based on the depth values ​​of each pixel in the region to be positioned to obtain a fitted contour corresponding to the initial positioning point; calculating the fitting difference result between each actual contour and the fitted contour, and determining each target pixel whose fitting difference result satisfies a preset difference condition; determining each region to be inspected in the depth map, and performing image feature detection on each region to be inspected to determine the defect detection result of the object to be inspected. Applying the method provided in this application can improve the efficiency and accuracy of workpiece surface defect detection.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a defect detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] In industrial production, when inspecting the quality of workpieces, it is usually necessary to detect surface defects. For example, when inspecting the quality of welds, it is usually necessary to detect surface defects in the welds; when inspecting the quality of rubber rings, it is usually necessary to detect surface defects in the rubber rings, and so on.

[0003] In related technologies, the detection of surface defects in workpieces mainly relies on human visual observation and simple measurement. For example, the presence of defects such as porosity and weld beads on the weld surface is detected by visual observation; the width and height of the weld are measured using measuring instruments (weld size measuring rulers) to determine whether the weld meets quality requirements, and so on.

[0004] However, among the aforementioned related technologies, the efficiency of detecting surface defects on workpieces manually is low, and the accuracy is easily affected by the subjective factors of the inspectors. Summary of the Invention

[0005] The purpose of this application is to provide a defect detection method, apparatus, electronic device, and storage medium to achieve automatic detection of surface defects in workpieces, thereby improving the efficiency and accuracy of workpiece surface defect detection. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a defect detection method, the method comprising:

[0007] Obtain the depth map of the workpiece to be inspected;

[0008] Using the preset contour shape of the object to be inspected in the workpiece to be inspected, the initial positioning points belonging to the object to be inspected in the depth map are determined by shape fitting;

[0009] For each initial positioning point, based on the size of the object to be detected, a region to be positioned about the initial positioning point is determined in the depth map, and the preset contour shape is fitted according to the depth value of each pixel in the region to be positioned to obtain the fitted contour corresponding to the initial positioning point.

[0010] Calculate the fitting difference between the actual contour and the fitted contour corresponding to each initial positioning point, and determine each target pixel point whose fitting difference satisfies the preset difference condition.

[0011] Based on each target pixel, each region to be detected in the depth map is determined, and image feature detection is performed on each region to be detected to determine the defect region in each region to be detected, which is taken as the defect detection result of the object to be detected.

[0012] Optionally, in one specific implementation, before determining each region to be detected in the depth map based on each target pixel, the method further includes:

[0013] According to the preset curve shape, curve fitting is performed on each of the initial positioning points to obtain each fitted curve;

[0014] For each fitted curve, identify the initial positioning points among the initial positioning points used to fit the fitted curve that are at a distance greater than a specified distance from the fitted curve, and designate them as outliers.

[0015] The step of determining each region to be detected in the depth map based on each target pixel includes:

[0016] Based on each target pixel and each outlier, each region to be detected in the depth map is determined.

[0017] Optionally, in one specific implementation, before determining each region to be detected in the depth map based on each target pixel and each outlier, the method further includes:

[0018] Calculate the average fitting error for each fitted profile, and determine the fitted profiles whose average fitting error is greater than a specified error as error profiles.

[0019] The step of determining each region to be detected in the depth map based on each target pixel and each outlier includes:

[0020] Based on each target pixel, each outlier, and each error contour, each region to be detected in the depth map is determined.

[0021] Optionally, in one specific implementation, the step of determining each region to be detected in the depth map based on each target pixel, and performing image feature detection on each region to determine the defect region in each region to be detected, includes:

[0022] The difference value of each target pixel, the first detection result for each outlier point, and the second detection result for each error contour are marked in the depth map to obtain the target residual map;

[0023] Blob analysis is performed on the target residual map to obtain the defect region in the depth map.

[0024] Optionally, in one specific implementation, obtaining the depth map of the workpiece to be inspected includes:

[0025] A depth map of the workpiece to be inspected, generated by a laser contour scanner, is obtained; wherein the laser contour scanner scans the workpiece in a direction perpendicular to each cross-section of the workpiece conforming to the preset contour shape.

[0026] The step of determining the initial positioning points belonging to the object to be inspected in the depth map by using the preset contour shape of the object to be inspected in the workpiece to be inspected through shape fitting includes:

[0027] For each row of pixels in the depth map, the initial positioning point of the object to be detected in that row of pixels is determined by shape fitting using the preset contour shape of the object to be detected in the workpiece to be inspected.

[0028] Alternatively, in one specific implementation,

[0029] If the object to be inspected is a weld, then the preset contour shape of the object to be inspected is a parabolic shape;

[0030] If the object to be tested is a rubber ring, then the preset outline shape of the object to be tested is a circle.

[0031] Optionally, in one specific implementation, the image features include at least one of area, major axis length, minor axis length, and volume.

[0032] Secondly, embodiments of this application provide a defect detection device, the device comprising:

[0033] The image acquisition module is used to acquire the depth map of the workpiece to be inspected;

[0034] The positioning point determination module is used to determine each initial positioning point belonging to the object to be inspected in the depth map by using the preset contour shape of the object to be inspected in the workpiece to be inspected and through shape fitting.

[0035] The contour fitting module is used to determine the region to be located about the initial positioning point in the depth map based on the size of the object to be detected for each initial positioning point, and to perform the preset contour shape fitting according to the depth value of each pixel in the region to be located to obtain the fitted contour corresponding to the initial positioning point.

[0036] The fitting difference module is used to calculate the fitting difference result between the actual contour and the fitted contour corresponding to each initial positioning point, and to determine each target pixel point whose fitting difference result satisfies the preset difference condition.

[0037] The region determination module is used to determine each region to be detected in the depth map based on each target pixel, and to perform image feature detection on each region to be detected to determine the defect region in each region to be detected, which is used as the defect detection result of the object to be detected.

[0038] Optionally, in one specific implementation, the apparatus further includes:

[0039] The curve fitting module is used to perform curve fitting on each of the initial positioning points to obtain each fitted curve.

[0040] The outlier identification module is used to identify outliers among the initial positioning points used to fit each fitted curve, where the distance from the fitted curve is greater than a specified distance.

[0041] The region determination module is specifically used for:

[0042] Based on each target pixel and each outlier, each region to be detected in the depth map is determined.

[0043] Optionally, in one specific implementation, the apparatus further includes:

[0044] The error calculation module is used to calculate the average fitting error of each fitted contour and determine the fitted contours whose average fitting error is greater than a specified error as error contours.

[0045] The region determination module is specifically used for:

[0046] Based on each target pixel, each outlier, and each error contour, each region to be detected in the depth map is determined.

[0047] Optionally, in one specific implementation, the region determination module is specifically used for:

[0048] The difference value of each target pixel, the first detection result for each outlier point, and the second detection result for each error contour are marked in the depth map to obtain the target residual map;

[0049] Blob analysis is performed on the target residual map to obtain the defect region in the depth map.

[0050] Optionally, in one specific implementation, the image acquisition module is specifically used for:

[0051] A depth map of the workpiece to be inspected, generated by a laser contour scanner, is obtained; wherein the laser contour scanner scans the workpiece in a direction perpendicular to each cross-section of the workpiece conforming to the preset contour shape.

[0052] The positioning point determination module is specifically used for:

[0053] For each row of pixels in the depth map, the initial positioning point of the object to be detected in that row of pixels is determined by shape fitting using the preset contour shape of the object to be detected in the workpiece to be inspected.

[0054] Alternatively, in one specific implementation,

[0055] If the object to be inspected is a weld, then the preset contour shape of the object to be inspected is a parabolic shape;

[0056] If the object to be tested is a rubber ring, then the preset outline shape of the object to be tested is a circle.

[0057] Optionally, in one specific implementation, the image features include at least one of area, major axis length, minor axis length, and volume.

[0058] Thirdly, embodiments of this application provide an electronic device, including:

[0059] Memory, used to store computer programs;

[0060] The processor, when executing a program stored in memory, implements any of the defect detection methods described above.

[0061] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the defect detection methods described above.

[0062] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the defect detection methods described above.

[0063] Beneficial effects of the embodiments in this application:

[0064] As can be seen from the above, by applying the solution provided in the embodiments of this application, since the outline of the object to be inspected in the workpiece to be inspected can have certain shape features, for example, when the object to be inspected is a weld, the outline of each section of the weld can be approximately parabolic. Therefore, the preset outline shape of the object to be inspected can be determined according to the shape features of the outline of the object to be inspected in the workpiece to be inspected. Therefore, when performing defect detection on a workpiece, a depth map of the workpiece can be obtained first. Then, using the preset contour shape of the object to be inspected within the workpiece, shape fitting is used to determine the initial positioning points belonging to the object in the depth map. Next, for each initial positioning point, based on the size of the object, a region to be located in the depth map is determined for that initial positioning point. Based on the depth values ​​of each pixel in the region to be located, a preset contour shape is fitted to obtain the fitted contour corresponding to that initial positioning point. Then, by calculating the fitting difference between the actual contour and the fitted contour corresponding to each initial positioning point, target pixels whose fitting difference satisfies preset difference conditions can be determined. Finally, based on each target pixel, each region to be inspected in the depth map is determined, and image feature detection is performed on each region to be inspected to identify defect regions within each region, thus obtaining the defect detection result for the object.

[0065] Based on this, the defect detection results of the object to be inspected in the workpiece can be determined according to the depth map of the workpiece, thereby realizing the automatic detection of surface defects of the workpiece and improving the efficiency and accuracy of surface defect detection of the workpiece. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0067] Figure 1 A schematic flowchart of a defect detection method provided in an embodiment of this application;

[0068] Figure 2(a) is a schematic diagram of a fitting difference result provided in an embodiment of this application;

[0069] Figure 2(b) is a schematic diagram of another fitting difference result provided by an embodiment of this application;

[0070] Figure 2(c) is a schematic diagram of a fitting difference method provided in an embodiment of this application;

[0071] Figure 3(a) is a depth diagram of the weld provided in an embodiment of this application;

[0072] Figure 3(b) is the target residual map corresponding to the depth map shown in Figure 3(a);

[0073] Figure 4 Another flowchart illustrating the defect detection method provided in this application embodiment;

[0074] Figure 5(a) is a depth diagram of the weld seam provided in an embodiment of this application;

[0075] Figure 5(b) is a depth diagram of the rubber ring provided in an embodiment of this application;

[0076] Figure 6 The positioning results of the depth map of the rubber ring shown in FIG5(b) provided for the embodiments of this application;

[0077] Figure 7 This is a schematic diagram of the structure of a defect detection device provided in an embodiment of this application;

[0078] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0079] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0080] In related technologies, the detection of surface defects in workpieces mainly relies on human visual observation and simple measurement. For example, the presence of defects such as porosity and weld beads on the weld surface is detected by visual observation; the width and height of the weld are measured using measuring instruments (weld size measuring rulers) to determine whether the weld meets quality requirements, and so on.

[0081] However, among the aforementioned related technologies, the efficiency of detecting surface defects on workpieces manually is low, and the accuracy is easily affected by the subjective factors of the inspectors.

[0082] To address the aforementioned problems, this application provides a defect detection method.

[0083] This method is applicable to various scenarios involving the detection of surface defects in workpieces, such as detecting surface defects in welds or rubber rings. The application scenarios described in this application are not specifically limited.

[0084] Furthermore, the executing entity of this method can be any electronic device capable of acquiring depth image data and processing the data. This electronic device can be a depth image acquisition device with data processing capabilities, or any device capable of processing data that has a communication connection with the depth image acquisition device, such as a mobile phone, laptop, or desktop computer. Moreover, this electronic device can be a standalone electronic device or a cluster of multiple electronic devices. This application does not specifically limit this aspect; the following term will be "electronic device."

[0085] This application provides a defect detection method that may include the following steps:

[0086] Obtain the depth map of the workpiece to be inspected;

[0087] Using the preset contour shape of the object to be inspected in the workpiece to be inspected, the initial positioning points belonging to the object to be inspected in the depth map are determined by shape fitting;

[0088] For each initial positioning point, based on the size of the object to be detected, a region to be positioned about the initial positioning point is determined in the depth map, and the preset contour shape is fitted according to the depth value of each pixel in the region to be positioned to obtain the fitted contour corresponding to the initial positioning point.

[0089] Calculate the fitting difference between the actual contour and the fitted contour corresponding to each initial positioning point, and determine each target pixel point whose fitting difference satisfies the preset difference condition.

[0090] Based on each target pixel, each region to be detected in the depth map is determined, and image feature detection is performed on each region to be detected to determine the defect region in each region to be detected, which is taken as the defect detection result of the object to be detected.

[0091] As can be seen from the above, by applying the solution provided in the embodiments of this application, since the outline of the object to be inspected in the workpiece to be inspected can have certain shape features, for example, when the object to be inspected is a weld, the outline of each section of the weld can be approximately parabolic. Therefore, the preset outline shape of the object to be inspected can be determined according to the shape features of the outline of the object to be inspected in the workpiece to be inspected. Therefore, when performing defect detection on a workpiece, a depth map of the workpiece can be obtained first. Then, using the preset contour shape of the object to be inspected within the workpiece, shape fitting is used to determine the initial positioning points belonging to the object in the depth map. Next, for each initial positioning point, based on the size of the object, a region to be located in the depth map is determined for that initial positioning point. Based on the depth values ​​of each pixel in the region to be located, a preset contour shape is fitted to obtain the fitted contour corresponding to that initial positioning point. Then, by calculating the fitting difference between the actual contour and the fitted contour corresponding to each initial positioning point, target pixels whose fitting difference satisfies preset difference conditions can be determined. Finally, based on each target pixel, each region to be inspected in the depth map is determined, and image feature detection is performed on each region to be inspected to identify defect regions within each region, thus obtaining the defect detection result for the object.

[0092] Based on this, the defect detection results of the object to be inspected in the workpiece can be determined according to the depth map of the workpiece, thereby realizing the automatic detection of surface defects of the workpiece and improving the efficiency and accuracy of surface defect detection of the workpiece.

[0093] The following is a detailed description of a defect detection method provided in an embodiment of this application, with reference to the accompanying drawings.

[0094] Figure 1 This is a schematic flowchart of a defect detection method provided in an embodiment of this application, such as... Figure 1 As shown, the method may include the following steps S101-S105.

[0095] S101: Obtain the depth map of the workpiece to be inspected.

[0096] Depth image acquisition equipment can acquire depth maps of a workpiece to be inspected, and based on the depth map of an object, the spatial relationships of various points on the object's surface can be determined. Therefore, when performing surface defect inspection on a workpiece, the depth map of the workpiece acquired by the depth image acquisition equipment can be obtained first.

[0097] The aforementioned depth image acquisition device can be various devices such as laser contour scanners and depth cameras, and this application embodiment does not impose specific limitations.

[0098] S102: Using the preset contour shape of the object to be inspected in the workpiece to be inspected, the initial positioning points belonging to the object to be inspected in the depth map are determined by shape fitting.

[0099] Since the outline of the object to be inspected in the workpiece can have certain shape characteristics—for example, when the object to be inspected is a weld, the outline of each cross-section of the weld can approximate a parabola, and when the object to be inspected is a rubber ring, the outline of each cross-section of the rubber ring can approximate a circle—the preset outline shape of the object to be inspected can be determined based on the shape characteristics of the outline of the object to be inspected in the workpiece. Therefore, when performing defect inspection on the workpiece, a depth map of the workpiece can be obtained first, and then, using the preset outline shape of the object to be inspected in the workpiece, shape fitting can be used to determine the initial positioning points of the object to be inspected in the depth map.

[0100] For example, the object to be inspected in the above-mentioned workpiece can be a weld, and the preset contour shape of the weld can be a parabola. Then, parabolic fitting can be used to traverse each row of pixels in the depth map according to the preset width range, and the position with the smallest fitting error in each row of pixels can be determined as the initial positioning point of the weld in that row of pixels, thereby obtaining each initial positioning point of the weld in the depth map.

[0101] S103: For each initial positioning point, based on the size of the object to be detected, determine the region to be positioned in the depth map related to the initial positioning point, and perform a preset contour shape fitting based on the depth value of each pixel in the region to be positioned to obtain the fitted contour corresponding to the initial positioning point.

[0102] After determining each initial positioning point, for each initial positioning point, based on the size of the object to be detected, the region to be located about the initial positioning point can be determined in the depth map, and a preset contour shape can be fitted according to the depth value of each pixel in the region to be located to obtain the fitted contour corresponding to the initial positioning point.

[0103] For example, when the object to be detected is a rubber ring, after determining each initial positioning point, for each initial positioning point, the area to be positioned about the initial positioning point can be determined in the depth map based on the diameter of the cross section of the rubber ring, and a preset contour shape is fitted according to the depth value of each pixel in the area to be positioned to obtain the fitted contour corresponding to the initial positioning point.

[0104] Optionally, since the contour shape of the defect area may not conform to the preset contour shape, the fitting error of the initial positioning points of the defect area may be large, and the positioning may not be accurate enough. Therefore, after determining each initial positioning point, the fitting error of each initial positioning point can be determined. Then, for each initial positioning point whose fitting error is greater than a specified error, interpolation processing is performed on the positioning point using the initial positioning points near that initial positioning point whose fitting error is less than the specified error.

[0105] Optionally, mean filtering or median filtering can be applied to each initial positioning point to further improve the accuracy of each initial positioning point.

[0106] S104: Calculate the fitting difference between the actual contour and the fitted contour corresponding to each initial positioning point, and determine each target pixel point whose fitting difference satisfies the preset difference condition.

[0107] For each initial positioning point, the actual contour corresponding to that initial positioning point can be determined based on the depth values ​​of each pixel in the region to be located relative to that initial positioning point. Since the actual contour at the defect area of ​​the object to be detected usually does not conform to the preset contour shape, there is often a significant difference between the actual contour and the fitted contour at the defect area of ​​the object to be detected. Therefore, for each initial positioning point, after determining the fitted contour corresponding to that initial positioning point, the fitting difference between the actual contour and the fitted contour corresponding to each initial positioning point can be calculated. For each pixel in the depth map, if the fitting difference result of that pixel satisfies a preset condition, the region where that pixel is located may be the region where the defect of the object to be detected is located. Therefore, after calculating the fitting difference result between the actual contour and the fitted contour corresponding to each initial positioning point, each target pixel whose fitting difference result satisfies the preset difference condition can be determined.

[0108] For example, the fitting difference results of the actual contour and the fitted contour of the defect area can be shown in Figure 2(a), and the fitting difference results of the actual contour and the fitted contour of the normal area can be shown in Figure 2(b). It can be seen that there is a large difference between the fitting difference results of the defect area and the normal area. Therefore, the area where the object to be detected has defects can be determined based on the fitting difference results.

[0109] For each pixel on the actual contour, the fitting difference result can be the difference between the depth value of the pixel and the depth value of the corresponding fitting point on the fitting contour, or it can be the projection of the difference onto the direction of the normal of the fitting contour at the fitting point. In this embodiment, no specific limitation is made. Those skilled in the art can determine the calculation method of the fitting difference result according to the actual application.

[0110] For example, as shown in Figure 2(c), for a pixel point A on the actual contour, the fitting difference result of the point can be the difference M between the depth value of the point and the depth value of the corresponding fitting point B on the fitting contour, or it can be the projection value N of the difference M onto the direction of the normal of the fitting contour at the fitting point.

[0111] S105: Based on each target pixel, determine each region to be detected in the depth map, and perform image feature detection on each region to determine the defect region in each region, which is used as the defect detection result of the object to be detected.

[0112] The regions where each target pixel meets the preset difference conditions may be the regions where defects of the object to be detected are located. Therefore, based on each target pixel, each region to be detected that may have defects in the depth map can be determined. By performing image feature detection on each region to be detected, the defect region in each region to be detected can be determined as the defect detection result of the object to be detected.

[0113] Optionally, the above image features may include at least one of area, major axis length, minor axis length, and volume.

[0114] For example, the image features mentioned above may include area. Then, by performing image feature detection on each region to be detected, regions to be detected with an area greater than a specified area can be identified as defect regions, thereby obtaining the defect detection result of the object to be detected.

[0115] Based on this, the defect detection results of the object to be inspected in the workpiece can be determined according to the depth map of the workpiece, thereby realizing the automatic detection of surface defects of the workpiece and improving the efficiency and accuracy of surface defect detection of the workpiece.

[0116] When performing defect detection on a workpiece, some areas of the workpiece may be defective. However, because the actual contour shape of these areas is close to the preset contour shape (for example, the contour shape of a weak weld area may still be parabolic), the fitting difference result between the actual contour and the fitted contour corresponding to the initial positioning point in these areas does not meet the preset difference condition. This results in the pixels in these areas not being identified as target pixels, and thus, these areas cannot be identified as defective areas. Therefore, to further improve the accuracy of defect detection, optionally, in one specific implementation, before determining each area to be detected in the depth map based on each target pixel, the defect detection method provided in this application embodiment may further include the following steps 11-12.

[0117] Step 11: According to the preset curve shape, perform curve fitting on each initial positioning point to obtain each fitted curve.

[0118] Before determining each detection region in the depth map based on each target pixel, curve fitting can be performed on each initial positioning point according to the preset curve shape to obtain each fitted curve.

[0119] The preset curve shape can be any shape, such as a straight line, parabola, circle, or sine curve. Those skilled in the art can set the preset curve shape according to the actual application.

[0120] For example, the object to be inspected can be a weld on a workpiece, and if there are no defective areas on the weld, the initial positioning points on the weld are approximately distributed in a straight line. Therefore, when performing curve fitting on the initial positioning points of the weld, the preset curve shape can be a straight line.

[0121] Step 12: For each fitted curve, identify the initial positioning points among the initial positioning points used to fit the fitted curve that are at a distance greater than a specified distance from the fitted curve, and designate them as outliers.

[0122] Since the initial positioning points located in the defect area are usually farther away from the above-mentioned fitting curves compared to other initial positioning points, for each fitting curve, the initial positioning points that are farther away from the fitting curve than a specified distance can be identified as outliers.

[0123] Therefore, the above-mentioned determination of each detection region in the depth map based on each target pixel may include the following step 13.

[0124] Step 13: Based on each target pixel and each outlier, determine each region to be detected in the depth map.

[0125] After identifying the outliers in the initial localization points, each region to be detected in the depth map can be determined based on each target pixel and each outlier.

[0126] Optionally, the pixels within a specified range around the location of each outlier point, as well as the connected regions formed by each target pixel, can be defined as each region to be detected.

[0127] When performing defect detection on a workpiece, some areas of the object to be inspected may be defective regions. However, the fitting difference results corresponding to some pixels in these areas may not meet the preset difference conditions, thus preventing these pixels from being identified as target pixels and potentially making these areas unidentifiable as defective regions, thereby affecting the defect detection results. Therefore, to further improve the accuracy of defect detection, in one specific implementation, to further enhance the accuracy of defect detection, before determining the various regions to be inspected in the depth map based on each target pixel and each outlier, the defect detection method provided in this application embodiment may further include the following step 21.

[0128] Step 21: Calculate the average fitting error for each fitted profile, and determine the fitted profiles with an average fitting error greater than a specified error as the error profiles.

[0129] Therefore, the above-mentioned determination of each region to be detected in the depth map based on each target pixel and each outlier may include the following step 22.

[0130] Step 22: Based on each target pixel, each outlier, and each error contour, determine each region to be detected in the depth map.

[0131] In other words, before determining the detection regions in the depth map based on each target pixel and each outlier, the average fitting error of each fitted contour can be calculated first, and the fitted contours with an average fitting error greater than a specified error can be identified as error contours. Therefore, when determining the detection regions in the depth map, each detection region can be determined based on each target pixel, each outlier, and each error contour.

[0132] Optionally, each region to be detected can be defined as the connected regions formed by the pixels within a specified range around the location of each outlier, each target pixel, and each pixel corresponding to each error contour.

[0133] Optionally, in one specific implementation, the step S105 above, which determines each region to be detected in the depth map based on each target pixel, may include the following steps 31-32.

[0134] Step 31: Mark the difference value of each target pixel, the first detection result for each outlier point, and the second detection result for each error contour in the depth map to obtain the target residual map.

[0135] Step 32: Perform Blob analysis on the target residual map to obtain the defect area in the depth map.

[0136] After obtaining each target pixel, outlier, and error contour, the difference value of each target pixel, the first detection result for each outlier, and the second detection result for each error contour can be marked on the depth map, thus obtaining the target residual map corresponding to the aforementioned depth map. Blob analysis can be used to analyze connected components of the same pixels in an image (this connected component is called a blob). Therefore, by performing blob analysis on the target residual map, the defect region in the depth map can be obtained.

[0137] For example, Figure 3(a) is a depth map of the weld, and the target residual map corresponding to this depth map can be shown in Figure 3(b).

[0138] Optionally, the first detection result of marking each outlier in the depth map can be: for each outlier, marking the difference value of each pixel within a specified range around the location of the outlier as a first value; the second detection result of marking each error contour in the depth map can be: marking the difference value of each pixel corresponding to each error contour as a second value.

[0139] Alternatively, in one specific implementation, such as Figure 4 As shown, step S101 above: obtaining the depth map of the workpiece to be inspected may include the following step S401.

[0140] S401: Obtain the depth map of the workpiece to be inspected generated by the laser contour scanner.

[0141] In this process, the laser contour scanner scans the workpiece in a direction perpendicular to each cross-section of the object having a preset contour shape.

[0142] The contours of various cross-sections of an object under inspection in a certain direction can have the same shape characteristics. For example, the contours of various cross-sections of a weld in a certain direction can approximate a parabola. Therefore, when scanning the workpiece under inspection using a laser profilometer, the laser profilometer can scan the workpiece in a direction perpendicular to the cross-sections of the object under inspection that have a preset contour shape. Consequently, in the depth map of the workpiece under inspection generated by the laser profilometer scanning the workpiece, the contours determined based on each row of pixels can have similar shapes.

[0143] For example, the depth map generated by the laser profile scanner scanning the weld seam on the workpiece to be inspected can be shown in Figure 5(a); the depth map generated by the laser profile scanner scanning the rubber ring on the workpiece to be inspected can be shown in Figure 5(b). It can be seen that in Figure 5(a), each row of pixels about the weld seam can have similar depth features; in Figure 5(b), each row of pixels about the rubber ring can also have similar depth features.

[0144] Step S102 above: Using the preset contour shape of the object to be inspected in the workpiece to be inspected, the initial positioning points of the object to be inspected in the depth map are determined by shape fitting, which may include the following step S402.

[0145] S402: For each row of pixels in the depth map, using the preset contour shape of the object to be inspected in the workpiece to be inspected, the initial positioning point of the object to be inspected in that row of pixels is determined by shape fitting.

[0146] For each row of pixels in the depth map, by using the preset contour shape of the object to be inspected in the workpiece to be inspected, the initial positioning point of the object to be inspected in that row of pixels can be determined through shape fitting.

[0147] For example, for each row of pixels in the depth map shown in Figure 5(b), by using the preset contour shape (arc) of the rubber ring and through shape fitting, the initial positioning point of the corresponding arc vertex in that row of pixels can be determined, thus obtaining... Figure 6 The positioning results are shown.

[0148] After determining the initial positioning point of the object to be detected in each row of pixels, a certain width interval can be selected on both sides of each initial positioning point in the depth map as the positioning area about that initial positioning point, based on the width of the object to be detected.

[0149] Furthermore, for each initial positioning point, based on the depth values ​​of each pixel in the region to be located corresponding to the initial positioning point and the preset contour shape of the object to be detected, a preset contour shape is fitted to obtain the fitted contour corresponding to the initial positioning point.

[0150] Corresponding to the defect detection method provided in the above embodiments of this application, this application also provides a defect detection device.

[0151] Figure 7 This is a schematic diagram of the structure of a defect detection device provided in an embodiment of this application, as shown below. Figure 7 As shown, the defect detection device may include the following modules:

[0152] Image acquisition module 701 is used to acquire a depth map of the workpiece to be inspected;

[0153] The positioning point determination module 702 is used to determine each initial positioning point belonging to the object to be inspected in the depth map by using the preset contour shape of the object to be inspected in the workpiece to be inspected and shape fitting.

[0154] The contour fitting module 703 is used to determine the region to be located about the initial positioning point in the depth map based on the size of the workpiece to be inspected for each initial positioning point, and to perform the preset contour shape fitting according to the depth value of each pixel in the region to be located to obtain the fitted contour corresponding to the initial positioning point.

[0155] The fitting difference module 704 is used to calculate the fitting difference result between the actual contour and the fitted contour corresponding to each initial positioning point, and to determine each target pixel point whose fitting difference result satisfies the preset difference condition.

[0156] The region determination module 705 is used to determine each region to be detected in the depth map based on each target pixel, and to perform image feature detection on each region to be detected to determine the defect region in each region to be detected, which is used as the defect detection result of the object to be detected.

[0157] Based on this, the defect detection results of the object to be inspected in the workpiece can be determined according to the depth map of the workpiece, thereby realizing the automatic detection of surface defects of the workpiece and improving the efficiency and accuracy of surface defect detection of the workpiece.

[0158] Optionally, in one specific implementation, the apparatus further includes:

[0159] The curve fitting module is used to perform curve fitting on each of the initial positioning points according to a preset curve shape to obtain each fitted curve;

[0160] The outlier identification module is used to identify outliers among the initial positioning points used to fit each fitted curve, where the distance from the fitted curve is greater than a specified distance.

[0161] The region determination module is specifically used for:

[0162] Based on each target pixel and each outlier, each region to be detected in the depth map is determined.

[0163] Optionally, in one specific implementation, the apparatus further includes:

[0164] The error calculation module is used to calculate the average fitting error of each fitted contour and determine the fitted contours whose average fitting error is greater than a specified error as error contours.

[0165] The region determination module is specifically used for:

[0166] Based on each target pixel, each outlier, and each error contour, each region to be detected in the depth map is determined.

[0167] Optionally, in one specific implementation, the region determination module is specifically used for:

[0168] The difference value of each target pixel, the first detection result for each outlier point, and the second detection result for each error contour are marked in the depth map to obtain the target residual map;

[0169] Blob analysis is performed on the target residual map to obtain the defect region in the depth map.

[0170] Optionally, in one specific implementation, the image acquisition module is specifically used for:

[0171] A depth map of the workpiece to be inspected, generated by a laser contour scanner, is obtained; wherein the laser contour scanner scans the workpiece in a direction perpendicular to each cross-section of the workpiece having the contour shape features.

[0172] The positioning point determination module is specifically used for:

[0173] For each row of pixels in the depth map, the initial positioning point of the object to be detected in that row of pixels is determined by shape fitting using the preset contour shape of the object to be detected in the workpiece to be inspected.

[0174] Alternatively, in one specific implementation,

[0175] If the object to be inspected is a weld, then the preset contour shape of the object to be inspected is a parabolic shape;

[0176] If the object to be tested is a rubber ring, then the preset outline shape of the object to be tested is a circle.

[0177] Optionally, in one specific implementation, the image features include at least one of area, major axis length, minor axis length, and volume.

[0178] This application also provides an electronic device, such as... Figure 8 As shown, it includes:

[0179] Memory 801 is used to store computer programs;

[0180] When the processor 802 executes the program stored in the memory 801, it implements any of the defect detection methods described above.

[0181] Furthermore, the aforementioned electronic device may also include a communication bus and / or a communication interface, with the processor 802, the communication interface, and the memory 801 communicating with each other via the communication bus.

[0182] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0183] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0184] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0185] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0186] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described defect detection methods.

[0187] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the defect detection methods described above.

[0188] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), etc.

[0189] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0190] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments, electronic device embodiments, computer-readable storage medium embodiments, and computer program product embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0191] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A defect detection method, characterized in that, The method includes: Obtain the depth map of the workpiece to be inspected; Using the preset contour shape of the object to be inspected in the workpiece to be inspected, the initial positioning points belonging to the object to be inspected in the depth map are determined by shape fitting; For each initial positioning point, based on the size of the object to be detected, a region to be positioned about the initial positioning point is determined in the depth map, and the preset contour shape is fitted according to the depth value of each pixel in the region to be positioned to obtain the fitted contour corresponding to the initial positioning point. Calculate the fitting difference between the actual contour and the fitted contour corresponding to each initial positioning point, and determine each target pixel point whose fitting difference satisfies the preset difference condition. Based on each target pixel, each region to be detected in the depth map is determined, and image feature detection is performed on each region to be detected to determine the defect region in each region to be detected, which is taken as the defect detection result of the object to be detected. Before determining each region to be detected in the depth map based on each target pixel, the method further includes: According to the preset curve shape, curve fitting is performed on each of the initial positioning points to obtain each fitted curve; For each fitted curve, identify the initial positioning points among the initial positioning points used to fit the fitted curve that are at a distance greater than a specified distance from the fitted curve, and designate them as outliers. The step of determining each region to be detected in the depth map based on each target pixel includes: Based on each target pixel and each outlier, each region to be detected in the depth map is determined.

2. The method according to claim 1, characterized in that, Before determining each region to be detected in the depth map based on each target pixel and each outlier, the method further includes: Calculate the average fitting error for each fitted profile, and determine the fitted profiles whose average fitting error is greater than a specified error as error profiles. The step of determining each region to be detected in the depth map based on each target pixel and each outlier includes: Based on each target pixel, each outlier, and each error contour, each region to be detected in the depth map is determined.

3. The method according to claim 2, characterized in that, The process of determining each region to be detected in the depth map based on each target pixel, and performing image feature detection on each region to determine the defect region in each region includes: The difference value of each target pixel, the first detection result for each outlier point, and the second detection result for each error contour are marked in the depth map to obtain the target residual map; Blob analysis is performed on the target residual map to obtain the defect region in the depth map.

4. The method according to any one of claims 1-3, characterized in that, The process of obtaining the depth map of the workpiece to be inspected includes: A depth map of the workpiece to be inspected, generated by a laser contour scanner, is obtained; wherein the laser contour scanner scans the workpiece in a direction perpendicular to each cross-section of the workpiece conforming to the preset contour shape. The step of determining the initial positioning points belonging to the object to be inspected in the depth map by using the preset contour shape of the object to be inspected in the workpiece to be inspected through shape fitting includes: For each row of pixels in the depth map, the initial positioning point of the object to be detected in that row of pixels is determined by shape fitting using the preset contour shape of the object to be detected in the workpiece to be inspected.

5. The method according to any one of claims 1-3, characterized in that, If the object to be inspected is a weld, then the preset contour shape of the object to be inspected is a parabolic shape; If the object to be tested is a rubber ring, then the preset outline shape of the object to be tested is a circle.

6. The method according to any one of claims 1-3, characterized in that, The image features include at least one of the following: area, major axis length, minor axis length, and volume.

7. A defect detection device, characterized in that, The device includes: The image acquisition module is used to acquire a depth map of the workpiece to be inspected; The positioning point determination module is used to determine each initial positioning point belonging to the object to be inspected in the depth map by using the preset contour shape of the object to be inspected in the workpiece to be inspected and shape fitting. The contour fitting module is used to determine the region to be located about the initial positioning point in the depth map based on the size of the object to be detected for each initial positioning point, and to perform the preset contour shape fitting according to the depth value of each pixel in the region to be located to obtain the fitted contour corresponding to the initial positioning point. The fitting difference module is used to calculate the fitting difference result between the actual contour and the fitted contour corresponding to each initial positioning point, and to determine each target pixel point whose fitting difference result satisfies the preset difference condition. The region determination module is used to determine each region to be detected in the depth map based on each target pixel, and to perform image feature detection on each region to be detected to determine the defect region in each region to be detected, which is used as the defect detection result of the object to be detected. The device further includes: The curve fitting module is used to perform curve fitting on each of the initial positioning points to obtain each fitted curve. The outlier identification module is used to identify outliers among the initial positioning points used to fit each fitted curve, where the distance from the fitted curve is greater than a specified distance. The region determination module is specifically used for: Based on each target pixel and each outlier, each region to be detected in the depth map is determined.

8. The apparatus according to claim 7, characterized in that, The device further includes: The error calculation module is used to calculate the average fitting error of each fitted contour and determine the fitted contours whose average fitting error is greater than a specified error as error contours. The region determination module is specifically used for: Based on each target pixel, each outlier, and each error contour, each region to be detected in the depth map is determined. And / or, The region determination module is specifically used for: The difference value of each target pixel, the first detection result for each outlier point, and the second detection result for each error contour are marked in the depth map to obtain the target residual map; Blob analysis is performed on the target residual map to obtain the defect region in the depth map; And / or, The image acquisition module is specifically used for: A depth map of the workpiece to be inspected, generated by a laser contour scanner, is obtained; wherein the laser contour scanner scans the workpiece in a direction perpendicular to each cross-section of the workpiece conforming to the preset contour shape. The positioning point determination module is specifically used for: For each row of pixels in the depth map, using the preset contour shape of the object to be detected in the workpiece to be detected, the initial positioning point of the object to be detected in that row of pixels is determined by shape fitting. And / or, If the object to be inspected is a weld, then the preset contour shape of the object to be inspected is a parabolic shape; If the object to be tested is a rubber ring, then the preset outline shape of the object to be tested is a circle; And / or, The image features include at least one of the following: area, major axis length, minor axis length, and volume.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images