A method and system for defect detection of hardware parts based on image recognition technology

Abstract

This invention provides a method and system for detecting defects in hardware parts based on image recognition technology. The method includes: S1: acquiring a video of the part to be inspected under a preset changing light source based on a preset shooting angle; S2: analyzing the visual change features of each pixel on the surface of the part to be inspected based on the video; S3: performing cluster analysis on the visual change features of all pixels to obtain multiple region contours corresponding to the preset shooting angle; S4: obtaining region division results based on all region contours; S5: determining the defect detection result based on the region shape and image features of the divided regions in the region division results. This method is used to divide the surface of the part to be inspected into regions based on the visual change features of the part under a preset changing light source, and then obtain the defect detection result of the part to be inspected based on the region shape and image features of the divided regions. This method can identify defects on the surface of the part without preparing a large number of defective part models.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a method and system for detecting defects in hardware parts based on image recognition technology. Background Technology

[0002] Currently, in industrial production processes, the inadequacy and limitations of existing technology and working conditions can easily affect the quality of finished products. Among these factors, surface defects are the most direct manifestation of compromised product quality. Therefore, to ensure a high pass rate and reliable quality, surface defect detection is essential. A "defect" can generally be understood as a missing element or defective area compared to a normal sample. Surface defect detection refers to inspecting the surface of a sample for defects such as scratches, foreign object obstruction, color contamination, and holes, thereby obtaining a series of relevant information regarding the type, outline, location, and size of these surface defects.

[0003] However, existing defect detection technologies mostly use visual algorithms or deep learning algorithms to identify and detect defects based on the overall visual features of the parts. This method requires preparing defect samples for each part in advance to build defect identification models or algorithms for different defect types for different parts. Furthermore, since the defect samples cannot cover all actual defect situations, this method also has identification errors in the actual defect identification process.

[0004] Therefore, this invention proposes a method and system for detecting defects in hardware parts based on image recognition technology. Summary of the Invention

[0005] This invention provides a method and system for detecting defects in hardware parts based on image recognition technology. It divides the surface of the part to be inspected into regions based on the visual change characteristics of the part under a preset changing light source, and then obtains the defect detection result of the part based on the shape and image features of the divided regions. Compared with existing defect recognition methods, it does not require the preparation of a large number of defect samples and the construction of defect recognition models corresponding to different defect types of different parts, and can identify defects on the surface of the part. This not only simplifies the defect recognition process, but also improves the efficiency and accuracy of defect recognition.

[0006] This invention provides a method for detecting defects in hardware parts based on image recognition technology, comprising:

[0007] S1: Obtain a video of the part to be inspected under a preset changing light source based on a preset shooting angle;

[0008] S2: Based on the video analysis of the part, the visual change features of each pixel on the surface of the part to be inspected are obtained;

[0009] S3: Perform cluster analysis on the visual change features of all pixels to obtain the contours of multiple regions corresponding to the preset shooting angle;

[0010] S4: Obtain the region division results based on the contours of all regions;

[0011] S5: Determine the defect detection result based on the region shape and image features of the region segmentation results.

[0012] Preferably, in the aforementioned method for detecting defects in hardware parts based on image recognition technology, S2: based on video analysis of the part, the visual change features of each pixel on the surface of the part to be detected are analyzed, including:

[0013] Based on the completeness of the part region in each video frame of the part video, the best video frame is selected from the part video, and the two-dimensional coordinates of each pixel in the part region of the best video frame are determined.

[0014] The visual values ​​corresponding to all two-dimensional coordinates in each video frame are determined. Based on the order of all video frames, all visual values ​​corresponding to the two-dimensional coordinates are sorted to obtain the visual value sequence of the corresponding pixel. The visual value sequence is used as the visual change feature of the corresponding pixel.

[0015] Preferably, in the aforementioned method for detecting defects in hardware parts based on image recognition technology, S3: clustering analysis is performed on the visual change features of all pixels to obtain multiple region contours corresponding to a preset shooting angle, including:

[0016] S301: Perform cluster analysis on the visual change features of all pixels on the surface of the part to be inspected to obtain multiple pixel clusters;

[0017] S302: Based on multiple pixel clusters, determine the contours of multiple regions corresponding to the preset shooting angle on the surface of the part to be inspected.

[0018] Preferably, in the method for detecting defects in hardware parts based on image recognition technology, S301: clustering analysis is performed on the visual change features of all pixels on the surface of the part to be detected to obtain multiple pixel clusters, including:

[0019] The visual value change curve of the corresponding pixel is fitted based on the visual value sequence in the visual change features of each pixel.

[0020] Cluster analysis is performed on the visual value change curves of all pixels to obtain curve clustering results, and multiple pixel clusters are obtained based on the curve clustering results.

[0021] Preferably, in the method for detecting defects in hardware parts based on image recognition technology, S4: obtaining region segmentation results based on the contours of all regions, including:

[0022] If there are intersecting first region contours among all the region contours corresponding to the preset shooting angles, then a supplementary video of the part to be detected under a preset changing light source is obtained based on the first region contour. This process continues until the multiple second region contours obtained based on the supplementary video do not intersect with the remaining third region contours among all region contours except for the first region contour. Then, the region division result of the surface of the part to be detected is determined based on the latest obtained second and third region contours. Otherwise, the region division result of the surface of the part to be detected is determined based on all the currently obtained region contours.

[0023] Preferably, the method for detecting defects in hardware parts based on image recognition technology determines whether there are intersecting first region contours among the region contours corresponding to all preset shooting angles, including:

[0024] The part region in the best video frame of the part video obtained from each preset shooting angle is matched with the standard 3D model of the part to be detected to obtain the matching result. Based on the matching result, the spatial coordinates of each region contour in the standard 3D model are determined.

[0025] Based on the spatial coordinates of each region contour in the standard 3D model, determine whether there is a first region contour that intersects with each other among the region contours corresponding to all preset shooting angles.

[0026] Preferably, the method for detecting defects in hardware parts based on image recognition technology includes acquiring supplementary video of the part to be detected under a preset changing light source based on the contour of a first region, comprising:

[0027] Supplementary shooting angles are determined based on the outline of the first region;

[0028] Supplementary video of the part to be inspected under a preset changing light source is obtained by supplementing the shooting angle.

[0029] Preferably, in the method for detecting defects in hardware parts based on image recognition technology, S5: determining the defect detection result based on the region shape and image features of the region division result, including:

[0030] Based on the shape of the divided regions in the region division results, it is determined whether the corresponding divided region is a defect region, and the defect judgment result is obtained. When the defect judgment result is that the corresponding divided region is a defect region, the defect type is determined based on the image features of the corresponding defect region.

[0031] The defect detection results of the part to be inspected are obtained based on the defect judgment results of all divided regions, the defect location and defect type of all defect regions.

[0032] Preferably, the method for detecting defects in hardware parts based on image recognition technology determines the defect type based on the image features of the corresponding defect area, including:

[0033] Determine the type of the corresponding defect area to determine the required data;

[0034] Based on the type, the required data is queried from the defect type determination rule table to determine the defect type of the corresponding defect area.

[0035] This invention provides a hardware parts defect detection system based on image recognition technology, comprising:

[0036] The analysis end is used to acquire video of the part to be inspected under a preset changing light source based on a preset shooting angle, and to analyze the visual change features of each pixel on the surface of the part to be inspected based on the part video.

[0037] The segmentation end is used to perform cluster analysis on the visual change features of all pixels to obtain multiple region contours corresponding to the preset shooting angle, and obtain the region segmentation result based on all region contours.

[0038] The determining end is used to determine the defect detection result based on the region shape and image features of the region in the region segmentation result.

[0039] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings.

[0040] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0041] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0042] Figure 1 This is a flowchart of a method for detecting defects in hardware parts based on image recognition technology in an embodiment of the present invention;

[0043] Figure 2 This is a flowchart of another method for detecting defects in hardware parts based on image recognition technology in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of another hardware parts defect detection system based on image recognition technology in an embodiment of the present invention. Detailed Implementation

[0045] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0046] Example 1:

[0047] This invention provides a method for defect detection of hardware parts based on image recognition technology, with reference to... Figure 1 ,include:

[0048] S1: Obtain a video of the part to be inspected under a preset changing light source based on a preset shooting angle;

[0049] S2: Based on the video analysis of the part, the visual change features of each pixel on the surface of the part to be inspected are obtained;

[0050] S3: Perform cluster analysis on the visual change features of all pixels to obtain the contours of multiple regions corresponding to the preset shooting angle;

[0051] S4: Obtain the region division results based on the contours of all regions;

[0052] S5: Determine the defect detection result based on the region shape and image features of the region segmentation results.

[0053] In this embodiment, the preset shooting angle is the preset angle at which the video of the part to be inspected is captured.

[0054] In this embodiment, the part to be inspected is the part that needs to be defect-detected.

[0055] In this embodiment, the preset variable light source is a light source whose illuminance changes or whose color changes.

[0056] In this embodiment, the part video is a video obtained by shooting the part to be tested under a preset changing light source from a preset shooting angle.

[0057] In this embodiment, the visual change feature is the visual change feature of each pixel in the image area where the surface of the part to be detected is located in the part video determined after analyzing the part video. The visual change feature is the visual value change sequence. When the preset changing light source is a light source with changing illuminance, the visual value is the brightness value or gray value of the pixel. When the preset changing light source is a light source with changing color, the visual value is the chromaticity value of the pixel.

[0058] In this embodiment, the region contour is the contour of the region composed of pixel clusters obtained after clustering analysis of the visual change features of all pixels.

[0059] In this embodiment, the region division result is the result obtained after dividing the surface of the part to be inspected into regions based on the region contour.

[0060] In this embodiment, the shape of the divided region is included in the region shape division result.

[0061] In this embodiment, the image features of the divided region can be the relative depth value determined based on the gray value of each pixel in the divided region, or the flatness of the corresponding divided region on the surface of the part to be inspected can be determined based on the brightness value of each pixel.

[0062] In this embodiment, the defect detection result is the result obtained after performing defect detection on the surface of the part to be inspected. It can be that the surface of the part to be inspected does not have surface defects, or that the part to be inspected shows that there are defects (the result includes the location and type of the defects).

[0063] The beneficial effects of the above technology are as follows: Based on the visual change characteristics of the part to be inspected under a preset changing light source, the surface of the part to be inspected is divided into regions, and then the defect detection results of the part to be inspected are obtained based on the region shape and region image features after the region division. Compared with the existing defect recognition methods, it is not necessary to prepare a large number of defect samples and build defect recognition models corresponding to different defect types of different parts. The defects on the surface of the part can be identified, which not only simplifies the defect recognition process, but also improves the efficiency and accuracy of defect recognition.

[0064] Example 2:

[0065] Based on Example 1, the method for detecting defects in hardware parts based on image recognition technology, S2: analyzing the visual change features of each pixel on the surface of the part to be detected based on the part video, including:

[0066] Based on the completeness of the part region in each video frame of the part video (representing the degree of completeness of the corresponding part region), the best video frame (the video frame containing the part region with the highest completeness in the part video) is selected from the part video, and the two-dimensional coordinates of each pixel in the part region of the best video frame are determined.

[0067] The visual values ​​corresponding to all two-dimensional coordinates in each video frame are determined. Based on the order of all video frames, all visual values ​​corresponding to the two-dimensional coordinates are sorted to obtain the visual value sequence of the corresponding pixel. The visual value sequence is used as the visual change feature of the corresponding pixel.

[0068] In this embodiment, the part region is the region corresponding to the surface of the part to be detected in the video frame.

[0069] In this embodiment, the best video frame is selected from the part video based on the completeness of the part region in each video frame, including:

[0070] In each video frame of the part video, the part outline (the outline of the part region in the video frame) is determined. The distance from a pixel in the part outline to the center point of the part outline (i.e., the average of the coordinate values ​​of all pixels in the part outline) is used as the magnitude of the vector, and the tangent angle of a pixel in the part outline (i.e., the angle between the tangent of the part outline at the corresponding pixel and the reference direction, which can be the positive direction of the horizontal axis of the preset coordinate system) is used as the angle of the vector to obtain the local outline feature vector of the corresponding pixel (i.e., the feature that represents the local outline shape of the part outline at the corresponding pixel). Based on the local outline feature vector of each pixel in the part outline, a feature vector sequence is obtained (i.e., the vector sequence obtained by sorting the local outline feature vectors of each pixel in the part outline).

[0071] Determine whether there are multiple consecutive local contour feature vectors with equal directions in the feature vector sequence, where the number of consecutive vectors exceeds a threshold (i.e., the minimum number of consecutive vectors with equal directions when determining that multiple consecutive local contour feature vectors with equal directions correspond to local contour feature vectors of pixels on a straight line segment). If so, obtain the first sub-feature vector sequence (i.e., the vector sequence obtained by sorting multiple consecutive local contour feature vectors with equal directions) based on the multiple consecutive local contour feature vectors with equal directions. Otherwise, determine the angle increment (i.e., the angle difference between the next local contour feature vector and the previous local contour feature vector in the adjacent local contour feature vector sequence) and the magnitude increment (i.e., the magnitude difference between the next local contour feature vector and the previous local contour feature vector in the adjacent local contour feature vector sequence).

[0072] Based on the angle increment and magnitude increment, the gradual change pattern of the angle and / or magnitude of the local contour feature vectors in the feature vector sequence is analyzed (that is, whether the change process of the angle and / or magnitude can be represented by a univariate function; if so, it is determined that the change process of the angle and / or magnitude has a gradual change pattern; otherwise, it is determined that the change process of the angle and / or magnitude does not have a gradual change pattern). The second sub-feature vector sequence in the feature vector sequence with a gradual change pattern of angle and / or magnitude is determined (that is, the sequence of continuous vectors in the feature vector sequence with the same gradual change pattern of angle and / or magnitude).

[0073] By summing up all the first sub-feature vector sequences and all the second sub-feature vector sequences in the feature vector sequence, we obtain the set of sub-feature vector sequences of the corresponding part contour (that is, the set obtained by summing up all the first sub-feature vector sequences and all the second sub-feature vector sequences in the feature vector sequence).

[0074] The product of the reciprocal of the total number of sub-feature vector sequences in the set of sub-feature vector sequences and the sum of the total values ​​in all sub-feature vector sequences is used as the completeness of the part region corresponding to the part contour (because a larger total number of sub-feature vector sequences represents a larger number of straight lines, arcs, or elliptical arcs, which means more defects in the corresponding part region; a larger total value in the sub-feature vector sequences represents a longer straight line, arc, or elliptical arc, which means fewer defects in the corresponding part region; therefore, the product of the reciprocal of the total number of sub-feature vector sequences in the set of sub-feature vector sequences and the sum of the total values ​​in all sub-feature vector sequences represents the completeness of the part region corresponding to the part contour).

[0075] The video frame containing the part area with the highest integrity in the part video is taken as the best video frame.

[0076] The above technology utilizes the tangent angle of a pixel in the part contour as the angle of the vector to obtain the local contour feature vector of the corresponding pixel as a vector representing the local shape feature of the part contour at the corresponding pixel. It uses multiple consecutive local contour feature vectors with equal directions to obtain the first sub-feature vector sequence to represent the straight line segments in the part contour. It uses the second sub-feature vector sequence with a gradual change in angle and / or magnitude to represent the arc or elliptical arc with a gradual change in the tangent angle and / or the distance from the pixel to the center point of the part contour. It uses the total number of sub-feature vector sequences and the total number of values ​​in the sub-feature vector sequences in the feature vector sequence set as the basis for calculating the completeness of the part region, accurately determining the completeness that can represent the completeness of the corresponding part region, and then determining the best video frame containing the most complete part region in the part video based on the completeness.

[0077] In this embodiment, the two-dimensional coordinates are the two-dimensional coordinate values ​​of each pixel in the part area in the best video frame.

[0078] In this embodiment, the visual value sequence is the sequence of changes in the visual value of the corresponding pixel in the video after sorting all visual values ​​corresponding to the corresponding two-dimensional coordinates based on the order of all video frames.

[0079] The beneficial effects of the above technology are as follows: the tracking of the visual value change process of the pixels in the best video frame selected based on the integrity of the video frames in the part video maximizes the acquisition of the visual change characteristics of the pixels on the surface of the part to be detected under the preset changing light source.

[0080] Example 3:

[0081] refer to Figure 2Based on Example 1, the method for detecting defects in hardware parts based on image recognition technology, S3: performs cluster analysis on the visual change features of all pixels to obtain multiple region contours corresponding to a preset shooting angle, including:

[0082] S301: Perform cluster analysis on the visual change features of all pixels on the surface of the part to be inspected to obtain multiple pixel clusters;

[0083] S302: Based on multiple pixel clusters, determine the contours of multiple regions corresponding to the preset shooting angle on the surface of the part to be inspected.

[0084] In this embodiment, a pixel cluster is a cluster obtained by performing cluster analysis on the visual change features of all pixels on the surface of the part to be inspected. Each cluster contains multiple clusters of visual change features that have been clustered and analyzed. The cluster is obtained by summing up the pixels corresponding to all visual change features contained in the cluster.

[0085] In this embodiment, multiple region contours corresponding to a preset shooting angle are determined based on multiple pixel clusters on the surface of the part to be inspected, namely:

[0086] The outline of the region composed of all pixels in the pixel cluster is taken as the corresponding region outline.

[0087] The beneficial effects of the above technology are as follows: by performing cluster analysis on the visual change features of pixels, the outline of the region with uniform visual change features can be determined in the part video corresponding to the preset shooting angle.

[0088] Example 4:

[0089] Based on Example 3, the method for detecting defects in hardware parts based on image recognition technology, S301: performs cluster analysis on the visual change features of all pixels on the surface of the part to be detected to obtain multiple pixel clusters, including:

[0090] The visual value change curve of the corresponding pixel is fitted based on the visual value sequence in the visual change features of each pixel (i.e., the curve is fitted sequentially from the visual values ​​in the visual value sequence).

[0091] Cluster analysis is performed on the visual value change curves of all pixels to obtain curve clustering results (i.e., multiple curve clusters obtained after cluster analysis of the visual value change curves of all pixels). Based on the curve clustering results, multiple pixel clusters are obtained (a pixel cluster is a pixel cluster obtained by summing up the pixels corresponding to all visual value change curves in the curve clusters included in the curve clustering results).

[0092] In this embodiment, cluster analysis is performed on the visual value change curves of all pixels to obtain curve clustering results. Based on the curve clustering results, multiple pixel clusters are obtained, including:

[0093] By unifying the visual value variation curves of all pixels under the same coordinate system, a unified result can be obtained.

[0094] Based on the number of first clusters in the preset list of number of first clusters (including multiple pre-prepared first cluster numbers), all visual value change curves in the unified result are divided into the number of first clusters (the total number of first curve clusters obtained after the division process) of first curve clusters, and the initial division result corresponding to each number of first clusters is obtained (i.e., the result containing the number of first clusters of first curve clusters).

[0095] Identify each first coordinate point (i.e., the coordinate point in the visual value change curve) in each visual value change curve, and identify all first visual value change curves in the initial division result that belong to the same first curve cluster as the visual value change curve containing the first coordinate point;

[0096] The average value of the difference in the vertical coordinate between the first coordinate point and the second coordinate point in all the curves of change of the first visual value that is equal to the horizontal coordinate value of the corresponding first coordinate point is determined as the first deviation value of the corresponding first coordinate point.

[0097] The average of the first deviation values ​​of all first coordinate points in the visual value change curve is taken as the first comprehensive deviation value of the corresponding visual value change curve.

[0098] Identify all second visual value change curves in the initial segmentation results that do not belong to the same first curve cluster as the visual value change curve of the first coordinate point;

[0099] The average value of the difference in the vertical coordinate between the first coordinate point and the third coordinate point in all the second visual value change curves that have the same horizontal coordinate value as the corresponding first coordinate point is determined as the second deviation value of the corresponding first coordinate point.

[0100] The average of the second deviation values ​​of all first coordinate points in the visual value change curve is taken as the second comprehensive deviation value of the corresponding visual value change curve.

[0101] The ratio of the difference between the second comprehensive deviation value and the first comprehensive deviation value to the larger of the first comprehensive deviation value and the second comprehensive deviation value is used as the contour coefficient of the corresponding visual value change curve, and the average of all contour coefficients is used as the comprehensive contour coefficient of the corresponding initial segmentation result.

[0102] The number of the first clusters corresponding to the initial division result with the maximum comprehensive contour coefficient is taken as the total number of clusters in K-Means clustering. Based on the determined total number of clusters, the visual value change curves of all pixels are clustered to obtain multiple second curve clusters (curve clusters obtained after clustering the visual value change curves of all pixels based on the determined total number of clusters).

[0103] Calculate the average of the ordinates of the same abscissa value in all visual value change curves in the second curve cluster obtained after each clustering (i.e., the average of all ordinates corresponding to the same abscissa value in all visual value change curves in the second curve cluster), and use the curve fitted based on the average of the ordinates of all abscissa values ​​as the cluster center of the corresponding second curve cluster.

[0104] The average of the absolute values ​​of the differences between the first ordinate value corresponding to each abscissa value in all visual value change curves in the second curve cluster and the second ordinate value corresponding to the curve corresponding to the cluster center is calculated as the third deviation value of the corresponding abscissa value. The difference between 1 and the average of the third deviation values ​​of all abscissa values ​​is taken as the density of the corresponding second curve cluster.

[0105] The average density of all second curve clusters obtained after each clustering is used as the comprehensive judgment value. It is then determined whether the comprehensive judgment value is not less than a preset threshold (i.e., the minimum comprehensive judgment value that must be met when clustering is stopped). If so, the pixels corresponding to all visual value change curves in all the latest obtained second curve clusters are summarized to obtain a pixel cluster. Otherwise, clustering continues until the latest obtained comprehensive judgment value is not less than the preset threshold. Then, the pixels corresponding to all visual value change curves in all the latest obtained second curve clusters are summarized to obtain a pixel cluster.

[0106] The beneficial effects of the above technology are as follows: the visual value change curve corresponding to the visual value change sequence in the visual change features of pixels is used as the sample to be clustered and analyzed. The average value of the difference between the vertical coordinates corresponding to the same horizontal coordinate value in the visual value change curves of different samples is used as the deviation between different samples. The total number of clusters for cluster analysis is determined by combining the method of determining the total number of cluster centers based on the silhouette coefficient. The density used to judge whether the results of cluster analysis meet the requirements is determined by combining the average value of the difference between the vertical coordinates corresponding to the same horizontal coordinate value in the visual value change curves of different samples as the deviation between different samples. Thus, cluster analysis of the visual change features of pixels is realized, and clustering of pixels is achieved based on visual change features.

[0107] Example 5:

[0108] Based on Example 1, the method for detecting defects in hardware parts based on image recognition technology, S4: obtaining region segmentation results based on the contours of all regions, including:

[0109] If there are intersecting first region contours among all the region contours corresponding to the preset shooting angles, then a supplementary video of the part to be detected under a preset changing light source is obtained based on the first region contour. This process continues until the multiple second region contours obtained based on the supplementary video do not intersect with the remaining third region contours among all region contours except for the first region contour. Then, the region division result of the surface of the part to be detected is determined based on the latest obtained second and third region contours. Otherwise, the region division result of the surface of the part to be detected is determined based on all the currently obtained region contours.

[0110] In this embodiment, the first region contour is the region contour that intersects with each other among all the region contours corresponding to the preset shooting angles.

[0111] In this embodiment, the supplementary video is the video obtained after supplementing the shooting of the part to be tested under a preset changing light source when it is determined whether there is a first area contour that intersects with each other among the area contours corresponding to all preset shooting angles.

[0112] In this embodiment, the second region contour is the region contour determined in the part region of the supplementary video.

[0113] In this embodiment, the third region contour is the region contour remaining after the first region contour among all region contours.

[0114] In this embodiment, the region division result of the surface of the part to be inspected is determined based on the newly obtained second region contour and third region contour, that is:

[0115] The regions enclosed by the newly obtained second and third region contours are used as the partitioned regions in the region partitioning results.

[0116] In this embodiment, the region division result of the surface of the part to be inspected is determined based on all currently obtained region contours, that is:

[0117] The region enclosed by all the currently obtained region outlines is used as the partitioned region in the region partitioning result.

[0118] The beneficial effects of the above technology are as follows: by determining whether there is an intersection of the contours of all regions, it is determined whether it is necessary to take supplementary pictures of the part to be inspected and to redetermine the region contours based on the supplementary video obtained from the supplementary pictures, thereby ensuring the accuracy of the obtained region contours.

[0119] Example 6:

[0120] Based on Example 5, the method for detecting defects in hardware parts based on image recognition technology determines whether there are intersecting first region contours among the region contours corresponding to all preset shooting angles, including:

[0121] The part region in the best video frame of the part video obtained from each preset shooting angle is matched with the standard 3D model of the part to be detected to obtain the matching result. Based on the matching result, the spatial coordinates of each region contour in the standard 3D model are determined.

[0122] Based on the spatial coordinates of each region contour in the standard 3D model, determine whether there is a first region contour that intersects with each other among the region contours corresponding to all preset shooting angles.

[0123] In this embodiment, the standard three-dimensional model is the three-dimensional model of the part to be inspected in a defect-free state.

[0124] In this embodiment, the matching result is the result obtained by matching the part region in the best video frame of the part video obtained from each preset shooting angle with the standard three-dimensional model of the part to be detected.

[0125] In this embodiment, the part region in the best video frame of the part video obtained from each preset shooting angle is matched with the standard 3D model of the part to be detected to obtain the matching result, which is:

[0126] Based on the shape and contour of the part region in the best video frame, the part region is matched with the standard 3D model of the part to be detected to determine the location of the part region in the standard 3D model, and the location is used as the matching result.

[0127] In this embodiment, the spatial coordinates of each region's contour in the standard 3D model are determined based on the matching results, including:

[0128] Based on the location of the part region contained in the matching position in the standard 3D model, the spatial coordinates of the corresponding part region in the standard 3D model are determined. Based on the spatial coordinates of the corresponding part region in the standard 3D model, the spatial coordinates of the contour of each region in the corresponding part region in the standard 3D model are determined.

[0129] In this embodiment, based on the spatial coordinates of each region contour in the standard 3D model, it is determined whether there are intersecting first region contours among all the region contours corresponding to the preset shooting angles, i.e.:

[0130] Based on the spatial coordinates of the region contour in the standard 3D model, it is determined whether there are at least two region contours whose spatial coordinates overlap. If so, it is determined that there are intersecting first region contours among all the region contours corresponding to the preset shooting angles; otherwise, it is determined that there are no intersecting first region contours among all the region contours corresponding to the preset shooting angles.

[0131] The beneficial effects of the above technology are as follows: by matching the part region in the best video frame of the part video with the standard 3D model of the part to be detected, the spatial coordinates of the part region are determined, and then the spatial coordinates of the region contour are determined. Based on the spatial coordinates of the region contour, it is determined whether the region contours intersect each other.

[0132] Example 7:

[0133] Based on Example 5, the method for detecting defects in hardware parts based on image recognition technology, which acquires supplementary video of the part to be detected under a preset changing light source based on the contour of a first region, includes:

[0134] Supplementary shooting angles are determined based on the outline of the first region;

[0135] Supplementary video of the part to be inspected under a preset changing light source is obtained by supplementing the shooting angle.

[0136] In this embodiment, the supplementary shooting angle is determined based on the contour of the first region, namely:

[0137] The average value of all pixels in the first region contour with intersections is used as the reference pixel. The cross plane at the reference pixel in the standard 3D model is determined, and the angle of the straight line passing through the reference pixel and perpendicular to the cross plane is used as the supplementary shooting angle.

[0138] The beneficial effect of the above technology is that by taking the angle of a straight line passing through the center point of the first region contour and perpendicular to the tangent plane corresponding to the center point as the shooting angle, the best shooting angle for capturing the region where the first region contour is located can be obtained.

[0139] Example 8:

[0140] Based on Example 1, the method for detecting defects in hardware parts based on image recognition technology, S5: determining the defect detection result based on the region shape and image features of the region division results, including:

[0141] Based on the shape of the divided regions in the region division results, it is determined whether the corresponding divided region is a defect region, and the defect judgment result is obtained. When the defect judgment result is that the corresponding divided region is a defect region, the defect type is determined based on the image features of the corresponding defect region.

[0142] The defect detection results of the part to be inspected are obtained based on the defect judgment results of all divided regions, the defect location and defect type of all defect regions.

[0143] In this embodiment, determining whether a corresponding region is a defect region based on the shape of the region in the region division result includes:

[0144] Determine whether the shape of the divided region in the region division result is a regular shape contained in the regular shape library. If it is, determine that the corresponding divided region is not a defect region; otherwise, determine that the corresponding divided region is a defect region.

[0145] The regular shape library (i.e., the preset graphic library containing regular shapes) includes regular shapes such as triangles, circles, rectangles, parallelograms, and regular polygons; irregular shapes are generally considered to be those shapes that cannot be defined or named.

[0146] In this embodiment, the defect judgment result is the result obtained after judging whether the corresponding divided region is a defect region based on the shape of the divided region in the region division result.

[0147] In this embodiment, the defect types include: scratches, foreign object obstruction, color contamination, holes, etc. on the surface of the part to be inspected.

[0148] In this embodiment, the defect detection results of the part to be inspected are obtained based on the defect judgment results of all divided regions and the defect locations and defect types of all defect regions, including:

[0149] When any of the defect judgment results of all the divided areas are determined to be defective areas, then the defect location and defect type of all defective areas are taken as the defect detection result of the part to be inspected.

[0150] If none of the divided regions are identified as defective regions in the defect judgment results of all regions, then the absence of surface defects in the part to be inspected will be taken as the defect detection result of the part to be inspected.

[0151] The beneficial effects of the above technology are as follows: based on the shape of the divided regions in the region division results, it can be determined whether there are defects on the surface of the part to be inspected, and when it is determined that there are defects, the corresponding defect type can be further determined based on the image features of the defect region, so as to realize the defect judgment and the identification of defect location and type on the surface of the part to be inspected.

[0152] Example 9:

[0153] Based on Example 8, the method for detecting defects in hardware parts based on image recognition technology determines the defect type based on the image features of the corresponding defect area, including:

[0154] Determine the type of the corresponding defect area to determine the required data;

[0155] Based on the type, the required data is queried from the defect type determination rule table to determine the defect type of the corresponding defect area.

[0156] In this embodiment, the data required for type determination includes: the average chromaticity value, average brightness value, and shape of the defective area.

[0157] In this embodiment, the defect type determination rule table is a list containing determination rules corresponding to each defect type, for example:

[0158] The rule for judging scratch defects is: the shape of the defect area is linear;

[0159] The rule for determining foreign object occlusion defects is: the average brightness value of the corresponding defect area is less than the average brightness value of the neighboring area of ​​the defect area (i.e., the area after summing up multiple neighboring areas determined by each edge pixel of the defect area as the center and a preset length as the radius) and the shape of the corresponding defect area is not linear.

[0160] The criteria for determining color pollution defects are: the difference between the average chromaticity value of the corresponding defect area and the average chromaticity value of the neighboring areas of the defect area is greater than the difference threshold, and the shape of the corresponding defect area is not linear.

[0161] The criteria for determining a hole defect are: the average brightness value of the corresponding defect area is greater than the average brightness value of the neighboring area of ​​the defect area (i.e., the area after summing up multiple neighboring areas centered on each edge pixel of the defect area and with a preset length as the radius), and the shape of the corresponding defect area is not linear.

[0162] The beneficial effects of the above technology are: based on the data in the image features of the corresponding defect area, the defect type determination rule table is queried to determine the defect type of the defect area.

[0163] Example 10:

[0164] This invention provides a defect detection system for hardware parts based on image recognition technology, with reference to... Figure 3 ,include:

[0165] The analysis end is used to acquire video of the part to be inspected under a preset changing light source based on a preset shooting angle, and to analyze the visual change features of each pixel on the surface of the part to be inspected based on the part video.

[0166] The segmentation end is used to perform cluster analysis on the visual change features of all pixels to obtain multiple region contours corresponding to the preset shooting angle, and obtain the region segmentation result based on all region contours.

[0167] The determining end is used to determine the defect detection result based on the region shape and image features of the region in the region segmentation result.

[0168] The beneficial effects of the above technology are as follows: Based on the visual change characteristics of the part to be inspected under a preset changing light source, the surface of the part to be inspected is divided into regions, and then the defect detection results of the part to be inspected are obtained based on the region shape and region image features after the region division. Compared with the existing defect recognition methods, it is not necessary to prepare a large number of defect samples and build defect recognition models corresponding to different defect types of different parts. The defects on the surface of the part can be identified, which not only simplifies the defect recognition process, but also improves the efficiency and accuracy of defect recognition.

[0169] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for detecting defects in hardware parts based on image recognition technology, characterized in that, include: S1: Obtain a video of the part to be inspected under a preset changing light source based on a preset shooting angle; S2: Based on the video analysis of the part, the visual change features of each pixel on the surface of the part to be inspected are obtained; S3: Perform cluster analysis on the visual change features of all pixels to obtain the contours of multiple regions corresponding to the preset shooting angle; S4: Obtain the region division results based on the contours of all regions; S5: Determine the defect detection result based on the region shape and image features of the region segmentation results; S2: Based on the video analysis of the part, the visual change features of each pixel on the surface of the part to be detected are analyzed, including: Based on the completeness of the part region in each video frame of the part video, the best video frame is selected from the part video, and the two-dimensional coordinates of each pixel in the part region of the best video frame are determined. The visual values ​​corresponding to all two-dimensional coordinates in each video frame are determined. Based on the order of all video frames, all visual values ​​corresponding to the two-dimensional coordinates are sorted to obtain the visual value sequence of the corresponding pixel. The visual value sequence is used as the visual change feature of the corresponding pixel. S3: Cluster analysis is performed on the visual change features of all pixels to obtain the contours of multiple regions corresponding to the preset shooting angle, including: S301: Perform cluster analysis on the visual change features of all pixels on the surface of the part to be inspected to obtain multiple pixel clusters; S302: Based on multiple pixel clusters, determine the contours of multiple regions corresponding to the preset shooting angle on the surface of the part to be inspected; Among them, based on the completeness of the part region in each video frame of the part video, the best video frames are selected from the part videos, including: In each video frame of the part video, the outline of the part is determined. The distance from the pixel in the part outline to the center point of the part outline is used as the magnitude of the vector, and the tangent angle of the pixel in the part outline is used as the angle of the vector to obtain the local outline feature vector of the corresponding pixel. Based on the local outline feature vector of each pixel in the part outline, a feature vector sequence is obtained. Determine whether there are multiple consecutive local contour feature vectors with equal directions in the feature vector sequence. If so, obtain the first sub-feature vector sequence based on the multiple consecutive local contour feature vectors with equal directions. Otherwise, determine the angle increment and magnitude increment between adjacent local contour feature vectors in the feature vector sequence. Based on the angle increment and magnitude increment, the gradual change pattern of the angle and / or magnitude of the local contour feature vectors in the feature vector sequence is analyzed, and the second sub-feature vector sequence in the feature vector sequence with the gradual change pattern of angle and / or magnitude is determined. Summarize all first sub-feature vector sequences and all second sub-feature vector sequences in the feature vector sequence to obtain the set of sub-feature vector sequences corresponding to the part contour; The product of the reciprocal of the total number of sub-feature vector sequences in the set of sub-feature vector sequences and the sum of the total number of values ​​in all sub-feature vector sequences is used as the completeness of the part region corresponding to the part contour. The video frame containing the part area with the highest integrity in the part video is taken as the best video frame.

2. The method for detecting defects in hardware parts based on image recognition technology according to claim 1, characterized in that, S301: Perform cluster analysis on the visual change features of all pixels on the surface of the part to be inspected to obtain multiple pixel clusters, including: The visual value change curve of the corresponding pixel is fitted based on the visual value sequence in the visual change features of each pixel. Cluster analysis is performed on the visual value change curves of all pixels to obtain curve clustering results, and multiple pixel clusters are obtained based on the curve clustering results.

3. The method for detecting defects in hardware parts based on image recognition technology according to claim 1, characterized in that, S4: Obtain the region division results based on the contours of all regions, including: If there are intersecting first region contours among all the region contours corresponding to the preset shooting angles, then a supplementary video of the part to be detected under a preset changing light source is obtained based on the first region contour. This process continues until the multiple second region contours obtained based on the supplementary video do not intersect with the remaining third region contours among all region contours except for the first region contour. Then, the region division result of the surface of the part to be detected is determined based on the latest obtained second and third region contours. Otherwise, the region division result of the surface of the part to be detected is determined based on all the currently obtained region contours.

4. The method for detecting defects in hardware parts based on image recognition technology according to claim 3, characterized in that, Determine whether there are intersecting first region contours among the region contours corresponding to all preset shooting angles, including: The part region in the best video frame of the part video obtained from each preset shooting angle is matched with the standard 3D model of the part to be detected to obtain the matching result. Based on the matching result, the spatial coordinates of each region contour in the standard 3D model are determined. Based on the spatial coordinates of each region contour in the standard 3D model, determine whether there is a first region contour that intersects with each other among the region contours corresponding to all preset shooting angles.

5. The method for detecting defects in hardware parts based on image recognition technology according to claim 3, characterized in that, Based on the contour of the first region, supplementary video of the part to be detected under a preset changing light source is obtained, including: Supplementary shooting angles are determined based on the outline of the first region; Supplementary video of the part to be inspected under a preset changing light source is obtained by supplementing the shooting angle.

6. The method for detecting defects in hardware parts based on image recognition technology according to claim 1, characterized in that, S5: Based on the region shape and image features of the region segmentation results, the defect detection results are determined, including: Based on the shape of the divided regions in the region division results, it is determined whether the corresponding divided region is a defect region, and the defect judgment result is obtained. When the defect judgment result is that the corresponding divided region is a defect region, the defect type is determined based on the image features of the corresponding defect region. The defect detection results of the part to be inspected are obtained based on the defect judgment results of all divided regions, the defect location and defect type of all defect regions.

7. The method for detecting defects in hardware parts based on image recognition technology according to claim 6, characterized in that, The defect type is determined based on the image features of the corresponding defect region, including: Determine the type of the corresponding defect area to determine the required data; Based on the type, the required data is queried from the defect type determination rule table to determine the defect type of the corresponding defect area.

8. A defect detection system for hardware parts based on image recognition technology, characterized in that, A method for detecting defects in hardware parts based on image recognition technology, as described in any one of claims 1 to 7, comprises: The analysis end is used to acquire video of the part to be inspected under a preset changing light source based on a preset shooting angle, and to analyze the visual change features of each pixel on the surface of the part to be inspected based on the part video. The segmentation end is used to perform cluster analysis on the visual change features of all pixels to obtain multiple region contours corresponding to the preset shooting angle, and obtain the region segmentation result based on all region contours. The determining end is used to determine the defect detection result based on the region shape and image features of the region in the region segmentation result.

Images