A method and system for hardware parts defect detection based on computer vision

Through computer vision technology, the image processing and analysis of hardware parts is solved, and the problem of high misjudgment rate of human detection is achieved, achieving more efficient detection accuracy and production quality assurance.

CN118537322BActive Publication Date: 2024-11-15佛山市名祥五金实业有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410683445.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-29
Publication Date
2024-11-15
Estimated Expiration
2044-05-29

AI Technical Summary

Technical Problem

The inspection of existing hardware parts mainly relies on human spot checks, and the probability of misjudgment is high, making it difficult to ensure production quality.

Method used

Using a computer vision-based method, the images of hardware parts are acquired for interference cancellation and global analysis, unevenly distributed blocks are dismantled, and intelligent comparisons are performed to reduce the probability of misjudgment.

Benefits of technology

It improves the accuracy and reliability of hardware parts inspection, effectively reduces the probability of misjudgment, and ensures production quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118537322B_ABST
    Figure CN118537322B_ABST
Patent Text Reader

Abstract

The present invention provides a hardware parts defect detection method and system based on computer vision, belonging to the field of computer vision technology, and the method includes: eliminating interference on the surface image to lock the possible defect area, and performing a first global analysis on the area; determining the center point of the uneven pixel in the area, and connecting them in sequence from left to right to obtain uneven distribution blocks; disassembling the uneven distribution blocks according to independent vertical units, sequentially obtaining the pixel wrinkle coefficient of each disassembled unit, and dividing the area to obtain a number of divided areas; performing a second global analysis on each divided area; comparing and analyzing the first global feature and all the second global features with the standard global feature, respectively, to obtain the significant defects of the hardware parts. Not only does it ensure the reasonable detection of each hardware part, but it also effectively reduces the probability of misjudgment of the hardware parts, thereby ensuring the production quality of the object parts.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a method and system for hardware parts defect detection based on computer vision. Background Art

[0002] Hardware parts refer to various metal parts made by physical processing such as forging, rolling, cutting, etc. of iron, steel, aluminum, etc. Hardware parts include mold hardware parts, mechanical hardware parts, daily hardware parts, window hardware parts, ship hardware parts, construction hardware parts, aerospace hardware parts, and security products.

[0003] Different hardware parts have different manufacturing standard requirements in different application fields. That is, after the hardware parts are manufactured, they need to undergo strict appearance inspection to determine whether they can be used in related fields.

[0004] The common testing method now is to conduct subjective judgment between standard parts and manufactured parts through manual spot checks to determine whether the manufactured parts are qualified. The probability of misjudgment is extremely high, and it is difficult to ensure the quality of the produced parts.

[0005] Therefore, the present invention proposes a hardware parts defect detection method and system based on computer vision. Summary of the invention

[0006] The present invention provides a hardware parts defect detection method and system based on computer vision, which are used to determine the reliability of the image by acquiring the image of each hardware part and eliminating interference, and subsequently perform overall global analysis on the possible defect area and separate global analysis on several divided areas after the possible defect area is disassembled according to unevenly distributed blocks, so as to perform comprehensive intelligent comparison with the standard, which not only ensures the reasonable detection of each hardware part, but also effectively reduces the probability of misjudgment of hardware parts, thereby ensuring the production quality of object parts.

[0007] The present invention provides a method for detecting defects of hardware parts based on computer vision, comprising:

[0008] Step 1: Acquire a surface image of a hardware part, and perform interference elimination on the surface image to lock possible defect areas in the interference eliminated image;

[0009] Step 2: Perform a first global analysis on the possible defect area to obtain a first global feature;

[0010] Step 3: Determine the center points of the uneven pixels in the possible defect area, and connect them in order from left to right to obtain uneven distribution blocks;

[0011] Step 4: disassemble the unevenly distributed block according to independent vertical units, obtain the pixel wrinkle coefficient of each disassembled unit in turn, and divide the possible defect area to obtain a plurality of divided areas;

[0012] Step 5: Perform a second global analysis on each divided area to obtain a second global feature;

[0013] Step 6: Compare and analyze the first global feature and all the second global features with the standard global features of the hardware parts to obtain significant defects of the hardware parts.

[0014] Preferably, eliminating interference from the surface image comprises:

[0015] Acquire a rough outline of light impact distribution acting on the surface image;

[0016] Determine the light sparseness value under each light unit in the rough outline of the light impact distribution and the direction of the main shadow line in each light unit based on the corresponding light unit;

[0017] Adjusting the boundaries of corresponding light units according to the light sparseness value and direction to obtain a light impact distribution surface;

[0018] Performing corner point detection on the surface image to obtain a plurality of corner points, and respectively obtaining in-plane features based on the in-plane corner points and out-of-plane features based on the out-of-plane corner points;

[0019] Performing a first interference analysis on the in-plane features according to a standard brightness range to obtain a first interference sequence, and performing a second interference analysis on the out-of-plane features according to the standard brightness range to obtain a second interference sequence;

[0020] A circle is selected with each corner point as the center according to the preset radius to obtain the signal-to-noise ratio of the corresponding circle, and the in-plane signal-to-noise sequence and the out-of-plane signal-to-noise sequence are constructed;

[0021] Determine interference points, in-plane filter coefficients, and out-plane filter coefficients based on the first interference sequence and the in-plane signal-noise sequence, the second interference sequence, and the out-plane signal-noise sequence;

[0022]

[0023] Wherein, Mn and Mw represent the in-plane filtering coefficient and the out-of-plane filtering coefficient respectively; z1, z2, z3, and z4 represent the standardized coefficients based on the first interference sequence, the in-plane signal-noise sequence, the second interference sequence, and the out-of-plane signal-noise sequence respectively; n1, n2, n3, and n4 represent the number of values ​​contained in the first interference sequence, the in-plane signal-noise sequence, the second interference sequence, and the out-of-plane signal-noise sequence respectively; g1 i1 represents the i1th value in the first interference sequence; g2i2 represents the i2th value of the signal-to-noise sequence within the plane; g3 i3 represents the i3th value in the second interference sequence; g4 i4 represents the i4th value in the out-of-plane signal-to-noise sequence; σ1 2 Indicates that based on all g2 i2 Variance of 2 Indicates that based on all g4 i4 variance; a1 and a2 represent the adjustment amount; n01 represents the amount of inward adjustment of the boundary; n02 represents the amount of non-adjustment of the boundary; n03 represents the amount of outward adjustment of the boundary;

[0024] According to the point position of the interference point, the in-plane filtering coefficient and the out-of-plane filtering coefficient, an elimination solution is obtained by matching from an elimination database, and interference elimination processing is performed on the surface image to obtain an interference eliminated image.

[0025] Preferably, locking interference to eliminate possible defect areas in the image includes:

[0026] According to the appearance complexity of each substructure under the standard structure of the hardware parts, a frame selection unit is assigned to each substructure. At the same time, the position points of the standard structure are aligned with the coordinate points of the interference-eliminating image one by one, and after the alignment is completed, the interference-eliminating image is framed and calibrated according to the assigned frame selection unit;

[0027] Obtaining the grayscale significance of each calibration unit respectively, if the grayscale significance is greater than the corresponding preset significance, and filling the corresponding calibration unit with a specified color according to the significant relationship between the grayscale significance and the preset significance;

[0028] Otherwise, the corresponding calibration unit will not be filled with color;

[0029] All the calibrated units under the color filling results are connected to obtain possible defect areas.

[0030] Preferably, performing a first global analysis on the possible defect area to obtain a first global feature includes:

[0031] According to the part type and standard structure of the hardware part, an image analysis model is obtained from a type-structure-analysis mapping table;

[0032] A first global analysis is performed on the possible defect area according to the image analysis model to obtain a first global feature.

[0033] Preferably, determining the center point of the uneven pixel in the possible defect area includes:

[0034] Calculating the pixel uniformity centered on each pixel in each calibration unit in the possible defect area, and extracting the pixel point corresponding to the maximum value of the pixel uniformity in the same calibration unit as the center point of the uneven pixel;

[0035] At the same time, according to the designated color of the color filling unit, a first number N1 of the uneven pixel center points of each color filling unit is determined from the color-number mapping table;

[0036] From the determined grayscale change rates of each pixel point and the 8 neighboring pixels, the first N1-1 pixels with the maximum change rates are selected as the uneven pixel center points.

[0037] Preferably, the pixel wrinkle coefficient of each disassembly unit is obtained in sequence, including:

[0038] Obtain the number of separate sub-regions in the disassembly unit except the unevenly distributed blocks, and determine the fluctuation direction and degree of each pixel point in each separate sub-region based on the 8-neighborhood points, and obtain the first wrinkle coefficient of the corresponding separate sub-region;

[0039] Based on the block position, the connecting pixel points of each individual sub-region in the corresponding disassembly unit and each first block point in the block position are locked, and a second wrinkle coefficient corresponding to the first block point is obtained;

[0040] According to all the second wrinkle coefficients and the block positions, a third wrinkle coefficient of the unevenly distributed blocks in the corresponding disassembly unit is obtained;

[0041] According to the first fold coefficient and the third fold coefficient, the pixel fold coefficient of the corresponding decomposition unit is obtained.

[0042] Preferably, the possible defect area is divided into several divided areas, including:

[0043] Append each pixel wrinkle coefficient to the corresponding decomposition unit;

[0044] The disassembled units are divided and classified according to the coefficient classification table to obtain several divided areas.

[0045] Preferably, obtaining the significant defects of the hardware parts includes:

[0046] Inputting all the second global features into the feature combination model to obtain the third global feature;

[0047] Compare the first global feature with the standard global feature and the third global feature with the standard global feature to obtain a first feature difference and a second feature difference;

[0048] The first feature difference and the second feature difference are subjected to intersection processing and union processing, and a difference weight is assigned to each independent feature difference respectively, and then the difference representation of each independent feature difference is combined to obtain the significant defect of the hardware part.

[0049] The present invention provides a hardware parts defect detection system based on computer vision, comprising:

[0050] An interference elimination module is used to obtain a surface image of a hardware part and perform interference elimination on the surface image to lock possible defect areas in the interference eliminated image;

[0051] A first global analysis module, used for performing a first global analysis on the possible defect area to obtain a first global feature;

[0052] A center point determination module is used to determine the center points of the uneven pixels in the possible defect area and connect them in sequence from left to right to obtain uneven distribution blocks;

[0053] A region division module, used to disassemble the unevenly distributed blocks according to independent vertical units, obtain the pixel wrinkle coefficient of each disassembled unit in turn, and divide the possible defect area to obtain a plurality of divided areas;

[0054] A second global analysis module is used to perform a second global analysis on each divided area to obtain a second global feature;

[0055] The feature comparison and analysis module is used to compare and analyze the first global feature and all the second global features with the standard global features of the hardware parts to obtain significant defects of the hardware parts.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] The reliability of the image is determined by acquiring the image of each hardware part and eliminating interference. Subsequently, an overall global analysis of the possible defective area and a separate global analysis of several divided areas after the possible defective area is disassembled according to unevenly distributed blocks are performed to conduct a comprehensive and intelligent comparison with the standard. This not only ensures the reasonable detection of each hardware part, but also effectively reduces the probability of misjudgment of hardware parts, thereby ensuring the production quality of object parts.

[0058] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0059] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0061] Figure 1 Flow chart of a method for hardware parts defect detection based on computer vision in an embodiment of the present invention;

[0062] Figure 2 It is a structural diagram of a hardware parts defect detection system based on computer vision in an embodiment of the present invention;

[0063] Figure 3 4 is a structural diagram of an unevenly distributed block in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0065] The present invention provides a method for detecting hardware parts defects based on computer vision. Figure 1 As shown, including:

[0066] Step 1: Acquire a surface image of a hardware part, and perform interference elimination on the surface image to lock possible defect areas in the interference eliminated image;

[0067] Step 2: Perform a first global analysis on the possible defect area to obtain a first global feature;

[0068] Step 3: Determine the center points of the uneven pixels in the possible defect area, and connect them in order from left to right to obtain uneven distribution blocks;

[0069] Step 4: disassemble the unevenly distributed block according to independent vertical units, obtain the pixel wrinkle coefficient of each disassembled unit in turn, and divide the possible defect area to obtain a plurality of divided areas;

[0070] Step 5: Perform a second global analysis on each divided area to obtain a second global feature;

[0071] Step 6: Compare and analyze the first global feature and all the second global features with the standard global features of the hardware parts to obtain significant defects of the hardware parts.

[0072] In this embodiment, the surface image is obtained by photographing the hardware parts with a camera, and the hardware parts can be parts in any field. Defect detection of the parts is mainly performed by computer vision.

[0073] In this embodiment, interference elimination refers to eliminating the interference caused by light and shadow in the surface image and restoring the blur of image position A caused by defocus blur, motion blur, etc. of the photographic equipment during the image capture process. In the case of distortion, image enhancement or optimization of the imaging process can be used to achieve it.

[0074] In this embodiment, the possible defect area refers to a continuous area block in the interference elimination image, wherein the possible defect refers to the abnormal appearance of the hardware parts, such as roughness, missing parts, redundant parts, etc.

[0075] In this embodiment, the first global analysis is implemented based on an image analysis model, and the image analysis model is obtained by training a neural network model based on a global threshold segmentation method by analyzing several historical part images as samples. Therefore, a global analysis of possible defect areas can be achieved to obtain the defect characteristics of the area.

[0076] In this embodiment, the non-uniform pixel center point refers to a point in a possible defect area where a grayscale value is obviously non-uniform compared with a periodic pixel value, and a point where the pixel grayscale change rate is obviously too high. The points in these two cases are regarded as non-uniform pixel points, mainly to capture more early parameters to facilitate reasonable analysis of defects in the later stage.

[0077] In this embodiment, if Figure 3 As shown, in the order from left to right and from top to bottom, there are uneven pixel center points in vertical position 1, vertical position 2, vertical position 3, vertical position 4, and vertical position 6, and the numbers are 2, 1, 1, and 1 respectively, and there is no uniform pixel center point in vertical position 5, wherein the black dot represents the uneven pixel center point. At this time, these points are connected in sequence from left to right, and the obtained B0 is the uneven distribution block. In the process of obtaining the uneven distribution block, the points involved in each vertical position are included in the widest range. For example, for the three uneven pixel center points in vertical position 4, at this time, the interval composed of the uppermost point and the lowermost point based on the vertical position 4 is locked as a part of the uneven distribution block, and Figure 3 The shaded rectangular parts connected in sequence are the unevenly distributed blocks.

[0078] In this embodiment, the independent vertical unit refers to vertically disassembling the unevenly distributed blocks by the width of two pixels in the order from left to right to obtain a plurality of disassembly units, that is, Figure 3In the figure, the vertical positions 1 and 2 form a whole unit, the vertical positions 3 and 4 form a whole unit, and the vertical positions 5 and 6 form a whole unit.

[0079] In this embodiment, the pixel wrinkle coefficient is determined based on the fluctuation of the pixels and the connection of the pixels on the unevenly distributed blocks.

[0080] In this embodiment, after the pixel wrinkle coefficient of each disassembly unit is determined, the coefficient is divided into levels to obtain a number of divided areas, and the divided areas refer to the division of the disassembly unit at a continuous and same level.

[0081] In this embodiment, the second global analysis is obtained based on the image analysis model analysis to obtain several second global features, namely, one is a direct analysis of the entire image, and the other is a separate analysis of several sub-images after the image is split. The analysis of the same image is achieved from two aspects, thereby improving the reliability of subsequent defect analysis.

[0082] In this embodiment, the standard global feature refers to the feature that the hardware parts are in a qualified state, which is determined before the hardware parts are manufactured, and provides a comparison basis for the hardware scarves produced in the subsequent actual manufacturing process.

[0083] The beneficial effect of the above technical solution is: by obtaining the image of each hardware part and eliminating interference, the reliability of the image is determined, and subsequently the overall global analysis of the possible defective area and the individual global analysis of several divided areas after the possible defective area is disassembled according to the unevenly distributed blocks are carried out to make a comprehensive and intelligent comparison with the standard, which not only ensures the reasonable detection of each hardware part, but also effectively reduces the probability of misjudgment of hardware parts, thereby ensuring the production quality of object parts.

[0084] The present invention provides a method for detecting defects of hardware parts based on computer vision, which eliminates interference of the surface image, comprising:

[0085] Acquire a rough outline of light impact distribution acting on the surface image;

[0086] Determine the light sparseness value under each light unit in the rough outline of the light impact distribution and the direction of the main shadow line in each light unit based on the corresponding light unit;

[0087] Adjusting the boundaries of corresponding light units according to the light sparseness value and direction to obtain a light impact distribution surface;

[0088] Performing corner point detection on the surface image to obtain a plurality of corner points, and respectively obtaining in-plane features based on the in-plane corner points and out-of-plane features based on the out-of-plane corner points;

[0089] Performing a first interference analysis on the in-plane features according to a standard brightness range to obtain a first interference sequence, and performing a second interference analysis on the out-of-plane features according to the standard brightness range to obtain a second interference sequence;

[0090] A circle is selected with each corner point as the center according to the preset radius to obtain the signal-to-noise ratio of the corresponding circle, and the in-plane signal-to-noise sequence and the out-of-plane signal-to-noise sequence are constructed;

[0091] Determine interference points, in-plane filter coefficients, and out-plane filter coefficients based on the first interference sequence and the in-plane signal-noise sequence, the second interference sequence, and the out-plane signal-noise sequence;

[0092]

[0093] Wherein, Mn and Mw represent the in-plane filtering coefficient and the out-of-plane filtering coefficient respectively; z1, z2, z3, and z4 represent the standardized coefficients based on the first interference sequence, the in-plane signal-noise sequence, the second interference sequence, and the out-of-plane signal-noise sequence respectively; n1, n2, n3, and n4 represent the number of values ​​contained in the first interference sequence, the in-plane signal-noise sequence, the second interference sequence, and the out-of-plane signal-noise sequence respectively; g1 i1 represents the i1th value in the first interference sequence; g2 i2 represents the i2th value of the signal-to-noise sequence within the plane; g3 i3 represents the i3th value in the second interference sequence; g4 i4 represents the i4th value in the out-of-plane signal-to-noise sequence; σ1 2 Indicates that based on all g2 i2 Variance of 2 Indicates that based on all g4 i4 variance; a1 and a2 represent the adjustment amount; n01 represents the amount of inward adjustment of the boundary; n02 represents the amount of non-adjustment of the boundary; n03 represents the amount of outward adjustment of the boundary;

[0094] According to the point position of the interference point, the in-plane filtering coefficient and the out-of-plane filtering coefficient, an elimination solution is obtained by matching from an elimination database, and interference elimination processing is performed on the surface image to obtain an interference eliminated image.

[0095] In this embodiment, the rough outline of light impact distribution refers to the fact that shooting in the current lighting environment based on the production, preparation and output of hardware parts will have a certain impact on the shooting, that is, the situation where light shadows fall on the image based on the shooting angle of the shooting device and the hardware parts, or the shadows of other equipment fall on the hardware parts, and then the outline of the existing light shadows is regarded as the rough outline of light impact distribution. At this time, the outline is only roughly determined.

[0096] In this embodiment, the light unit refers to a light unit obtained by dividing a rough outline into smaller pieces and then magnifying the same. The light sparseness value = the shadow area in the corresponding light unit / the total area of ​​the corresponding light unit.

[0097] In this embodiment, the main shadow line refers to the situation where there are multiple light sources in the production environment of the hardware parts during the shooting process. At this time, there may be multiple shadows for the same object, and the boundary line of the darkest shadow is taken as the main shadow line.

[0098] In this embodiment, the direction of the corresponding light unit refers to the angle between the main shadow line and the light unit.

[0099] In this embodiment, the boundary adjustment is obtained from a value-direction-adjustment mapping table, which includes boundary adjustment directions under different light sparseness values ​​and direction combinations, and the adjustment directions are generally: inward adjustment, no adjustment, and outward adjustment. Inward adjustment refers to selecting the endpoint of the main shadow line close to the light impact distribution surface, and moving the other endpoint inward according to this endpoint, and the inward approach position is vertically horizontal with the point.

[0100] Outward adjustment means selecting the endpoint of the main shadow line that is farthest from the light impact distribution surface, and moving the other endpoint outward according to the endpoint, and the outward moving position is vertically horizontal with the point.

[0101] In this embodiment, the area after the boundary adjustment is the light-affected boundary surface.

[0102] In this embodiment, a corner detection algorithm is used to detect corners of a surface image, which belongs to the prior art.

[0103] In this embodiment, both the in-plane features and the out-of-plane features can be directly obtained based on the corner point detection algorithm. At the same time, it is also necessary to clarify the positional relationship between the corner points and the plane. The positional relationship includes whether the corner points are in the plane and whether the corner points are not in the plane, and then determine the in-plane corner points and out-of-plane corner points.

[0104] In this embodiment, the standard brightness range is pre-set, and the standard features with shadows and the standard features without shadows based on the standard brightness range can be obtained from the standard brightness-feature part mapping table. Therefore, the standard features are used to compare and analyze the in-plane and out-of-plane features respectively to determine the existing interference and form an interference sequence, wherein the standard brightness-feature part mapping table contains features of standard parts in shadow and non-shadow compared with different standard brightness, which are pre-stored as a basis for comparative analysis.

[0105] In this embodiment, the interference sequence refers to the difference based on several sub-features after the standard feature is compared with the in-plane feature or the out-of-plane feature, and the interference sequence = {difference values ​​of several sub-sequences}.

[0106] For example, the in-plane features include: sub-feature 01, sub-feature 02 and sub-feature 03, and the standard features of the shadow include: sub-standard 11, sub-standard 12, sub-standard 13. At this time, the first interference sequence = {the difference value between sub-feature 01 and sub-standard 11, the difference value between sub-feature 02 and sub-standard 12, the difference value between sub-feature 03 and sub-standard 13}, and the difference value = 1-the similarity value between the same sub-feature and the standard sub-feature. The principle of the second interference sequence is similar to that of the first interference sequence, and will not be repeated here.

[0107] In this embodiment, the preset radius is related to the size of the hardware part. If the part is 5 cm in size, a circular frame is selected with a radius of 3 mm to retain as many analysis samples as possible.

[0108] In this embodiment, the signal-to-noise ratio refers to the signal-to-noise ratio obtained by averaging the signal-to-noise ratio of each pixel point in the corresponding circle, and the signal-to-noise ratio of the pixel point is achieved by using the PSNR technology, which belongs to the prior art.

[0109] In this embodiment, the in-plane signal-to-noise sequence = {the signal-to-noise ratio of all framed circles in the plane}.

[0110] Out-of-plane signal-to-noise sequence = {Signal-to-noise ratio of all selected circles outside the plane}.

[0111] In this embodiment, the interference points are obtained by determining the pixel points with the maximum value and the second maximum value in the first interference sequence, the pixel points with the maximum signal-to-noise ratio and the second maximum signal-to-noise ratio in the in-plane signal-to-noise sequence, the pixel points with the maximum value and the second maximum value in the second interference sequence, the pixel points with the maximum signal-to-noise ratio and the second maximum signal-to-noise ratio in the out-of-plane signal-to-noise sequence, and performing overlapping analysis of the in-plane pixels and the out-of-plane pixels.

[0112] In this embodiment, the elimination database includes in-plane filtering coefficients, out-of-plane filtering coefficients, point positions of interference points and matching elimination schemes in different combinations, so as to realize interference elimination processing of the surface image and ensure the reliability of the image in subsequent analysis. The elimination scheme is to eliminate various irrelevant information such as noise and shadows in a certain manner inside and outside the plane.

[0113] The beneficial effects of the above technical solution are: boundary adjustment is achieved by analyzing the light sparse value and direction of each light unit on the rough contour, and an interference sequence is constructed by detecting the image corner points. At the same time, the signal-to-noise ratio analysis is continued to construct a signal-to-noise sequence with the corner points as the center. The interference points inside and outside the plane and the filtering sparseness are comprehensively determined from two aspects to ensure the targeted elimination processing of the inside and outside the plane by the acquired solution, thereby ensuring the reliability of image acquisition and providing a basis for subsequent analysis.

[0114] The present invention provides a hardware parts defect detection method based on computer vision, which locks possible defect areas in interference elimination images, comprising:

[0115] According to the appearance complexity of each substructure under the standard structure of the hardware parts, a frame selection unit is assigned to each substructure. At the same time, the position points of the standard structure are aligned with the coordinate points of the interference-eliminating image one by one, and after the alignment is completed, the interference-eliminating image is framed and calibrated according to the assigned frame selection unit;

[0116] Obtaining the grayscale significance of each calibration unit respectively, if the grayscale significance is greater than the corresponding preset significance, and filling the corresponding calibration unit with a specified color according to the significant relationship between the grayscale significance and the preset significance;

[0117] Otherwise, the corresponding calibration unit will not be filled with color;

[0118] All the calibrated units under the color filling results are connected to obtain possible defect areas.

[0119] In this embodiment, the standard structure refers to a qualified model structure of the hardware parts before production.

[0120] In this embodiment, because each hardware part may be composed of multiple parts, and each part is regarded as a substructure and has an appearance complexity, which is set in advance, the higher the complexity, the smaller the corresponding box selection unit (the unit division unit that realizes the set unit), that is, the higher the accuracy required for analysis.

[0121] In this embodiment, the purpose of one-to-one registration is to achieve one-to-one correspondence between the image and the structure based on the same position, thereby facilitating frame selection and calibration of the interference-eliminating image according to corresponding units.

[0122] In this embodiment, gray significance = (the maximum gray value of all pixels in the corresponding calibration unit + the average gray value of all pixels in the corresponding calibration unit) / 2,

[0123] In this embodiment, the preset significance is preset.

[0124] In this embodiment, the significant relationship refers to grayscale significance / preset significance, and the corresponding color is obtained from the ratio-color table for filling according to the ratio, that is, the larger the ratio, the more significant the relationship.

[0125] In this embodiment, the connectivity processing refers to performing regional connectivity processing on all calibration units so that they form an overall connected region and are regarded as a possible defect region.

[0126] The beneficial effects of the above technical solution are: by performing one-to-one registration between the standard structure and the image, it is convenient to frame the image with different degrees of precision according to the frame selection unit, thereby ensuring the effectiveness of subsequent analysis, and by determining the significant relationship for color filling and connectivity processing, it is convenient to obtain a complete possible defect area, thereby facilitating subsequent global analysis.

[0127] The present invention provides a hardware parts defect detection method based on computer vision, which performs a first global analysis on the possible defect area to obtain a first global feature, including:

[0128] According to the part type and standard structure of the hardware part, an image analysis model is obtained from a type-structure-analysis mapping table;

[0129] A first global analysis is performed on the possible defect area according to the image analysis model to obtain a first global feature.

[0130] In this embodiment, the part type is, for example, an aviation type, and the standard structure is planned by experts before production.

[0131] In this embodiment, the type-structure-analysis mapping table includes different part types, standard structures under the type, and image analysis models set by experts for matching, and is a pre-planned table.

[0132] The beneficial effect of the above technical solution is: by obtaining the model from the mapping table and then performing regional global analysis based on the model, it is convenient to provide an overall global feature, which provides a basis for subsequent defect acquisition.

[0133] The present invention provides a hardware parts defect detection method based on computer vision, which determines the center point of the uneven pixels in the possible defect area, including:

[0134] Calculating the pixel uniformity centered on each pixel in each calibration unit in the possible defect area, and extracting the pixel point corresponding to the maximum value of the pixel uniformity in the same calibration unit as the center point of the uneven pixel;

[0135] At the same time, according to the designated color of the color filling unit, a first number N1 of the uneven pixel center points of each color filling unit is determined from the color-number mapping table;

[0136] From the determined grayscale change rates of each pixel point and the 8 neighboring pixels, the first N1-1 pixels with the maximum change rates are selected as the uneven pixel center points.

[0137] In this embodiment, Among them, JD j1 represents the pixel uniformity of the j1th pixel in the corresponding calibration unit; m1 represents the number of remaining pixels in the corresponding calibration unit except the j1th pixel; r1 j1 Represents the gray value of the j1th pixel in the corresponding calibration unit; r2 j2 represents the gray value of the j2th remaining pixel in the corresponding calibration unit; Represents all r2 j2 The variance of .

[0138] In this embodiment, the color-number mapping table includes the number of uneven pixel center points matching the color, which can be directly obtained.

[0139] In this embodiment, the grayscale change rate r4 j1 Represents the gray value of the j3rd neighborhood pixel; r3 o Indicates the grayscale value of the corresponding pixel.

[0140] The beneficial effect of the above technical solution is: by comprehensively screening out N1 uneven pixel center points from the two aspects of uniformity and grayscale change rate, a basis is provided for the subsequent analysis of the wrinkle system of each disassembled unit, and then the global characteristics of each divided area are effectively analyzed, providing an effective basis for the subsequent comparative analysis with the standard global characteristics, thereby indirectly improving the analysis efficiency.

[0141] The present invention provides a hardware parts defect detection method based on computer vision, which sequentially obtains the pixel wrinkle coefficient of each disassembled unit, including:

[0142] Obtain the block positions of the unevenly distributed blocks involved in each disassembly unit respectively;

[0143] Obtain the number of separate sub-regions in the disassembly unit except the unevenly distributed blocks, and determine the fluctuation direction and degree of each pixel point in each separate sub-region based on the 8-neighborhood points, and obtain the first wrinkle coefficient of the corresponding separate sub-region;

[0144] Based on the block position, the connecting pixel points of each individual sub-region in the corresponding disassembly unit and each first block point in the block position are locked, and a second wrinkle coefficient corresponding to the first block point is obtained;

[0145] According to all the second wrinkle coefficients and the block positions, a third wrinkle coefficient of the unevenly distributed blocks in the corresponding disassembly unit is obtained;

[0146] According to the first fold coefficient and the third fold coefficient, the pixel fold coefficient of the corresponding decomposition unit is obtained.

[0147] In this embodiment, the block position is Figure 3 The block distribution of each disassembled unit.

[0148] In this embodiment, the number of separate sub-areas is 0, 1 or 2. If the unevenly distributed block completely covers the disassembly unit, the number is 0. If it partially covers and two upper and lower spaces are reserved, the number is 2. If it partially covers and one space is reserved, the number is 1.

[0149] In this embodiment, the boundary detection method Canny can be used to obtain the fluctuation intensity (gradient intensity) and fluctuation direction of each pixel point.

[0150] At this time, the first wrinkle coefficient = max (the degree of fluctuation of the 8 neighboring points) × sin (|the angle between the vector arrow of the fluctuation direction of the 8 neighboring points and the horizontal line|), where the angle is 0 to Π.

[0151] In this embodiment, the connected pixel points refer to all pixel points connected to the block position in a separate sub-area, and the second wrinkle coefficient = the average fluctuation value of all connected pixel points × sin(|the angle between the fluctuation direction corresponding to the maximum fluctuation degree among all connected pixel points and the horizontal line|).

[0152] In this embodiment, the third fold coefficient is obtained by averaging all accumulated sums of the second fold coefficient multiplied by the corresponding block position weight.

[0153] In this embodiment, the pixel fold coefficient=(first fold coefficient+third fold coefficient) / 2.

[0154] The beneficial effect of the above technical solution is: starting from the 8-neighborhood points of the individual sub-area and the connecting pixel points of the first block point, the wrinkle coefficient is determined, and then the pixel wrinkle coefficient is obtained, thereby ensuring the accuracy of the acquired exclamation.

[0155] The present invention provides a hardware parts defect detection method based on computer vision, which divides the possible defect area to obtain several divided areas, including:

[0156] Append each pixel wrinkle coefficient to the corresponding decomposition unit;

[0157] The disassembled units are divided and classified according to the coefficient classification table to obtain several divided areas.

[0158] In this embodiment, the purpose of adding coefficients is to effectively classify and divide the units.

[0159] In this embodiment, the coefficient classification level table includes different coefficients and the levels to which the coefficients belong.

[0160] The beneficial effect of the above technical solution is: by dividing the disassembly units according to the pixel wrinkle coefficient, the divided areas can be effectively acquired.

[0161] The present invention provides a method for detecting defects of hardware parts based on computer vision, which obtains significant defects of the hardware parts, comprising:

[0162] Inputting all the second global features into the feature combination model to obtain the third global feature;

[0163] Compare the first global feature with the standard global feature and the third global feature with the standard global feature to obtain a first feature difference and a second feature difference;

[0164] The first feature difference and the second feature difference are subjected to intersection processing and union processing, and a difference weight is assigned to each independent feature difference respectively, and then the difference representation of each independent feature difference is combined to obtain the significant defect of the hardware part.

[0165] In this embodiment, the feature combination model is obtained by training the neural network model based on the features of different divided areas and the combined analysis of all features by experts as samples, thereby obtaining the third global feature.

[0166] In this embodiment, the standard global features are pre-set and serve as a reference basis for comparison.

[0167] In this embodiment, the characteristic difference refers to the first difference from the standard, and the third difference from the standard.

[0168] In this embodiment, for example, the first feature difference is: difference 1, difference 2, difference 3, and the second feature difference is: difference 1, difference 4, difference 2. At this time, the handover result is: difference 1, difference 2, and the processing result is: difference 1, difference 2, difference 3, difference 4.

[0169] In this embodiment, the difference weight=the probability of occurrence of the corresponding feature difference based on the intersection result and the union result×the originally set feature weight.

[0170] And if it appears in the intersection result and in the union result, then the probability of occurrence is (1+1) / 2=1.

[0171] In this embodiment, the difference representation is obtained based on a feature-difference representation mapping table, which includes different features and representation symbols based on the differences under the features, and the difference weight is used to reasonably highlight the representation symbols. The larger the weight, the more obvious the corresponding significance calibration result.

[0172] The beneficial effect of the above technical solution is: by combining the second global features, and then comparing the third global features and the first global features with the standard global features respectively, and combining the intersection processing and the union processing to assign weights, the significance representation is achieved, and the reasonable output of the defects of the hardware parts is effectively guaranteed. And through the feature comparison of the two methods, the integrity and efficiency of the defect acquisition are guaranteed.

[0173] The present invention provides a hardware parts defect detection system based on computer vision, such as Figure 2 As shown, including:

[0174] An interference elimination module is used to obtain a surface image of a hardware part and perform interference elimination on the surface image to lock possible defect areas in the interference eliminated image;

[0175] A first global analysis module, used for performing a first global analysis on the possible defect area to obtain a first global feature;

[0176] A center point determination module is used to determine the center points of the uneven pixels in the possible defect area and connect them in sequence from left to right to obtain uneven distribution blocks;

[0177] A region division module, used to disassemble the unevenly distributed blocks according to independent vertical units, obtain the pixel wrinkle coefficient of each disassembled unit in turn, and divide the possible defect area to obtain a plurality of divided areas;

[0178] A second global analysis module is used to perform a second global analysis on each divided area to obtain a second global feature;

[0179] The feature comparison and analysis module is used to compare and analyze the first global feature and all the second global features with the standard global features of the hardware parts to obtain significant defects of the hardware parts.

[0180] The beneficial effect of the above technical solution is: by obtaining the image of each hardware part and eliminating interference, the reliability of the image is determined, and subsequently the overall global analysis of the possible defective area and the individual global analysis of several divided areas after the possible defective area is disassembled according to the unevenly distributed blocks are carried out to make a comprehensive and intelligent comparison with the standard, which not only ensures the reasonable detection of each hardware part, but also effectively reduces the probability of misjudgment of hardware parts, thereby ensuring the production quality of object parts.

[0181] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A hardware parts defect detection method based on computer vision, characterized in that: include: Step 1: Acquire a surface image of a hardware part and perform interference elimination on the surface image to locate possible defect areas in the interference eliminated image; Step 2: Perform a first global analysis on the possible defect area to obtain a first global feature; Step 3: Determine the center points of the uneven pixels in the possible defect area and connect them in order from left to right to obtain unevenly distributed blocks; Step 4: Decomposing the unevenly distributed block according to independent vertical units, obtaining the pixel wrinkle coefficient of each decomposed unit in turn, and dividing the possible defect area to obtain a number of divided areas; Step 5: Perform a second global analysis on each divided area to obtain a second global feature; Step 6: Compare and analyze the first global feature and all the second global features with the standard global features of the hardware part to obtain significant defects of the hardware part; Wherein, step 2 includes: Obtaining an image analysis model from a type-structure-analysis mapping table according to the part type and standard structure of the hardware part; Performing a first global analysis on the possible defect area according to the image analysis model to obtain a first global feature; The pixel wrinkle coefficient of each disassembly unit is obtained in turn, including: Obtain the block positions of the unevenly distributed blocks involved in each disassembly unit respectively; Obtain the number of individual sub-regions in the disassembly unit excluding the unevenly distributed blocks, and determine the fluctuation direction and degree of each pixel point in each individual sub-region based on the eight neighborhood points, and obtain a first wrinkle coefficient corresponding to the individual sub-region; Based on the block position, locking the connecting pixel points of each individual sub-region in the corresponding disassembly unit and each first block point in the block position, and obtaining a second wrinkle coefficient corresponding to the first block point; Obtaining a third wrinkle coefficient of the unevenly distributed blocks in the corresponding disassembled unit according to all the second wrinkle coefficients and the block positions; Obtaining a pixel fold coefficient of a corresponding decomposition unit according to the first fold coefficient and the third fold coefficient; The possible defect area is divided into several divided areas, including: Append each pixel wrinkle coefficient to the corresponding decomposition unit; The disassembled units are divided and classified according to the coefficient classification table to obtain several division areas; Wherein, obtaining the significant defects of the hardware parts includes: Inputting all the second global features into the feature combination model to obtain the third global feature; Comparing the first global feature with the standard global feature and the third global feature with the standard global feature, respectively, to obtain a first feature difference and a second feature difference; The first feature difference and the second feature difference are subjected to intersection processing and union processing, and a difference weight is assigned to each independent feature difference respectively, and then the difference representation of each independent feature difference is combined to obtain the significant defect of the hardware part.

2. The method for hardware parts defect detection based on computer vision according to claim 1, characterized in that: Eliminating interference on the surface image, comprising: obtaining a rough outline of light impact distribution acting on the surface image; Determining a light sparseness value under each light unit in the rough outline of the light impact distribution and a direction of a main shadow line in each light unit based on the corresponding light unit; Adjusting the boundaries of corresponding light units according to the light sparseness value and direction to obtain a light impact distribution surface; Performing corner point detection on the surface image to obtain a plurality of corner points, and respectively obtaining in-plane features based on the in-plane corner points and out-of-plane features based on the out-of-plane corner points; Performing a first interference analysis on the in-plane features according to a standard brightness range to obtain a first interference sequence, and simultaneously performing a second interference analysis on the out-of-plane features according to the standard brightness range to obtain a second interference sequence; A circle is selected with a preset radius centered at each corner point to obtain the signal-to-noise ratio of the corresponding circle, and an in-plane signal-to-noise sequence and an out-of-plane signal-to-noise sequence are constructed. Determining interference points, in-plane filter coefficients, and out-of-plane filter coefficients based on the first interference sequence and the in-plane signal-noise sequence, the second interference sequence, and the out-of-plane signal-noise sequence; ; ; ; ; in, 、 represent the in-plane filtering coefficient and out-of-plane filtering coefficient respectively; 、 、 、 They represent the normalization coefficients based on the first interference sequence, the in-plane signal-noise sequence, the second interference sequence, and the out-of-plane signal-noise sequence, respectively; Respectively represent the number of values ​​contained in the first interference sequence, the in-plane signal-noise sequence, the second interference sequence, and the out-of-plane signal-noise sequence; represents the i1th value in the first interference sequence; represents the i2th value of the signal-to-noise sequence within the plane; represents the i3th value in the second interference sequence; represents the i4th value in the out-of-plane signal-to-noise sequence; Indicates that based on all variance; Indicates that based on all variance; a1 and a2 represent the adjustment amount; n01 represents the amount of inward adjustment of the boundary; n02 represents the amount of non-adjustment of the boundary; n03 represents the amount of outward adjustment of the boundary; According to the point position of the interference point, the in-plane filtering coefficient and the out-of-plane filtering coefficient, an elimination solution is obtained by matching from an elimination database, and interference elimination processing is performed on the surface image to obtain an interference eliminated image.

3. The method for hardware parts defect detection based on computer vision according to claim 1, characterized in that: Locking interference eliminates possible defect areas in the image, including: Assigning a frame selection unit to each substructure according to the appearance complexity of each substructure under the standard structure of the hardware part, and at the same time, aligning the position points of the standard structure with the coordinate points of the interference-removed image one by one, and after the alignment is completed, performing frame selection calibration on the interference-removed image according to the assigned frame selection unit; Obtaining the grayscale significance of each calibration unit respectively, if the grayscale significance is greater than the corresponding preset significance, and filling the corresponding calibration unit with a specified color according to the significant relationship between the grayscale significance and the preset significance; Otherwise, the corresponding calibration unit will not be filled with color; All the calibrated units under the color filling results are connected to obtain possible defect areas.

4. The method for hardware parts defect detection based on computer vision according to claim 3, characterized in that: Determining the center point of the uneven pixels in the possible defect area includes: Calculating the pixel uniformity centered on each pixel in each calibration unit in the possible defect area, and extracting the pixel point corresponding to the maximum value of the pixel uniformity in the same calibration unit as the center point of the uneven pixel; At the same time, according to the designated color of the color filling unit, a first number N1 of the uneven pixel center points of each color filling unit is determined from the color-number mapping table; From the determined grayscale change rates of each pixel point and the eight neighboring pixel points, the first N1-1 pixels with the largest change rates are selected as the uneven pixel center points.

5. A hardware parts defect detection system based on computer vision, characterized in that: include: An interference elimination module is used to obtain a surface image of a hardware part and perform interference elimination on the surface image to locate possible defect areas in the interference eliminated image; A first global analysis module, configured to perform a first global analysis on the possible defect area to obtain a first global feature; A center point determination module is used to determine the center points of the uneven pixels in the possible defect area and connect them in sequence from left to right to obtain unevenly distributed blocks; A region division module is used to disassemble the unevenly distributed blocks according to independent vertical units, obtain the pixel wrinkle coefficient of each disassembled unit in turn, and divide the possible defect area to obtain a plurality of divided areas; A second global analysis module is used to perform a second global analysis on each divided area to obtain a second global feature; a feature comparison and analysis module, configured to compare and analyze the first global feature and all the second global features with the standard global features of the hardware part, to obtain significant defects of the hardware part; The first global analysis module is specifically used to: Obtaining an image analysis model from a type-structure-analysis mapping table according to the part type and standard structure of the hardware part; Performing a first global analysis on the possible defect area according to the image analysis model to obtain a first global feature; Among them, the area division module is specifically used to: Obtain the block positions of the unevenly distributed blocks involved in each disassembly unit respectively; Obtain the number of individual sub-regions in the disassembly unit excluding the unevenly distributed blocks, and determine the fluctuation direction and degree of each pixel point in each individual sub-region based on the eight neighborhood points, and obtain a first wrinkle coefficient corresponding to the individual sub-region; Based on the block position, locking the connecting pixel points of each individual sub-region in the corresponding disassembly unit and each first block point in the block position, and obtaining a second wrinkle coefficient corresponding to the first block point; Obtaining a third wrinkle coefficient of the unevenly distributed blocks in the corresponding disassembled unit according to all the second wrinkle coefficients and the block positions; Obtaining a pixel fold coefficient of a corresponding decomposition unit according to the first fold coefficient and the third fold coefficient; The area division module is also specifically used for: Append each pixel wrinkle coefficient to the corresponding decomposition unit; The disassembled units are divided and classified according to the coefficient classification table to obtain several division areas; Among them, the feature comparison and analysis module is specifically used to: Inputting all the second global features into the feature combination model to obtain the third global feature; Comparing the first global feature with the standard global feature and the third global feature with the standard global feature, respectively, to obtain a first feature difference and a second feature difference; The first feature difference and the second feature difference are subjected to intersection processing and union processing, and a difference weight is assigned to each independent feature difference respectively, and then the difference representation of each independent feature difference is combined to obtain the significant defect of the hardware part.