A robot part quality detection system based on machine vision

Abstract

The application discloses a kind of robot parts quality detection systems based on machine vision, it is related to machine vision technical field, for solving the problem that it is difficult to accurately distinguish abnormal bright area generated by light reflection and the real defect of part surface in detection process, part detection image is collected by machine vision, the abnormal area of gray is detected using bright area identification module and the bright area position sequence is generated in combination with part surface coordinate system, the theoretical highlight drift trajectory is constructed according to the light incidence direction and the local normal distribution of curved surface by light analysis module, the actual bright area position is matched with the theoretical highlight trajectory in space, the real defect candidate area is screened in combination with bright area edge gradient sharpness, and the light direction is supplemented according to the highlight blocking relationship for insufficient confidence bright area, the change state of bright area is verified by light supplement imaging, the accuracy of robot part surface defect detection and the anti-highlight interference capability are improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and more specifically, to a machine vision-based quality inspection system for robot parts. Background Technology

[0002] As industrial robots are increasingly used in precision manufacturing, automated assembly, and parts processing quality control, machine vision-based parts surface quality inspection technology has gradually become an important means of automated inspection. Existing inspection methods typically use industrial cameras to collect images of parts surfaces and utilize information such as image grayscale changes, texture differences, and edge features to identify surface defects, thereby achieving automatic detection of scratches, dents, cracks, and abnormal surface areas.

[0003] The existing technology has the following shortcomings: Currently, in the process of robot component inspection, due to the influence of changes in the surface curvature of the components, the reflective properties of the materials, and the changes in the illumination angle during robot movement, bright areas that move with changes in illumination conditions are easily generated in the acquired images. Since the real defect area and the bright area are similar in visual features, it is difficult to accurately distinguish between the abnormal bright area caused by light reflection and the real defects on the surface of the component during the inspection process, which reduces the accuracy and reliability of visual inspection of complex curved surface components. Therefore, a robot component quality inspection system based on machine vision is proposed.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a robot component quality inspection system based on machine vision. This system addresses the problems mentioned in the background art by employing a bright area feature extraction method based on component surface spatial mapping, a method for constructing theoretical specular change trajectories by combining surface normal distribution and illumination incidence relationship, a defect screening method that integrates actual bright area positions with theoretical specular trajectories, and a suspected area verification and correction method based on supplementary illumination response.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a robot parts quality inspection system based on machine vision, comprising a bright area recognition module, an illumination analysis module, a defect screening module, and a correction judgment module, wherein the modules are electrically connected to each other. The bright area recognition module is used to detect gray-scale abnormal areas in each frame of the image and identify suspicious bright areas. It collects the image coordinates of the reference feature points on the surface of the part, performs inter-frame geometric registration to map the suspicious bright areas to the coordinate system of the part surface, generates a normalized position sequence of the bright area surface, and inputs the normalized position sequence of the bright area surface into the defect screening module. The illumination analysis module is used to read the incident direction of illumination in each frame, obtain the local normal distribution of the surface, calculate the surface position where the theoretical specular highlight should appear in each frame in combination with the incident direction of illumination in each frame, and generate the theoretical specular highlight drift trajectory. The theoretical specular highlight drift trajectory is then transmitted to the defect screening module. The defect screening module performs spatial matching between the normalized position sequence of the bright area surface and the theoretical specular drift trajectory, detects the edge gradient sharpness of suspicious bright areas, and after screening the real defect candidate list by combining the spatial matching results and edge gradient sharpness, marks the insufficient confidence bright areas with contradictory trajectory evidence and passes the insufficient confidence bright areas to the correction judgment module. The correction and judgment module reverses the direction of supplementary lighting for bright areas with insufficient confidence based on the geometric relationship of specular occlusion and triggers supplementary lighting imaging. It collects the gray value peak of the bright area under supplementary lighting imaging, and determines whether the bright area is removed with supplementary lighting based on the gray value peak of the bright area and performs loop correction on the real defect candidate list.

[0007] In a preferred embodiment, the bright area recognition module receives images from the industrial camera during the detection cycle. Collect feature point cloud data of the part surface, extract several reference feature points from the point cloud data, and obtain the three-dimensional coordinates of the reference feature points in the coordinate system of the part surface. Perform grayscale anomaly detection on each frame of the image sequence. For each pixel in the image, read its grayscale value and calculate the average grayscale value of all pixels in the neighborhood of that pixel as the local background grayscale average value. The grayscale deviation is obtained by subtracting the grayscale value of a pixel from the mean grayscale value of the local background and taking the absolute value. If the grayscale deviation is greater than the grayscale abnormality threshold, the pixel is marked as an abnormal pixel.

[0008] In a preferred embodiment, the bright area identification module performs connected component analysis on all locations marked as abnormal pixels in the image, and clusters spatially connected abnormal pixels into a suspicious bright area. Extract the image coordinates of the reference feature points in each frame, perform inter-frame geometric registration, and locate the coordinates of the reference feature points on the image plane using a corner detection algorithm; Establish a correspondence between the image coordinates of the reference feature points in the image and the three-dimensional coordinates of the reference feature points in the coordinate system of the part surface. Use the RANSAC algorithm to solve the homography matrix. For all suspicious bright areas identified in the image, substitute the center pixel coordinates into the homography matrix to obtain the position of the suspicious bright area in the coordinate system of the part surface. Import the 3D model of the part from the CAD system, read the envelope size of the part, divide the surface coordinates of the suspicious bright areas in the image by the envelope size in the corresponding direction to obtain the normalized position, and merge the normalized positions of all frames and all suspicious bright areas in the detection period to generate a sequence of normalized positions of bright area surfaces.

[0009] In a preferred embodiment, the illumination analysis module reads the illumination incident direction corresponding to each frame of image, records the azimuth and elevation angles of the light source at the time of acquisition of each frame of image within the detection period, and converts the azimuth and elevation angles into an illumination incident direction vector. Arrange the incident light directions corresponding to the images within the detection period according to the image acquisition time sequence to form a historical light direction set. Import the three-dimensional model of the part from the CAD system, extract the surface geometric information, obtain the local normal distribution of the surface, discretize the surface of the part into several surface units, and calculate the normal vector of each surface unit. By combining the incident light direction and the local normal distribution of the curved surface, the theoretical surface position where the specular highlight should appear in each frame is calculated.

[0010] In a preferred embodiment, the illumination analysis module traverses all surface units on the curved surface of the part for the illumination incident direction corresponding to the image, and calculates the cosine value of the angle between the normal vector and the illumination incident direction for the normal vector of each surface unit. When the cosine value of the included angle is greater than the highlight threshold, it is determined that the surface unit should theoretically have a highlight in the image. The center coordinates of the surface unit are recorded as the theoretical highlight position. The theoretical highlight positions corresponding to the image are connected in time sequence to generate the theoretical highlight drift trajectory.

[0011] In a preferred embodiment, the defect screening module receives a normalized position sequence of bright area surfaces and a theoretical specular drift trajectory, and performs spatial matching. For each suspicious bright area in the normalized position sequence of the bright area surface, calculate the Chebyshev distance between its normalized position and each point on the theoretical specular drift trajectory, and take the minimum value of the Chebyshev distance in all frames as the spatial matching distance of the suspicious bright area.

[0012] In a preferred embodiment, the defect screening module detects the edge gradient sharpness of suspicious bright areas and obtains the edge images of suspicious bright areas in each frame image from the bright area recognition module. For the edge image of the suspicious bright area, the Sobel operator is applied to calculate the gradient in the x-direction and the gradient in the y-direction respectively, and the average value of the gradient magnitude of all pixels in the edge image is taken as the edge gradient sharpness of the suspicious bright area. By combining the normalized spatial matching distance and edge gradient sharpness, a logistic regression model is used to calculate the probability that the suspicious bright area is a real defect. Based on the actual defect probability, the confidence level of suspicious bright areas is graded, and high confidence thresholds and low confidence thresholds are set to formulate grading rules: Rule 1: When the probability of a true defect exceeds the high confidence threshold, it is determined to be a high confidence defect, and the suspicious bright area is added to the list of candidate true defects. Rule 2: When the actual defect probability is lower than the low confidence threshold, it is judged as a high confidence highlight and the suspicious bright area is removed; Rule 3: When the actual defect probability is greater than or equal to the low confidence threshold and less than or equal to the high confidence threshold, it is determined to be a bright area with insufficient confidence. The suspicious bright area is marked and sent to the supplementary lighting verification module for further verification.

[0013] In a preferred embodiment, the supplementary lighting verification module receives a set of insufficient confidence bright areas and performs supplementary lighting direction reverse calculation for each insufficient confidence bright area; For bright areas with insufficient confidence, read their position in the coordinate system of the part surface and query the local normal of the surface corresponding to that position; Based on the geometric relationship of specular occlusion, calculate the direction of supplementary illumination that will cause the specular highlight to disappear at this location.

[0014] In a preferred embodiment, the supplementary lighting verification module triggers supplementary lighting imaging, converts the supplementary lighting direction into the azimuth and pitch angles of the light source controller, and drives the light source controller to adjust the light source to that direction; Image acquisition is performed on the surface of the part under supplemental lighting conditions to obtain supplemental lighting images; In the supplementary lighting image, locate the pixel region of insufficient brightness, extract the gray values ​​of all pixels in the region, take the maximum gray value as the gray value peak of the bright area under supplementary lighting, and read the gray value peak of the bright area in the original image within the detection period.

[0015] In a preferred embodiment, the supplementary lighting verification module determines whether the bright area is removed with the supplementary lighting based on the change in grayscale peak value before and after supplementary lighting, performs loop closure correction on the real defect candidate list, calculates the grayscale peak value reduction rate of the insufficient confidence bright area, and sets the removal judgment threshold. Establish judgment rules: Rule 1: When the grayscale peak decrease rate is greater than the removal decision threshold, the grayscale of the bright area is determined to decrease significantly with the supplementary light, and the bright area is a normal highlight. The bright area is then removed from the list of real defect candidates (if the bright area was previously misjudged as a high-confidence defect and added to the list). Rule 2: When the grayscale peak decrease rate is less than or equal to the removal decision threshold, the grayscale of the bright area is determined not to decrease significantly with the supplementary light, the bright area is a real defect, and the bright area is added to the real defect candidate list. After completing the supplementary lighting verification by traversing all insufficient confidence bright areas, the supplementary lighting verification module outputs a corrected list of real defect candidates as the final detection result.

[0016] The technical effects and advantages of this invention are as follows: This invention acquires component inspection images using machine vision, detects grayscale anomalies using a bright area recognition module, and generates a bright area position sequence based on the component surface coordinate system. A lighting analysis module constructs a theoretical specular drift trajectory based on the incident light direction and the local normal distribution of the curved surface. The actual bright area positions are spatially matched with the theoretical specular trajectory. Candidate regions for real defects are screened based on the edge gradient sharpness of the bright areas. For bright areas with insufficient confidence, the lighting direction is supplemented by inferring the specular occlusion relationship. The change in bright area status is verified through supplementary lighting imaging, and the defect detection results are corrected. This improves the accuracy of surface defect detection on robot components and enhances the resistance to specular interference. It effectively distinguishes between real defects and light reflection areas during the inspection of complex curved surface components, reducing the false detection and false negative rates caused by lighting changes. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the implementation of a machine vision-based robot component quality inspection system according to the present invention.

[0018] Figure 2 This is a schematic diagram of a robot component quality inspection system based on machine vision according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention acquires component inspection images using machine vision, detects grayscale anomalies using a bright area recognition module, and generates a bright area position sequence based on the component surface coordinate system. A lighting analysis module constructs a theoretical specular drift trajectory based on the incident light direction and the local normal distribution of the curved surface. The actual bright area positions are spatially matched with the theoretical specular trajectory. The edge gradient sharpness of the bright area is used to filter candidate regions for true defects. For bright areas with insufficient confidence, the lighting direction is inferred from the specular occlusion relationship. The change in bright area status is verified through supplementary lighting imaging, and the defect detection results are corrected. This improves the accuracy of surface defect detection on robot components and enhances the resistance to specular interference.

[0021] Example 1

[0022] Please see Figures 1 to 2A robot parts quality inspection system based on machine vision includes a bright area recognition module, an illumination analysis module, a defect screening module, and a correction judgment module, with electrical signal connections between the modules; The functions of each module are as follows: The bright area recognition module is used to detect gray-scale abnormal areas in each frame of the image and identify suspicious bright areas. It collects the image coordinates of the reference feature points on the surface of the part, performs inter-frame geometric registration to map the suspicious bright areas to the coordinate system of the part surface, generates a normalized position sequence of the bright area surface, and inputs the normalized position sequence of the bright area surface into the defect screening module. The illumination analysis module is used to read the incident direction of illumination in each frame, obtain the local normal distribution of the surface, calculate the surface position where the theoretical specular highlight should appear in each frame in combination with the incident direction of illumination in each frame, and generate the theoretical specular highlight drift trajectory. The theoretical specular highlight drift trajectory is then transmitted to the defect screening module. The defect screening module performs spatial matching between the normalized position sequence of the bright area surface and the theoretical specular drift trajectory, detects the edge gradient sharpness of suspicious bright areas, and after screening the real defect candidate list by combining the spatial matching results and edge gradient sharpness, marks the insufficient confidence bright areas with contradictory trajectory evidence and passes the insufficient confidence bright areas to the correction judgment module. The correction and judgment module reverses the direction of supplementary lighting for bright areas with insufficient confidence based on the geometric relationship of specular occlusion and triggers supplementary lighting imaging. It collects the gray value peak of the bright area under supplementary lighting imaging, and determines whether the bright area is removed with supplementary lighting based on the gray value peak of the bright area and performs loop correction on the real defect candidate list.

[0023] The bright area recognition module receives images from the industrial camera during the detection cycle. The detection cycle is set to T seconds. The industrial camera captures images of the part surface at 0.1-second intervals and acquires N frames of images during the detection cycle. The N frames of images are then merged into an image sequence. The industrial camera is a CCD area array camera with a resolution of 2048×2048 pixels and an exposure time of 5 milliseconds. Simultaneously, a 3D scanner scans the part surface before inspection, acquiring feature point cloud data of the part surface. Several reference feature points are extracted from the point cloud data, and the 3D coordinates of these reference feature points in the part surface coordinate system are obtained. .

[0024] The reference feature points are locations with obvious geometric features such as corners, edge intersections, or hole centers on the surface of the part, and the number shall not be less than four.

[0025] It should be noted that the reference feature points in this embodiment are not randomly selected, but obtained through point cloud geometric feature analysis. Specifically, neighborhood curvature calculation is performed on the point cloud data of the part surface. Based on the degree of local curvature abrupt change, regions with obvious geometric features such as corner points, edge intersections, and hole structure boundary points are identified. The point with the largest curvature response is selected from the candidate regions as the reference feature point. The number of reference feature points is not less than four, which are used to establish the spatial correspondence between the image coordinate system and the part surface coordinate system.

[0026] The local curvature is calculated using the neighborhood covariance matrix of the point cloud. Points with larger curvature responses indicate more significant surface geometric changes and are preferentially used as corner points, edge intersections, or pore structure boundary points.

[0027] The bright area recognition module performs grayscale anomaly detection on each frame of the image sequence, and performs grayscale anomaly detection on each pixel of the i-th frame. Read its grayscale value Calculate the average gray value of all pixels within the w×w neighborhood of the given pixel as the local background average gray value. ; pixel grayscale values Compared with the local background grayscale average The difference is calculated by taking the absolute value, and the grayscale deviation is obtained. . If the grayscale deviation Greater than the grayscale anomaly threshold If so, the pixel is marked as an abnormal pixel.

[0028] Perform connected component analysis on all locations marked as anomalous pixels in the i-th frame image, and cluster spatially connected anomalous pixels into a suspicious bright area; Images of defect-free standard parts are acquired, and the 95th percentile value of the grayscale deviation across the entire image is used as the grayscale anomaly threshold. For example, setting the neighborhood size w to 5 and the grayscale anomaly threshold... Set it to 30.

[0029] This method targets abrupt changes in grayscale caused by scratches, dents, cracks, and localized surface anomalies. Since these defects alter local reflectivity, creating grayscale differences between the defective and surrounding areas, they can all be used to identify candidate bright areas through grayscale deviation detection.

[0030] Furthermore, to ensure that local grayscale deviations reflect the grayscale changes between the defective region and its neighboring normal regions, while avoiding unstable grayscale variation representation due to an excessively small neighborhood or grayscale differences in non-defective regions due to an excessively large neighborhood, the grayscale deviation distribution under different neighborhood sizes is statistically analyzed during the standard part calibration process. The neighborhood size is then determined based on the local stability of the grayscale deviation and the distinguishability of the defective region. In this embodiment, based on the standard part calibration results, the neighborhood size N is preferably set to 5.

[0031] Furthermore, to avoid misjudging normal grayscale fluctuations under different part materials and lighting conditions as abnormal due to a fixed grayscale anomaly threshold, this method acquires images of defect-free standard parts, statistically analyzes the grayscale deviation distribution in the normal part area, and determines the 95th percentile value of the grayscale deviation distribution as the grayscale anomaly threshold. This allows the grayscale anomaly threshold to be adaptively determined based on the grayscale fluctuation range under normal conditions. In this embodiment, under the current part material and lighting conditions, the grayscale anomaly threshold obtained based on the 95th percentile statistical result is 30.

[0032] Therefore, the neighborhood size is used to determine the statistical range of local gray-level deviation, and the gray-level anomaly threshold is used to distinguish between normal gray-level fluctuations and abnormal gray-level changes. Together, they are used to form gray-level anomaly candidate regions for scratches, dents, cracks, and local surface anomalies.

[0033] It should be explained that the connected component analysis uses an 8-connectivity approach, meaning that if two outlier pixels are adjacent in a horizontal, vertical, or diagonal direction, they are considered connected. The local background grayscale mean... It is used to eliminate the influence of the inherent texture of the part surface and the gradual change of light, so that the detection focuses on the area of ​​gray-scale change.

[0034] The bright area recognition module extracts the image coordinates of the reference feature points in each frame and performs inter-frame geometric registration; in the i-th frame, the corner detection algorithm locates the coordinates of the reference feature points on the image plane. , where p is the reference feature point number.

[0035] The coordinates of the reference feature points in the i-th frame image Three-dimensional coordinates of reference feature points in the coordinate system of the part surface Establish the correspondence and use the RANSAC algorithm to solve the homography matrix. Homography matrix Given a 3×3 matrix, the pixel coordinates (u, v, 1) in the i-th frame image are transformed into the surface coordinates (x, y, z) of the part. The transformation relationship is as follows: For all suspicious bright areas identified in the i-th frame image, substitute their center pixel coordinates into the homography matrix. The position of the suspicious bright area in the coordinate system of the part surface is obtained.

[0036] It should be explained that the RANSAC algorithm improves the accuracy of homography matrix solution by iteratively sampling reference feature point pairs and eliminating outliers with large matching errors. The corner detection algorithm uses the Harris algorithm or the Shi-Tomasi algorithm, which can locate feature points with sub-pixel accuracy.

[0037] The bright area identification module calculates the normalized position of suspicious bright areas, imports the 3D model of the part from the CAD system, and reads the envelope dimensions of the part in the x, y, and z directions, which are denoted as follows: The surface coordinates of the m-th suspected bright area in the i-th frame image. Divide each position by the envelope size in the corresponding direction to obtain the normalized position. The calculation formula is: .

[0038] The normalized positions of all frames and all suspicious bright areas within the detection period are merged to generate a normalized position sequence of the bright area surface. Where M is the total number of suspicious bright areas, and m is the number of the suspicious bright area. The normalized position ranges from [0, 1]; the bright area identification module inputs the normalized position sequence of the bright area surface into the defect screening module.

[0039] It should be noted that the suspicious bright areas identified in all frames of images within the detection period are numbered m in order of appearance, where m = 1, 2, ..., M, and M is the total number of suspicious bright areas within the detection period. Each suspicious bright area corresponds to a normalized coordinate of the part surface, and the coordinates of all suspicious bright areas are combined to form a normalized position sequence of the bright area surface.

[0040] The illumination analysis module reads the incident light direction corresponding to each frame of the image, and the light source controller records the azimuth angle of the light source at the time of acquisition of each frame of the image within the detection cycle. and pitch angle azimuth angle and pitch angle Convert to incident light vector The conversion formula is: Light incident direction vector It is a unit vector pointing in the direction from the light source to the surface of the part; Arrange the incident light directions corresponding to N frames of images within the detection period according to the image acquisition time sequence to form a historical set of light directions. ,in, Let N represent the incident light direction vector at the moment the i-th frame of the image is acquired, and let N represent the total number of images acquired within the detection period. This represents the set of all lighting directions used within the detection period.

[0041] The lighting analysis module imports the 3D model of the part from the CAD system, extracts the surface geometry information, obtains the local normal distribution of the surface, discretizes the surface of the part into several surface elements, and calculates the normal vector for each surface element. If the surface is represented by the parametric equation P(u, v), then the formula for calculating the normal vector is: Where × represents the cross product operation. and Let be the partial derivative vectors of the surface along the u and v parameter directions, respectively. Let n be the magnitude of the cross product result. The normal vector n is a unit vector after normalization and points to the outside of the surface.

[0042] It should be noted that the size of the surface unit is set to 1% of the part envelope size to ensure that the normal distribution can capture surface details without causing excessive computation due to excessive discretization.

[0043] The illumination analysis module combines the incident direction of illumination and the local normal distribution of the surface to calculate the surface location where the theoretical specular highlights should appear in each frame. For the incident light direction corresponding to the i-th frame image Iterate through all surface elements on the part's surface, and for each surface element's normal vector... Calculate the normal vector and the direction of incident light cosine value of the included angle .

[0044] when Greater than the specular threshold When determining that the surface unit should theoretically have a highlight in the i-th frame image, the center coordinates of the surface unit are recorded as the theoretical highlight position in the i-th frame. .

[0045] High light threshold The specular reflection coefficient is set based on the surface material of the part. For metal materials, in this example... Set it to between 0.8 and 0.95.

[0046] Connect the theoretical highlight positions corresponding to N frames of images in chronological order to generate the theoretical highlight drift trajectory. The illumination analysis module transmits the theoretical specular drift trajectory to the defect screening module.

[0047] It needs to be explained that the cosine value of the included angle The larger the value, the closer the normal vector is to the opposite direction of the illumination, resulting in stronger specular reflection and more pronounced highlights. The negative sign in the formula reflects the direction of incident light. The direction pointing from the light source to the surface is opposite to the definition of the incident light direction in the law of reflection.

[0048] The defect screening module receives the normalized position sequence of the bright area surface. And theoretical specular drift trajectory Perform space matching.

[0049] For each suspicious bright region m in the normalized position sequence of bright region surfaces, calculate its normalized position. With each point on the theoretical specular drift trajectory Chebyshev distance between .

[0050] Take the minimum Chebyshev distance across all frames as the spatial matching distance for the suspected bright area. .

[0051] Spatial matching distance The smaller the value, the closer the suspected bright area is to the theoretical highlight trajectory, and the more likely the bright area is a normal highlight; spatial matching distance The larger the value, the more likely the suspicious bright area deviates from the theoretical highlight trajectory, and the more likely the bright area is a real defect.

[0052] The defect screening module detects the edge gradient sharpness of suspicious bright areas and obtains the edge images of suspicious bright areas in each frame image from the bright area recognition module. For the edge image of the m-th suspicious bright region, the Sobel operator is applied to calculate the gradient in the x-direction. gradient in the y-direction The Sobel operator has a 3×3 convolution kernel, with the kernel in the x-direction being... The core in the y-direction is Calculate the gradient magnitude. The average gradient magnitude of all pixels in the edge image is taken as the edge gradient sharpness of the suspected bright area.

[0053] It should be explained that real defects such as scratches and dents have a large difference in grayscale between the edges and the background, and the edge gradient sharpness is high; normal highlights have blurred edges, smooth grayscale transitions, and low edge gradient sharpness.

[0054] Before inputting the spatial matching distance and edge gradient sharpness into the logistic regression model, the defect screening module normalizes both the spatial matching distance and edge gradient sharpness to obtain the normalized spatial matching distance. and edge gradient sharpness This method converts data with different physical dimensions into dimensionless feature values. Specifically, it uses a max-min normalization method to map the spatial matching distance and edge gradient sharpness to preset ranges, so that the normalized spatial matching distance and edge gradient sharpness no longer have physical dimensions, thereby ensuring that the input parameters of the logistic regression model's exponential calculation part meet the dimensionless requirement.

[0055] Spatial matching distance after comprehensive normalization processing by the defect screening module and edge gradient sharpness The probability that the suspicious bright area m is a real defect is calculated using a logistic regression model. The formula for the logistic regression model is: ,in For the intercept term, The regression coefficients for spatial matching distance are denoted as . The regression coefficient for edge gradient sharpness is . The normalized spatial matching distance. This represents the edge gradient sharpness after normalization.

[0056] Among them, the intercept term is a dimensionless parameter, and the regression coefficients corresponding to the spatial matching distance and the edge gradient sharpness are used to adjust the influence of the corresponding dimensionless features on the defect probability, so that the exponential part of the logistic regression model is dimensionless as a whole.

[0057] Logistic regression coefficient The training was conducted using historically calibrated samples: 50 samples each of known defects and normal highlights were collected, and their spatial matching distance and edge gradient sharpness were extracted as features. The maximum likelihood estimation method was then used to solve for the logistic regression coefficients. For example, the logistic regression coefficients obtained during training were... .

[0058] It needs to be explained that spatial matching distance The larger the value, the sharper the edge gradient. The larger the value, the higher the probability of a true defect. The higher the value, the more likely the suspicious bright area is to exhibit both abnormal location and sharp edges, indicating a defect.

[0059] The defect screening module is based on the actual defect probability. Confidence levels are assigned to suspicious bright areas.

[0060] Set a high confidence threshold and low confidence threshold ,in Establish grading rules: Rule 1: When If the area is identified as a high-confidence defect, the suspicious bright area is added to the list of candidate real defects. Rule 2: When If the area is identified as a high-confidence highlight, the suspicious bright area is removed. Rule 3: When If the area is deemed a bright area with insufficient confidence, it is marked as a suspicious bright area and sent to the supplementary lighting verification module for further verification.

[0061] Plot the distribution curve of the true defect probability on the calibration sample set, and take the two critical points that maximize the classification accuracy as the threshold; for example, the high confidence threshold. Setting it to 0.75 will lower the low confidence threshold. Set to 0.25; the defect screening module will pass the set of bright areas with insufficient confidence to the supplementary lighting verification module.

[0062] The supplementary lighting verification module receives a set of insufficient confidence bright areas and performs supplementary lighting direction inverse calculation for each insufficient confidence bright area; For the k-th bright area with insufficient confidence, read its position in the coordinate system of the part surface. Query the local normal of the surface at this location. .

[0063] Based on the geometric relationship of specular occlusion, calculate the direction of supplementary illumination that would cause the specular highlight to disappear at that location. Supplement the direction of illumination. The normal vector must satisfy and supplemental lighting direction The included angle is greater than 90 degrees, that is This prevents the light reflected from the mirror from entering the camera's field of view; Meanwhile, to avoid duplication with the illumination direction already used within the detection cycle, a supplementary illumination direction is calculated. Set with historical lighting direction Each direction of illumination The included angle ; It should be added that verification is required during the selection process. Check whether the corresponding azimuth and elevation angles of the light source are within the mechanical limit range of the light source. If they exceed the limit, select a direction that meets the limit conditions from the suboptimal directions. If all directions that meet the extinction conditions exceed the limit, abandon the supplementary lighting verification, directly determine the bright area as a real defect and record the anomaly.

[0064] The supplementary lighting verification module triggers supplementary lighting imaging, which will adjust the direction of the supplementary lighting. Converted to azimuth angle of the light source controller and pitch angle The driver adjusts the light source to that direction using the light source controller; An industrial camera captures images of a part surface under supplemental lighting conditions, obtaining supplemental lighting images. The pixel region of the k-th low-confidence bright area is located within the supplemental lighting image. Extract the grayscale values ​​of all pixels within the region, and take the maximum grayscale value as the peak grayscale value of the bright area under supplementary lighting. The calculation formula is: ,in To fill in the pixels in the image The grayscale value of the bright area is recorded. Simultaneously, the peak grayscale value of the bright area in the original image within the detection period is read. .

[0065] The supplementary lighting verification module determines whether the bright area is removed with the supplementary lighting based on the change in grayscale peak value before and after supplementary lighting, and performs loop-back correction on the real defect candidate list; Calculate the grayscale peak reduction rate of the kth low-confidence bright region. Set the removal threshold. .

[0066] Establish judgment rules: Rule 1: When If the grayscale of the bright area decreases significantly with the supplementary light, the bright area is considered a normal highlight and is removed from the list of real defect candidates (if the bright area was previously mistakenly identified as a high-confidence defect and added to the list). Rule 2: When If the grayscale of the bright area does not decrease significantly with the supplementary light, the bright area is considered a real defect and is added to the list of candidate real defects.

[0067] For known normal highlight samples, supplemental lighting was used for verification, and the median value of the grayscale peak reduction rate was used as the statistical value. For example, removing the decision threshold. Set it to 0.6.

[0068] After completing the supplementary lighting verification by traversing all insufficient confidence bright areas, the supplementary lighting verification module outputs a corrected list of real defect candidates as the final detection result.

[0069] Grayscale peak descent rate The larger the value, the more sensitive the grayscale of the bright area is to changes in the direction of illumination, which is consistent with specular reflection characteristics and is a normal highlight; grayscale peak decrease rate The smaller the value, the more likely the grayscale of the bright area is not affected by the direction of illumination, which is consistent with diffuse reflection or inherent defect characteristics, and is a real defect.

[0070] The loop correction mechanism uses the feedback results from supplementary light verification to correct misjudgments in the first-stage screening, thereby improving the final detection accuracy.

[0071] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0072] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0073] In this document, the singular forms “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that terms such as “comprising / including” or “having” specify the presence of the stated features, integrals, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integrals, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0074] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0075] The above description of the disclosed embodiments will enable those skilled in the art to make or use various modifications to these embodiments. It will be readily apparent to those skilled in the art that the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A machine vision-based quality inspection system for robot parts, characterized in that: It includes a bright area recognition module, an illumination analysis module, a defect screening module, and a correction judgment module, with electrical signal connections between the modules; The bright area recognition module is used to detect gray-scale abnormal areas in each frame of the image and identify suspicious bright areas. It collects the image coordinates of the reference feature points on the surface of the part, performs inter-frame geometric registration to map the suspicious bright areas to the coordinate system of the part surface, generates a normalized position sequence of the bright area surface, and inputs the normalized position sequence of the bright area surface into the defect screening module. The illumination analysis module is used to read the incident direction of illumination in each frame, obtain the local normal distribution of the surface, calculate the surface position where the theoretical specular highlight should appear in each frame in combination with the incident direction of illumination in each frame, and generate the theoretical specular highlight drift trajectory. The theoretical specular highlight drift trajectory is then transmitted to the defect screening module. The defect screening module performs spatial matching between the normalized position sequence of the bright area surface and the theoretical specular drift trajectory, detects the edge gradient sharpness of suspicious bright areas, and after screening the real defect candidate list by combining the spatial matching results and edge gradient sharpness, marks the insufficient confidence bright areas with contradictory trajectory evidence and passes the insufficient confidence bright areas to the correction judgment module. The correction and judgment module reverses the direction of supplementary lighting for bright areas with insufficient confidence based on the geometric relationship of specular occlusion and triggers supplementary lighting imaging. It collects the gray value peak of the bright area under supplementary lighting imaging, and determines whether the bright area is removed with supplementary lighting based on the gray value peak of the bright area and performs loop correction on the real defect candidate list.

2. The robot component quality inspection system based on machine vision according to claim 1, characterized in that: The bright area recognition module receives images from the industrial camera during the detection cycle. Collect feature point cloud data of the part surface, extract several reference feature points from the point cloud data, and obtain the three-dimensional coordinates of the reference feature points in the coordinate system of the part surface. Perform grayscale anomaly detection on each frame of the image sequence. For each pixel in the image, read its grayscale value and calculate the average grayscale value of all pixels in the neighborhood of that pixel as the local background grayscale average value. The grayscale deviation is obtained by subtracting the grayscale value of a pixel from the mean grayscale value of the local background and taking the absolute value. If the grayscale deviation is greater than the grayscale abnormality threshold, the pixel is marked as an abnormal pixel.

3. The robot component quality inspection system based on machine vision according to claim 2, characterized in that: The bright area identification module performs connected component analysis on all locations marked as abnormal pixels in the image, and clusters spatially connected abnormal pixels into a suspicious bright area; Extract the image coordinates of the reference feature points in each frame, perform inter-frame geometric registration, and locate the coordinates of the reference feature points on the image plane using a corner detection algorithm; Establish a correspondence between the image coordinates of the reference feature points in the image and the three-dimensional coordinates of the reference feature points in the coordinate system of the part surface. Use the RANSAC algorithm to solve the homography matrix. For all suspicious bright areas identified in the image, substitute the center pixel coordinates into the homography matrix to obtain the position of the suspicious bright area in the coordinate system of the part surface. Import the 3D model of the part from the CAD system, read the envelope size of the part, divide the surface coordinates of the suspicious bright areas in the image by the envelope size in the corresponding direction to obtain the normalized position, and merge the normalized positions of all frames and all suspicious bright areas in the detection period to generate a sequence of normalized positions of bright area surfaces.

4. The robot component quality inspection system based on machine vision according to claim 1, characterized in that: The illumination analysis module reads the illumination incident direction corresponding to each frame of the image, records the azimuth and elevation angles of the light source at the time of acquisition of each frame of the image within the detection period, and converts the azimuth and elevation angles into an illumination incident direction vector. Arrange the incident light directions corresponding to the images within the detection period according to the image acquisition time sequence to form a historical light direction set. Import the three-dimensional model of the part from the CAD system, extract the surface geometric information, obtain the local normal distribution of the surface, discretize the surface of the part into several surface units, and calculate the normal vector of each surface unit. By combining the incident light direction and the local normal distribution of the curved surface, the theoretical surface position where the specular highlight should appear in each frame is calculated.

5. The robot component quality inspection system based on machine vision according to claim 4, characterized in that: The lighting analysis module iterates through all surface units on the curved surface of the part for the incident lighting direction corresponding to the image, and calculates the cosine of the angle between the normal vector and the incident lighting direction for the normal vector of each surface unit. When the cosine value of the included angle is greater than the highlight threshold, it is determined that the surface unit should theoretically have a highlight in the image. The center coordinates of the surface unit are recorded as the theoretical highlight position. The theoretical highlight positions corresponding to the image are connected in time sequence to generate the theoretical highlight drift trajectory.

6. The robot component quality inspection system based on machine vision according to claim 5, characterized in that: The defect screening module receives the normalized position sequence of the bright area surface and the theoretical specular drift trajectory, and performs spatial matching. For each suspicious bright area in the normalized position sequence of the bright area surface, calculate the Chebyshev distance between its normalized position and each point on the theoretical specular drift trajectory, and take the minimum value of the Chebyshev distance in all frames as the spatial matching distance of the suspicious bright area.

7. The robot component quality inspection system based on machine vision according to claim 6, characterized in that: The defect screening module detects the edge gradient sharpness of suspicious bright areas and obtains the edge images of suspicious bright areas in each frame image from the bright area recognition module. For the edge image of the suspicious bright area, the Sobel operator is applied to calculate the gradient in the x-direction and the gradient in the y-direction respectively, and the average value of the gradient magnitude of all pixels in the edge image is taken as the edge gradient sharpness of the suspicious bright area. By combining the normalized spatial matching distance and edge gradient sharpness, a logistic regression model is used to calculate the probability that the suspicious bright area is a real defect. Based on the actual defect probability, the confidence level of suspicious bright areas is graded, and high confidence thresholds and low confidence thresholds are set to formulate grading rules: Rule 1: When the probability of a true defect exceeds the high confidence threshold, it is determined to be a high confidence defect, and the suspicious bright area is added to the list of candidate true defects. Rule 2: When the actual defect probability is lower than the low confidence threshold, it is judged as a high confidence highlight and the suspicious bright area is removed; Rule 3: When the actual defect probability is greater than or equal to the low confidence threshold and less than or equal to the high confidence threshold, it is determined to be a bright area with insufficient confidence. The suspicious bright area is marked and sent to the supplementary lighting verification module for further verification.

8. The robot component quality inspection system based on machine vision according to claim 1, characterized in that: The supplementary lighting verification module receives a set of insufficient confidence bright areas and performs supplementary lighting direction inverse calculation for each insufficient confidence bright area; For bright areas with insufficient confidence, read their position in the coordinate system of the part surface and query the local normal of the surface corresponding to that position; Based on the geometric relationship of specular occlusion, calculate the direction of supplementary illumination that will cause the specular highlight to disappear at this location.

9. A robot component quality inspection system based on machine vision according to claim 8, characterized in that: The supplementary lighting verification module triggers supplementary lighting imaging, converts the direction of supplementary lighting into the azimuth and elevation angles of the light source controller, and drives the light source controller to adjust the light source to that direction; Image acquisition is performed on the surface of the part under supplemental lighting conditions to obtain supplemental lighting images; In the supplementary lighting image, locate the pixel region of insufficient brightness, extract the gray values ​​of all pixels in the region, take the maximum gray value as the gray value peak of the bright area under supplementary lighting, and read the gray value peak of the bright area in the original image within the detection period.

10. A machine vision-based robot component quality inspection system according to claim 9, characterized in that: The supplementary lighting verification module determines whether the bright area is removed with supplementary lighting based on the change in grayscale peak value before and after supplementary lighting, performs loop correction on the real defect candidate list, calculates the grayscale peak value reduction rate of the insufficient confidence bright area, and sets the removal judgment threshold. Establish judgment rules: Rule 1: When the grayscale peak decrease rate is greater than the removal decision threshold, the grayscale of the bright area is determined to decrease significantly with the supplementary light, and the bright area is a normal highlight. The bright area is then removed from the list of real defect candidates (if the bright area was previously misjudged as a high-confidence defect and added to the list). Rule 2: When the grayscale peak decrease rate is less than or equal to the removal decision threshold, the grayscale of the bright area is determined not to decrease significantly with the supplementary light, the bright area is a real defect, and the bright area is added to the real defect candidate list. After completing the supplementary lighting verification by traversing all insufficient confidence bright areas, the supplementary lighting verification module outputs a corrected list of real defect candidates as the final detection result.

Images