A surface finish detection system and method based on scale-invariant feature matching
By using multi-view imaging and 3D reconstruction technology, combined with surface curvature optimization lighting correction and feature matching, the problem of blind spots in the detection of irregularly shaped workpieces was solved, and high-precision surface finish detection of the entire workpiece surface was achieved.
Patent Information
- Application Number
- CN202611278534.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-21
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies are insufficient for precise surface inspection of complex spatial structures of irregularly shaped workpieces, especially for continuous curved surface corners, dense bolt holes, and assembly gaps. There are blind spots in the inspection, and there is a lack of a refined comparison and evaluation system, making it impossible to accurately identify subtle surface problems.
By employing multi-view imaging and 3D reconstruction, combined with surface curvature optimization lighting correction and feature matching, a curvature-guided non-uniform multi-scale space is constructed through a texture matching module to perform cross-view feature matching and texture evaluation, generate a 3D surface model, and perform surface finish detection.
It achieves high-precision surface finish inspection of the entire workpiece surface, eliminates blind spots in inspection, accurately identifies microscopic surface finish anomalies, and provides full-surface coverage and high-precision quantitative inspection results.
Smart Images

Figure CN123071284A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a surface finish detection system and method based on scale-invariant feature matching, belonging to the field of intelligent detection technology. Background Technology
[0002] In industrial inspection scenarios, various metal workpieces play a crucial role in mechanical fixation, electrical conduction, and electric field shielding of equipment. The surface finish of the workpiece directly affects the operational stability, corrosion resistance, and overall service life of the components. Workpiece surface quality inspection is an important link in ensuring the reliable operation of complete sets of equipment. With the development of machine vision and multispectral imaging technology, non-contact optical inspection has become the mainstream direction for surface quality inspection of power equipment, effectively improving the level of inspection automation.
[0003] A Chinese patent with publication number CN120852725A discloses a method and device for identifying defects in a multispectral ultra-high voltage valve hall, comprising: acquiring multispectral image data of devices in an ultra-high voltage valve hall and performing preprocessing; constructing a multispectral feature extraction network; designing a cross-modal feature fusion network; inputting the fused features into an improved YOLOv11 network; forming a defect identification model; and training the defect identification model using a three-stage progressive framework.
[0004] Although existing technologies have improved the identification performance of macroscopic defects in valve hall equipment through multimodal data fusion, their design for macroscopic defect detection of the entire valve hall device is difficult to adapt to the fine surface inspection requirements of complex spatial structures of various irregular workpieces. Specifically, the fixed-view imaging acquisition mode is used only, without establishing a feature matching and three-dimensional spatial stitching mechanism for multi-view images. When facing continuous curved corners, dense bolt holes, and assembly gaps of workpieces, blind spots are easily formed on the back surface of components, the inner walls of holes, and structurally obscured areas, making it impossible to achieve complete inspection coverage of the entire workpiece surface. At the same time, focusing only on the classification and location of macroscopic defects, there is a lack of a fine comparison and evaluation system based on standard benchmark models, which makes it impossible to accurately identify and quantify minor surface problems such as deterioration of workpiece surface finish and contaminant adhesion, making it difficult to meet the requirements of full-area coverage and high-precision surface quality inspection of workpieces. Summary of the Invention
[0005] To address the shortcomings of existing technologies, the present invention aims to provide a surface finish inspection system and method based on scale-invariant feature matching. By eliminating the detection blind spots caused by structural occlusion through multi-view imaging and three-dimensional reconstruction, and by combining surface curvature optimization illumination correction, feature matching, and texture evaluation benchmarks, the system achieves high-precision and refined surface finish inspection of the entire surface of the target workpiece.
[0006] To achieve the above objectives, the present invention provides the following technical solution:
[0007] A surface finish detection system based on scale-invariant feature matching includes: an image acquisition module, a texture matching module, and an anomaly detection module;
[0008] The image acquisition module is used to acquire multi-view images of the workpiece. By using the chromaticity and saturation of the highlight pixels on the workpiece surface, the highlight attribute of each pixel in each view image is determined, the highlight reflection area is identified and neighborhood texture interpolation is performed, and the gamma correction coefficient is adaptively adjusted according to the curvature distribution of the image edge to obtain a multi-view workpiece image set.
[0009] The texture matching module is used to construct a curvature-guided non-uniform multi-scale space, and generate a cross-view matching point pair set through cross-view feature matching and curvature attribute label-driven anomaly matching screening. It generates an initial mesh model through triangulation space back projection and surface mesh reconstruction, and extracts texture features from the multi-view workpiece image set to map them into the initial mesh model to generate the workpiece model to be tested.
[0010] The anomaly detection module is used to acquire a standard workpiece model, combine the curvature normalization correction of the smooth texture attribute with the surface curvature, calculate the feature difference between the workpiece model to be tested and the standard workpiece model vertex by vertex, thereby determining the smooth abnormal area and marking its spatial position.
[0011] Specifically, the texture matching module includes a feature extraction unit and a texture reconstruction unit;
[0012] The feature extraction unit adaptively sets the scale sampling interval based on the edge curvature of the workpiece image to construct a curvature-guided non-uniform multi-scale space. Through the consistency verification of feature extraction and cross-scale curvature, a workpiece feature set with curvature attribute labels is constructed. Cross-view feature matching is performed on the feature subsets of each view in the workpiece feature set to obtain initial matching point pairs. Anomaly matching is filtered and eliminated by combining the curvature attribute labels to generate a cross-view matching point pair set.
[0013] The texture reconstruction unit is used to obtain the camera intrinsic parameter matrix, combine the cross-view matching point pair set, solve the camera pose transformation matrix between each view, generate an initial mesh model through triangulation space back projection and surface mesh reconstruction, extract the texture features of each workpiece image from the multi-view workpiece image set through the gray-level co-occurrence matrix, and map the texture features to the corresponding vertices of the initial mesh model using the camera pose transformation matrix as the mapping relationship to generate the workpiece model to be tested.
[0014] Specifically, the steps for identifying specular reflection regions and performing neighborhood texture interpolation include:
[0015] The image is converted to a color space to obtain a converted workpiece image, and the decision components are extracted, including the chroma component, saturation component and luminance component.
[0016] By combining chroma, saturation, and local brightness, pixel-by-pixel attribute determination is performed on the converted workpiece image to filter out highlight pixels. Then, the eight-neighbor connected component analysis method is used to perform region clustering to obtain the highlight reflection region.
[0017] Edge detection is performed on the converted workpiece image to generate a set of structural edges;
[0018] The image is divided into multiple structural regions using the closed contours formed by edge pixels as segmentation boundaries;
[0019] For the pixels to be repaired in the highlight reflection area, select pixels that are neither highlight nor edge and belong to the same structural region to form an effective interpolation sample set;
[0020] By using distance-brightness dual-weighted interpolation, the target brightness of the pixel to be repaired is calculated and replaced to obtain the brightness image of the workpiece.
[0021] Specifically, obtaining a multi-view workpiece image set includes:
[0022] A local fitting neighborhood is constructed for the edge pixels, and a quadratic curve is fitted. The local curvature of the edge pixels is calculated using the second derivative of the fitted curve.
[0023] For non-edge regions, bilinear interpolation is performed using the local curvature and distance weights of edge pixels within the neighborhood window.
[0024] The maximum local curvature value in the brightness image of the workpiece is selected to normalize the local curvature of all pixels and generate a curvature distribution map of the brightness image of the workpiece.
[0025] Set a baseline gamma coefficient, and calculate the gamma adjustment coefficient of the corresponding pixel based on the normalized curvature and curvature adjustment coefficient to obtain the adaptive gamma coefficient.
[0026] Substitute the adaptive gamma coefficients corresponding to each pixel into the gamma transform to perform grayscale correction on the brightness image of the workpiece pixel by pixel.
[0027] Highlight suppression and adaptive gamma correction are performed on the brightness images of the workpiece from all acquisition perspectives. Combined with the corresponding perspective labels, a multi-view workpiece image set is generated.
[0028] Specifically, the steps for constructing a curvature-guided non-uniform multi-scale space include:
[0029] Based on the SIFT Gaussian pyramid, a set of reference parameters for multi-scale space is set.
[0030] Traverse the curvature distribution map and map the normalized curvature to the density adjustment factor of the corresponding pixel;
[0031] Based on the reference sampling interval in the reference parameter set, and combined with the density adjustment factor, the scale sampling interval for each pixel is calculated;
[0032] By using the first difference of the curvature distribution, the curvature gradient of each pixel is calculated, and pixels with curvature gradients greater than the mutation threshold are identified as mutation pixels, thereby constructing gradient mutation boundaries.
[0033] Based on the gradient abrupt boundary, the transition zone is expanded to both sides of the boundary to form a transition zone, with the boundary pixels on both sides of the transition zone as the endpoints;
[0034] For pixels within the transition zone, the distance ratio from the pixel to both endpoints is used as the endpoint interval weight, and linear interpolation is performed by weighted summation.
[0035] Based on the scale sampling interval of each pixel in the workpiece image, a non-uniform multi-scale space is constructed using Gaussian blur.
[0036] Specifically, the steps for constructing a workpiece feature set with curvature attribute labels include:
[0037] In the non-uniform multi-scale space, initial feature points are extracted through multi-scale neighborhood extremum detection;
[0038] By scaling the scale, the pixel coordinates of the initial feature points at each scale layer are mapped to the original image coordinate system, and the local curvature of the pixel coordinates of the initial feature points at the current scale layer and the adjacent scale layers above and below is extracted.
[0039] Calculate the curvature difference between adjacent scale layers and retain only feature points whose curvature difference is lower than the curvature consistency threshold;
[0040] Based on the descriptor generation rules of the SIFT algorithm, feature points are assigned principal directions, and feature descriptors of corresponding dimensions are generated.
[0041] The local curvature corresponding to each feature point is used as a curvature attribute label and bound to the feature descriptor, the pixel coordinates of the original image, and the scale parameter to which the feature point belongs, to form a single feature point data.
[0042] Based on the viewpoint identifier, all feature point data under the same viewpoint are integrated into a feature subset to generate a workpiece feature set.
[0043] Specifically, the steps for generating a cross-view matching point pair set include:
[0044] Based on the Euclidean distance between the camera position of a single viewpoint and the geometric center point of all viewpoints, a reference viewpoint and a viewpoint to be matched are divided, and viewpoint task pairs are constructed.
[0045] Each viewpoint task pair is traversed sequentially, and a global search is performed in the baseline viewpoint using the Euclidean distance between feature descriptors to obtain the nearest neighbor candidate point and the second nearest neighbor candidate point.
[0046] Calculate the ratio of the nearest neighbor distance to the second nearest neighbor distance, and use it as the nearest neighbor distance ratio;
[0047] When the nearest neighbor distance ratio is less than the initial matching threshold, it is determined that the feature point in the view to be matched and the nearest neighbor candidate point constitute an initial matching point pair.
[0048] If the nearest neighbor distance ratio is not less than the initial matching threshold, then the feature point is determined to be unmatched.
[0049] Specifically, the steps for generating a cross-view matching point pair set also include:
[0050] For each initial pair of matching points, extract the curvature attribute labels corresponding to the two feature points and calculate the relative curvature deviation;
[0051] Set the curvature matching threshold and perform curvature filtering on the initial matching point pairs;
[0052] If the relative curvature deviation is less than the curvature matching threshold, the curvature screening is deemed successful; otherwise, the current initial matching point pair is discarded.
[0053] For the matching point pairs filtered by curvature, the Euclidean distance and relative curvature deviation between feature descriptors are called, and normalized and weighted sum of the reciprocals are performed to obtain the matching confidence.
[0054] All matching point pairs filtered by curvature are aggregated, categorized and stored according to viewpoint identifiers, and accompanied by corresponding matching confidence scores, generating a cross-viewpoint matching point pair set.
[0055] Specifically, the steps for identifying areas of abnormal smoothness and marking their spatial location include:
[0056] Extract the geometric datum features of the workpiece model to be tested and the standard workpiece model, solve the rigid body transformation matrix based on the singular value decomposition method, and align the workpiece model to be tested to the coordinate system of the standard workpiece model.
[0057] With the optimization objective of minimizing the sum of Euclidean distances between corresponding vertices of the workpiece model to be tested and the standard workpiece model, the iterative nearest point algorithm is used to perform global fine registration and establish a spatial mapping relationship.
[0058] For each mesh vertex of the workpiece model under test, calculate the average curvature value of the vertex, and combine the curvature-standard texture mapping relationship to normalize and correct the original smooth texture of each vertex of the workpiece model under test.
[0059] From the standard workpiece model and the normalized and calibrated workpiece model under test, the texture under test and the standard texture of each pair of vertices are extracted sequentially, and the smoothness difference of each vertex is calculated.
[0060] Filter out abnormal candidate vertices, aggregate them into abnormal candidate regions, and remove abnormal candidate regions whose total number of vertices is less than the minimum region threshold;
[0061] For the retained abnormal areas, extract attribute information and annotate them on the workpiece model to be tested.
[0062] Smoothness detection methods based on scale-invariant feature matching include:
[0063] Acquire multi-view images of the workpiece;
[0064] Spectrum attributes are determined for each viewpoint image, specular reflection regions are identified, and neighborhood texture interpolation is performed.
[0065] The gamma correction coefficients for different regions of the image are adaptively adjusted by the image edge curvature distribution and applied to the corresponding image regions to generate a multi-view workpiece image set.
[0066] A non-uniform multi-scale space is constructed by setting the scale sampling interval based on the edge curvature of each workpiece image.
[0067] By extracting SIFT features and verifying the consistency of curvature across scales, a workpiece feature set with curvature attribute labels is constructed.
[0068] Cross-view feature matching is performed on the feature subsets of each view in the workpiece feature set, and abnormal matching is filtered and eliminated by combining curvature attribute labels to obtain a cross-view matching point pair set.
[0069] An initial mesh model is constructed by triangulation spatial back projection and surface mesh reconstruction;
[0070] Texture features of each workpiece image are extracted from the multi-view workpiece image set and mapped to the initial mesh model to generate the workpiece model to be tested;
[0071] The model of the workpiece under test is globally registered and aligned with the standard workpiece model, and curvature normalization correction is performed. The abnormal smoothness area is determined by the feature difference between the model of the workpiece under test and the standard workpiece model.
[0072] The beneficial effects of this invention are:
[0073] Highlight regions on the workpiece surface are identified by jointly judging chroma and saturation, and texture restoration is completed by combining neighborhood interpolation. The gamma correction coefficient is adaptively adjusted based on the edge curvature distribution to balance the illumination differences in different curvature regions of the surface, so as to obtain multi-view workpiece images with uniform illumination and complete texture, avoiding feature distortion caused by specular reflection and uneven illumination, and providing a reliable data foundation for subsequent feature matching. Curvature-guided non-uniform multi-scale space is constructed, and scale sampling resources are adaptively allocated according to the curvature of the surface to improve the feature detection stability of high curvature regions. At the same time, curvature attribute labels are introduced to perform geometric constraint screening on cross-view matching results, effectively filtering out mismatched point pairs and ensuring the pose solution accuracy of 3D reconstruction. A 3D model of the workpiece under test with texture attributes is generated by triangulation spatial back projection and surface mesh reconstruction, integrating multi-view 2D image information into complete 3D surface data, covering the backlit surface, the inner wall of the hole, and the structural occlusion area, eliminating the blind spots of traditional single-view detection, and achieving complete detection coverage of the entire surface of the workpiece. Finally, in the anomaly detection stage, the smooth texture is normalized and corrected by combining the curvature of the surface, the inherent influence of the geometric shape on the texture baseline is removed, the evaluation benchmark of different curvature regions is unified, and microscopic smoothness anomalies are accurately identified by comparing vertex feature differences and topological connectivity constraints, avoiding misjudging normal texture differences caused by the curved surface structure as defects, and achieving full surface coverage and high-precision quantitative workpiece smoothness detection. Attached Figure Description
[0074] Figure 1 This is a structural diagram of a surface finish detection system based on scale-invariant feature matching.
[0075] Figure 2 This is a flowchart of obtaining a multi-view workpiece image set in this invention;
[0076] Figure 3 This is a flowchart of generating a cross-view matching point pair set in this invention;
[0077] Figure 4 This is a flowchart of a surface finish detection method based on scale-invariant feature matching. Detailed Implementation
[0078] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.
[0079] Example 1
[0080] refer to Figures 1 to 3As shown in the figure, this embodiment introduces a surface smoothness detection system based on scale-invariant feature matching, including: an image acquisition module, a texture matching module, and an anomaly detection module;
[0081] The image acquisition module is used to acquire multi-view images of valve hall hardware under different shooting distances, pitch angles, and horizontal orientations. By analyzing the chromaticity and saturation of the highlight pixels on the metal surface, the module determines the highlight attributes of each pixel in each view image, identifies the highlight reflection area in each image, and performs neighborhood texture interpolation on the highlight reflection area to achieve highlight suppression. The module adaptively adjusts the gamma correction coefficients of different regions of the image based on the edge curvature distribution of each view image, and applies the adjusted regional gamma correction coefficients to the corresponding image regions to complete the surface illumination equalization processing, thereby obtaining a multi-view workpiece image set with surface illumination equalization.
[0082] The texture matching module is used to construct a curvature-guided non-uniform multi-scale space, and generates a cross-view matching point pair set through cross-view feature matching and curvature attribute label-driven anomaly matching filtering. It generates an initial mesh model through triangulated space back projection and surface mesh reconstruction, and extracts texture features from the multi-view workpiece image set to map them into the initial mesh model to generate the model of the hardware to be tested.
[0083] The anomaly detection module is used to acquire the standard workpiece model and perform global registration and alignment between the model of the hardware under test and the standard workpiece model. Based on the surface curvature of each region of the 3D model, the smooth texture attribute is normalized and corrected. The feature difference between the model of the hardware under test and the standard workpiece model is calculated vertex by vertex. The smooth abnormal area is determined by the feature difference threshold and the spatial position is marked, generating the smoothness detection result of the entire surface of the valve hall hardware.
[0084] The texture matching module includes a feature extraction unit and a texture reconstruction unit.
[0085] The feature extraction unit is used to adaptively set the scale sampling interval based on the edge curvature of the workpiece image, thereby constructing a curvature-guided non-uniform multi-scale space. By using SIFT feature extraction and cross-scale curvature consistency verification, a workpiece feature set with curvature attribute labels is constructed. Then, the nearest neighbor distance ratio (NNDR) method is used to perform cross-view feature matching on the feature subsets of each view in the workpiece feature set to obtain initial matching point pairs. Finally, abnormal matching is filtered and eliminated from the initial matching point pairs by using curvature attribute labels to generate a cross-view matching point pair set.
[0086] The texture reconstruction unit is used to obtain the camera intrinsic parameter matrix. Combined with the cross-view matching point pair set, the camera pose transformation matrix between each view is solved. The sparse point cloud of the valve hall hardware is generated by triangulation spatial back projection. The initial mesh model is obtained by combining surface mesh reconstruction. The texture features of each workpiece image are extracted from the multi-view workpiece image set through the gray-level co-occurrence matrix. The texture features are mapped to the corresponding vertices of the initial mesh model with the camera pose transformation matrix as the mapping relationship. Each mesh vertex is given a smooth texture attribute to generate the model of the hardware to be tested.
[0087] Furthermore, the steps to identify the highlight reflection areas in each image include:
[0088] Each image is converted to the HSI color space from RGB format to obtain a converted hardware image. The chromaticity component, saturation component and luminance component are extracted from the converted hardware image as the decision components for highlight attribute determination, so as to remove the coupling interference of luminance information on color features.
[0089] Based on the optical characteristics of metal surfaces—high gloss, low saturation, stable chromaticity, and localized high brightness—a joint judgment of chromaticity, saturation, and localized brightness is used to perform pixel-by-pixel attribute analysis on the converted hardware images, filtering out high-brightness pixels. This includes: constructing a 5×5 neighborhood window centered on a single pixel, statistically analyzing the maximum and minimum brightness values within the neighborhood window, and using the ratio of the maximum to minimum brightness values as the local brightness contrast of a single pixel. Simultaneously, a reference image of standard valve hall hardware is retrieved. The reference image is a set of clean, defect-free hardware images covering various commonly used valve hall hardware, the curvature of the entire curved surface, and normal workshop lighting intensity. The chromaticity mean of the reference image is used as the reference chromaticity, and the chromaticity is compared with the reference chromaticity... The absolute value of the difference in saturation, divided by the reference chromaticity, is used as the chromaticity deviation of a single pixel. For a single pixel, it is marked as a highlight pixel only if the saturation is below the saturation threshold, the chromaticity deviation is below the chromaticity deviation threshold, and the local brightness contrast is above the contrast threshold. Specifically, the mean and standard deviation of saturation of pixels in the reference image are statistically analyzed, and the mean saturation minus twice the standard deviation is used as the saturation threshold. The chromaticity deviation distribution of pixels in the reference image is statistically analyzed, and the 95th percentile value is used as the chromaticity deviation threshold to exclude non-highlight pixels with abnormal chromaticity. The local brightness contrast distribution of pixels in the reference image is statistically analyzed, and the 95th percentile value is used as the contrast threshold to distinguish the local brightness of specular highlights from the uniform brightness of a flat surface.
[0090] By integrating all the highlight pixels in a single image, region clustering is performed using the eight-neighborhood connected component analysis method. Isolated noisy highlight regions with a total number of pixels less than the minimum area threshold are removed, resulting in several continuous highlight reflection regions. Specifically, the pixel resolution of a single frame image is obtained, including the number of pixels in width and height. Based on the noise statistics of the industrial metal surface detection scenario, a noise area ratio coefficient is set, exemplarily set to 0.01%. At the same time, a baseline pixel count is obtained. The product of the number of pixels in width, the number of pixels in height, and the noise area ratio coefficient is used as the correction pixel count. The larger value between the baseline pixel count and the correction pixel count is used as the minimum area threshold. The area of a 3×3 pixel connected component is used as the baseline noise size, corresponding to the typical isolated misjudged spot scale formed by sensor thermal noise and surface dust reflection, with a corresponding baseline pixel count of 9.
[0091] The Sobel edge detection operator is used to perform edge detection on the transformed hardware image, extract the edge pixels of the valve hall hardware, and form a set of structural edges;
[0092] Using the closed contours formed by edge pixels as the segmentation boundaries, the image is divided into multiple independent structural regions, and pixels within the same boundary contour are determined to belong to the same structural region.
[0093] Each pixel in the high-light reflection region is taken as the pixel to be repaired, and a neighborhood window is constructed for the pixel to be repaired. Within the neighborhood window, all pixels that simultaneously satisfy the conditions of being a non-high-light pixel, a non-edge pixel, and belonging to the same structural region as the pixel to be repaired are selected to form an effective interpolation sample set for the pixel to be repaired.
[0094] Using distance-luminance dual-weighted interpolation, the target luminance of the pixel to be repaired is calculated, and the luminance value of the original highlight pixel in the hardware image is replaced to complete the texture repair of the highlight area and obtain the hardware luminance image. Among them, the target luminance of the pixel to be repaired is the weighted sum of the luminances of all valid interpolation samples. The weight of each valid interpolation sample is obtained by multiplying the distance weight and the luminance similarity weight and then normalizing it. The distance weight is the reciprocal of the Euclidean distance between the sample pixel and the pixel to be repaired, and the luminance similarity weight is the reciprocal of the absolute value of the difference between the luminance of the sample pixel and the average luminance of the non-highlight area in the neighborhood window.
[0095] Furthermore, the steps for obtaining a multi-view workpiece image set with surface lighting equalization include:
[0096] For a single image of hardware brightness, the set of structural edges is traversed. Taking an edge pixel as the center, three consecutive edge pixels before and after it are selected along the edge direction, for a total of seven edge pixels, as the local fitting neighborhood. Quadratic curve fitting is used to fit the coordinates of pixels within the local fitting neighborhood. The local curvature of the corresponding edge pixel is calculated using the second derivative. This includes: to eliminate the interference of randomness in edge direction on the fitting, the edge pixel to be calculated is used as the origin of the coordinate system, and the coordinates of the sample point in the local tangent direction are used as the independent variable, and the coordinates in the local normal direction are used as the dependent variable. A local tangent coordinate system is established, and the original image coordinates of 7 pixels in the local fitting neighborhood are transformed one by one to the local tangent coordinate system to obtain 7 sets of discrete sample points. A quadratic polynomial is used as the fitting curve, and the least squares method is used to fit and solve the 7 sets of discrete sample points in the neighborhood. The obtained fitting curve approximately represents the curvature of the edge in the local neighborhood. The curvature of the plane curve is essentially the rate of change of the tangent direction of the curve with the arc length. Based on the relationship between the second derivative of the curve and the curvature, that is, the curvature is the absolute value of the second derivative, the local curvature of the corresponding edge pixel is obtained.
[0097] For non-edge regions, a neighborhood window is constructed for each pixel. All edge pixels within the neighborhood window whose local curvature has been calculated are used as interpolation reference points. Bilinear interpolation is used to calculate the linear distance weights of each interpolation reference point in the horizontal and vertical directions. The product of the linear distance weights in the two directions is used as the interpolation weight of the interpolation reference point. The weighted average of the local curvatures of all interpolation reference points is used as the local curvature of the corresponding pixel.
[0098] Traverse all pixels in a single hardware brightness image to complete the assignment, and filter the maximum local curvature of the hardware brightness image. Normalize the local curvature of all pixels by the ratio of local curvature to the maximum local curvature to obtain the normalized curvature, and construct a curvature distribution map that corresponds one-to-one with the image pixels.
[0099] To ensure that the gamma transform does not change the neutral state of the original image's grayscale, the reference gamma coefficient is set to 1, which serves as the correction reference for the flat surface area.
[0100] Obtain the curvature adjustment coefficient and combine it with the normalized curvature to obtain the gamma adjustment coefficient of the corresponding pixel. The product of the gamma adjustment coefficient and the reference gamma coefficient is used as the adaptive gamma coefficient of the corresponding pixel. Specifically, obtain calibration images with manually labeled normal grayscale reference ranges for each curvature region to construct a calibration dataset. A candidate coefficient set is constructed with an initial value of 0.2, an upper limit of 0.5, and a traversal step size of 0.05. For each candidate coefficient value, perform adaptive gamma correction on all images in the calibration dataset. Calculate the mean and variance of grayscale values in different curvature regions of the corrected image. A smaller variance indicates more uniform illumination in different curvature regions. The better the performance, the more we calculate the percentage of pixels with grayscale overflow (grayscale value reaching 0 or 255) in the image as a constraint. The optimization objective is to minimize the mean variance of grayscale values in each curvature region after correction, while ensuring that the percentage of grayscale overflow pixels is less than 0.5%. The optimal candidate value is selected from the candidate coefficient set as the curvature adjustment coefficient. The product of the curvature adjustment coefficient and the normalized curvature is subtracted from 1 as the gamma adjustment coefficient. After the single-channel pixel grayscale is transformed by gamma, pixels whose grayscale mapping range exceeds 0-255 are uniformly judged as overflow grayscale overflow pixels. The percentage of grayscale overflow pixels is the ratio of the total number of grayscale overflow pixels to the total number of pixels in the corresponding image.
[0101] Substitute the adaptive gamma coefficients corresponding to each pixel into the gamma transform to perform pixel-by-pixel grayscale correction on the brightness image of the hardware, complete the illumination equalization processing of different curvature areas of the surface, and obtain the workpiece image.
[0102] Curvature calculation and adaptive gamma correction are performed sequentially on the brightness images of the hardware from all acquisition angles. Combined with the viewpoint identifiers recorded synchronously during the image acquisition stage, the size and grayscale dynamic range of all images are unified to generate a multi-view workpiece image set. Among them, the viewpoint identifiers are assigned numbers to each image according to the shooting orientation, distance, and pitch angle, and correspond one-to-one with the pose record of the acquisition hardware.
[0103] Furthermore, the steps for constructing a curvature-guided non-uniform multi-scale space include:
[0104] Based on the SIFT Gaussian pyramid, a multi-scale spatial reference parameter set is set, including the reference scale, reference sampling interval, number of reference layers in each group, and total number of pyramid groups;
[0105] Traversing the curvature distribution map, the normalized curvature is mapped to the density adjustment factor of the corresponding pixel; where the density adjustment factor is the product of the normalized curvature and the standard adjustment degree. Standard valve hall hardware images covering different curvature structures are collected. The optimization target is the comprehensive ratio of feature point recall rate and single image computation time. The optimal coefficient is determined by traversing the optimization with a value range of 0.1-0.6 and a step size of 0.05 to obtain the standard adjustment degree. The proportion of effective feature points extracted through non-uniform multi-scale space in the total number of manually labeled ground truth feature points in the same image is the feature point recall rate. The single image computation time is the processor runtime for constructing a complete multi-scale space for a single frame image.
[0106] Based on the baseline sampling interval and combined with the density adjustment factor, the scale sampling interval for each pixel is calculated; where the scale sampling interval is the product of the baseline sampling interval and 1 minus the density adjustment factor.
[0107] For the workpiece image, the curvature gradient of each pixel is calculated by the first difference of the curvature distribution. That is, the curvature difference between the pixel and its neighboring pixels in the horizontal and vertical directions is calculated respectively, and the square root of the sum of the squares of the curvature differences in the two directions is used as the curvature gradient of the corresponding pixel.
[0108] The 95th percentile of all curvature gradients in the workpiece image is used as the abrupt change threshold. Pixels with curvature gradients greater than the abrupt change threshold are identified as abrupt change pixels. All consecutive abrupt change pixels are integrated to form a gradient abrupt change boundary, which corresponds to the transition boundary between the flat area and the curved corner area of the hardware surface.
[0109] Based on the gradient abrupt boundary, the transition zone is expanded to both sides of the boundary to form a transition zone. Specifically, the curvature distribution in the standard valve hall hardware image is obtained, the pixel width of the natural curvature transition area at the junction of the curved surface corner and the flat surface is statistically analyzed, and the 90th percentile value of the statistical result is taken as the unilateral expansion width to ensure that the transition zone completely covers the natural gradual change range of curvature abrupt change.
[0110] Using the boundary pixels on both sides of the transition zone as endpoints, for each pixel within the transition zone, the distance ratio from the pixel to the two endpoints is used as the interval weight of the endpoints. Linear interpolation is performed by weighted summation, and the interpolation result is used to replace the calculated scale sampling interval to eliminate scale parameter discontinuities between different curvature regions and ensure the continuous smoothness of the scale space.
[0111] Based on the scale sampling interval of each pixel in the workpiece image, Gaussian blur is used to construct a non-uniform multi-scale space in which the hierarchical distribution adapts to the curvature.
[0112] Furthermore, the steps for constructing a set of workpiece features labeled with curvature attributes include:
[0113] In a non-uniform multi-scale space, initial feature points are extracted through multi-scale neighborhood extremum detection to initially locate potential scale-invariant feature locations. This includes: for each pixel on each scale layer, comparing its grayscale response value with the 8 neighboring pixels of the same scale layer and the 9 neighboring pixels of the corresponding positions of the two adjacent scale layers above and below. When the response value of a pixel is the maximum or minimum value among all neighboring pixels, it is determined to be an initial feature point. Each initial feature point is accompanied by the pixel coordinates of its scale layer, the pyramid group number, and the layer number.
[0114] For each initial feature point, the pyramid group number and layer number are called. The scaling ratio of the scale layer relative to the spatial resolution of the original image is used as the scale ratio. The pixel coordinates of the initial feature point in the current scale layer, the previous adjacent scale layer, and the next adjacent scale layer are mapped back to the original image coordinate system through the scale ratio, so as to obtain three sets of coordinate positions corresponding to the original image.
[0115] In the curvature distribution map, the local curvature corresponding to three sets of coordinate positions is extracted respectively, and the curvature difference between the current scale layer and the previous scale layer, and the curvature difference between the current scale layer and the next scale layer are calculated respectively.
[0116] When both curvature differences are below the curvature consistency threshold, the initial feature point is determined to have cross-scale geometric stability and is retained; otherwise, it is discarded as a pseudo feature point. Feature points are extracted from the standard valve hall hardware image, the distribution of curvature differences of the retained feature points is statistically analyzed, and the 95th percentile value is taken as the curvature consistency threshold.
[0117] Based on the standard descriptor generation rules of the SIFT algorithm, gradient histogram statistics and principal direction assignment are performed on the retained feature points, and feature descriptors of corresponding dimensions are generated based on the principal directions, so that the feature points have rotation and scale invariance.
[0118] The local curvature corresponding to each feature point is used as a curvature attribute label and bound one by one with the feature descriptor, the pixel coordinates of the original image, and the scale parameter to which the feature point belongs (including the pyramid group number, layer number, and corresponding scale value) to form a single complete feature point data.
[0119] Based on the viewpoint identifier, all feature point data under the same viewpoint are integrated into the feature subset of the corresponding viewpoint, and the feature subsets of all viewpoints together constitute the workpiece feature set.
[0120] Furthermore, the steps for generating a cross-view matching point pair set include:
[0121] Calculate the coordinates of the geometric center point of the camera position for all shooting angles based on the camera spatial position coordinates attached to all viewpoints.
[0122] Calculate the Euclidean distance between the camera position and the geometric center point for each viewpoint, select the viewpoint with the smallest distance as the reference viewpoint, and use the remaining viewpoints as viewpoints to be matched. Construct viewpoint task pairs that correspond one-to-one with the reference viewpoint and each viewpoint to be matched. All viewpoints use the reference viewpoint as a unified matching reference to avoid the accumulation of errors in pairwise matching of multiple viewpoints.
[0123] Each viewpoint task pair is traversed sequentially. For each feature point within the viewpoint to be matched, the Euclidean distance between feature descriptors is used as the similarity measure. A global search for candidate matching points is performed in the feature subset of the baseline viewpoint. The two candidate feature points with the smallest similarity measure are obtained in the baseline viewpoint and are denoted as the nearest neighbor candidate point and the second nearest neighbor candidate point, respectively. The Euclidean distance between the nearest neighbor candidate point and the feature point within the viewpoint to be matched is denoted as the nearest neighbor distance, and the Euclidean distance between the second nearest neighbor candidate point and the feature point within the viewpoint to be matched is denoted as the second nearest neighbor distance.
[0124] The ratio of the nearest neighbor distance to the second nearest neighbor distance is calculated as the nearest neighbor distance ratio, and an initial matching threshold is set. In this process, multiple sets of standard surface finish metal fitting images from different perspectives are collected, and a set of ground truth matching points is constructed by manual annotation. The sum of matching accuracy and recall is used as the optimization objective. Within the normal value range (i.e., within the 0-1 range), the search is carried out with a fixed step size (i.e., 0.05) to determine the optimal initial matching threshold.
[0125] When the nearest neighbor distance ratio is less than the initial matching threshold, the feature point in the view to be matched is determined to form an initial matching point pair with the nearest neighbor candidate point; when the nearest neighbor distance ratio is not less than the initial matching threshold, the feature point is determined to have no reliable match and is excluded.
[0126] For each initial matching point pair, the curvature attribute labels corresponding to the two feature points are extracted respectively. The relative curvature deviation of the initial matching point pair is obtained by dividing the absolute difference of the local curvature of the two points by the larger value of the two local curvatures, so as to eliminate the difference in absolute numerical scale between different curvature intervals.
[0127] Set a curvature matching threshold and perform curvature filtering on initial matching point pairs. If the relative curvature deviation is less than the curvature matching threshold, the current initial matching point pair is deemed to meet the surface geometric consistency constraint, and the curvature filtering is passed and retained. If the relative curvature deviation is not less than the curvature matching threshold, it is determined to be a geometrically mismatched pair and the current initial matching point pair is removed. In this process, multi-view true value matching point pairs of standard valve hall hardware are obtained, the distribution of the relative curvature deviation of the correct matching point pairs is statistically analyzed, and the 95th percentile value is taken as the curvature matching threshold to ensure that the vast majority of correct matches can pass the filtering.
[0128] For the matching point pairs filtered by curvature, the Euclidean distance and the relative deviation of curvature between the feature descriptors are called, and the maximum and minimum normalization processes are performed. The reciprocals of the normalized values are used for weighted summation to obtain the matching confidence; where the weights of the weighted summation are equal.
[0129] All matching point pairs filtered by curvature are aggregated, categorized and stored according to viewpoint identifiers, and accompanied by corresponding matching confidence scores, generating a cross-viewpoint matching point pair set.
[0130] Furthermore, the steps for identifying areas of abnormal smoothness and marking their spatial location include:
[0131] Geometric datum features, such as bolt hole center, end face contour vertex, and assembly surface boundary point, are extracted from the model of the hardware to be tested and the standard workpiece model respectively to construct a three-dimensional feature point pair;
[0132] The rigid body transformation matrix is solved by the singular value decomposition method, and the model of the hardware under test is aligned to the unified coordinate system of the standard workpiece model to complete the initial coarse registration.
[0133] Based on the coarse registration results, an iterative nearest-point algorithm is used for global fine registration. The optimization objective is to minimize the sum of the Euclidean distances between corresponding vertices of the hardware model under test and the standard workpiece model. The rotation matrix and translation vector are iteratively optimized until the registration error converges to the registration convergence threshold, thus establishing a one-to-one spatial mapping relationship between the mesh vertices of the hardware model under test and the standard workpiece model. In particular, a registration test is performed on the standard hardware model with a known pose, and the value at which the registration error stably converges is taken as the registration convergence threshold.
[0134] For each grid vertex of the hardware model to be tested, all adjacent vertices within its one-ring neighborhood are selected to form a local fitting neighborhood. The average curvature value of the vertex is calculated using the quadratic surface fitting method. After traversing all vertices, the vertex-level three-dimensional curvature distribution of the hardware model to be tested is obtained.
[0135] Based on the curvature-standard texture mapping relationship of the standard workpiece model, the original smooth texture of each vertex of the hardware model under test is normalized and corrected. This includes: obtaining the standard texture reference value under the average curvature value by interpolation through a lookup table based on the average curvature value of the vertex under test; using the global standard texture mean as a unified reference, proportionally correcting the original texture value to obtain the corrected smooth texture after eliminating curvature interference; wherein, the curvature-standard texture mapping relationship is pre-constructed through the standard workpiece model, the average standard smooth texture value corresponding to different curvature intervals in the standard workpiece model is statistically analyzed to generate a discrete mapping table, and for any curvature value, the corresponding standard texture reference value is obtained by linear interpolation. The standard workpiece model reuses the entire set of steps in this embodiment, including image acquisition, specular restoration, adaptive gamma correction, curvature-guided non-uniform multi-scale space, cross-view feature matching, and triangulation mesh reconstruction, to acquire multi-view images of standard defect-free valve hall hardware from the factory, generate a standard multi-view workpiece image set through specular suppression and adaptive gamma correction, and perform feature extraction, cross-view matching, and 3D mesh reconstruction. The process involves grid reconstruction to generate a standard workpiece model. It iterates through all mesh vertices of the standard workpiece model, calculates the average curvature value of each vertex, and extracts texture features from the corresponding gray-level co-occurrence matrix. Texture mean values are grouped by curvature interval, generating a discrete curvature-texture mapping table. This table is stored as key-value pairs, where the key is the discrete normalized curvature interval and the value is the standard texture mean value for that interval. The curvature interval is determined based on normalized curvature. The process then iterates through all mesh vertices of the standard workpiece model, retrieving the normalized curvature of each vertex. A standard curvature sample set is compiled, covering all surface shapes of hardware, including planes, arcs, sharp corners, and inner walls of holes. The normalized curvature is sorted in ascending order and divided into equal frequency intervals to obtain several curvature intervals. Positive integers are used as the number of trial divisions in sequence, and the minimum number of vertices that a single curvature interval can accommodate is used as the constraint benchmark. The trial divisions are iterated until the number of grid vertices that each curvature interval can accommodate reaches the benchmark capacity. In the true value matching point pair set, the benchmark capacity is the minimum number of vertex samples that can distinguish normal surface textures from microscopic clean defects.
[0136] Based on spatial mapping relationships, the texture to be tested and the standard texture of each pair of vertices are extracted sequentially from the standard workpiece model and the normalized and calibrated hardware model. The smoothness difference of each vertex is obtained by the absolute difference between the texture to be tested and the standard texture.
[0137] A difference threshold is set to initially determine abnormal vertices; if the smoothness difference is greater than the difference threshold, the current vertex is determined to be a smoothness abnormal candidate and marked as an abnormal candidate vertex; if the smoothness difference is not greater than the difference threshold, the current vertex is determined to be a normal vertex.
[0138] Integrate all anomalous candidate vertices, perform connected component analysis based on the one-ring adjacency topology of the 3D mesh, and aggregate spatially consecutive adjacent anomalous candidate vertices into anomalous candidate regions;
[0139] Set a minimum region threshold to remove abnormal candidate regions whose total number of vertices is less than the minimum region threshold, and retain them as abnormal regions; for the set of true matching point pairs, statistically analyze the distribution of the number of grid vertices corresponding to various real microscopic cleanliness defects, and take the lower limit of the number of vertices of the true defect sample as the minimum region threshold.
[0140] For the retained abnormal areas, attribute information is extracted, including three-dimensional spatial boundary coordinates, number of covered vertices, maximum difference value and average difference value. Visual boundary annotations are performed on the model of the hardware to be tested. The normal area judgment results and abnormal area annotation information of the whole surface are integrated to generate the full surface finish test results of the valve hall hardware.
[0141] Example 2
[0142] Please see Figure 4 Another embodiment of the present invention provides a surface finish detection method based on scale-invariant feature matching, comprising the following steps:
[0143] Step S1: Since the valve hall hardware is made of metal, its surface has strong specular highlight reflection, and the illumination distribution is uneven in different curvature areas of the curved surface. Direct feature extraction is prone to producing false feature points and feature description deviations. Multi-view images of the valve hall hardware under different shooting distances, pitch angles and horizontal orientations are obtained. By using the chromaticity and saturation of the highlight pixels on the metal surface, the highlight attribute of each pixel in each view image is determined, thereby identifying the highlight reflection area and performing neighborhood texture interpolation. The gamma correction coefficients of different regions of the image are adaptively adjusted by the curvature distribution of the image edge. The adjusted regional gamma correction coefficients are applied to the corresponding image regions to obtain a multi-view workpiece image set.
[0144] Step S2: Based on each workpiece image in the multi-view workpiece image set, the scale sampling interval is set adaptively by edge curvature to construct a curvature-guided non-uniform multi-scale space. SIFT feature extraction is used, and feature points are selected by cross-scale curvature consistency verification. The principal direction assignment and descriptor generation are combined, and curvature attribute labels are added to each feature point to construct a workpiece feature set with curvature attribute labels.
[0145] Step S3: To reduce feature mismatch caused by complex surface deformation, the nearest neighbor distance ratio (NNDR) method is used to perform cross-view feature matching on the feature subsets of each view in the workpiece feature set to obtain initial matching point pairs. Then, abnormal matching is filtered and eliminated from the initial matching point pairs through curvature attribute labels to obtain a cross-view matching point pair set.
[0146] Step S4: Obtain the camera intrinsic parameter matrix, combine it with the cross-view matching point pair set, solve the camera pose transformation matrix between each view based on the epipolar geometry principle, generate the sparse point cloud of the valve hall hardware through triangulation spatial back projection, obtain the initial mesh model through surface mesh reconstruction, extract the texture features of each workpiece image from the multi-view workpiece image set through the gray-level co-occurrence matrix, map the texture features to the corresponding vertices of the initial mesh model with the camera pose transformation matrix as the mapping relationship, assign smooth texture attributes to each mesh vertex, and generate the hardware model to be tested.
[0147] Step S5: To eliminate the interference of surface curvature differences on the smooth texture features and achieve refined surface finish inspection of the entire surface, the model of the hardware under test is globally registered and aligned with the standard workpiece model. The surface texture attributes are normalized and corrected according to the surface curvature of each region of the 3D model. The feature difference between the model of the hardware under test and the standard workpiece model is calculated vertex by vertex. The abnormal surface areas are determined by the feature difference threshold and their spatial positions are marked to generate the surface finish inspection results of the entire surface of the valve hall hardware.
[0148] Working principle and effects:
[0149] Based on the joint determination of high-gloss areas on metal surfaces using chroma and saturation, repair is completed through neighborhood texture interpolation. Simultaneously, the gamma correction coefficient is adaptively adjusted according to the edge curvature distribution to balance the illumination differences in different areas of the surface, outputting high-quality multi-view images. This eliminates the interference of specular reflection and uneven illumination on subsequent feature extraction from the source, ensuring the reliability of feature data. A curvature-guided non-uniform multi-scale space is constructed to extract scale-invariant features. Curvature attribute constraints are introduced to filter cross-view matching point pairs, improving the geometric consistency of matching point pairs. A 3D hardware model with texture attributes is obtained through triangulation back projection and mesh reconstruction. Multi-view data is integrated to eliminate detection blind spots caused by structural occlusion, achieving complete coverage of the entire hardware surface. Curvature normalization correction unifies the texture evaluation benchmark for different curvature regions. Vertex-by-vertex comparison with the standard model identifies surface finish anomalies. Combined with topological connectivity constraints, isolated misjudgments are filtered out. This ensures the accuracy of microscopic surface finish anomaly identification while avoiding false detections caused by differences in surface geometry, adapting to the refined inspection needs of UHV valve hall hardware.
[0150] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.
Claims
1. A surface finish detection system based on scale-invariant feature matching, characterized in that, include: Image acquisition module, texture matching module, and anomaly detection module; The image acquisition module is used to acquire multi-view images of the workpiece. By using the chromaticity and saturation of the highlight pixels on the workpiece surface, the highlight attribute of each pixel in each view image is determined, the highlight reflection area is identified and neighborhood texture interpolation is performed, and the gamma correction coefficient is adaptively adjusted according to the curvature distribution of the image edge to obtain a multi-view workpiece image set. The texture matching module is used to construct a curvature-guided non-uniform multi-scale space, and generate a cross-view matching point pair set through cross-view feature matching and curvature attribute label-driven anomaly matching screening. It generates an initial mesh model through triangulation space back projection and surface mesh reconstruction, and extracts texture features from the multi-view workpiece image set to map them into the initial mesh model to generate the workpiece model to be tested. The anomaly detection module is used to acquire a standard workpiece model, combine the curvature normalization correction of the smooth texture attribute with the surface curvature, calculate the feature difference between the workpiece model to be tested and the standard workpiece model vertex by vertex, thereby determining the smooth abnormal area and marking its spatial position.
2. The surface finish detection system based on scale-invariant feature matching according to claim 1, characterized in that: The texture matching module includes a feature extraction unit and a texture reconstruction unit; The feature extraction unit adaptively sets the scale sampling interval based on the edge curvature of the workpiece image to construct a curvature-guided non-uniform multi-scale space. Through the consistency verification of feature extraction and cross-scale curvature, a workpiece feature set with curvature attribute labels is constructed. Cross-view feature matching is performed on the feature subsets of each view in the workpiece feature set to obtain initial matching point pairs. Anomaly matching is filtered and eliminated by combining the curvature attribute labels to generate a cross-view matching point pair set. The texture reconstruction unit is used to obtain the camera intrinsic parameter matrix, combine the cross-view matching point pair set, solve the camera pose transformation matrix between each view, generate an initial mesh model through triangulation space back projection and surface mesh reconstruction, extract the texture features of each workpiece image from the multi-view workpiece image set through the gray-level co-occurrence matrix, and map the texture features to the corresponding vertices of the initial mesh model using the camera pose transformation matrix as the mapping relationship to generate the workpiece model to be tested.
3. The surface finish detection system based on scale-invariant feature matching according to claim 1, characterized in that, The steps for identifying specular reflection regions and performing neighborhood texture interpolation include: The image is converted to a color space to obtain a converted workpiece image, and the decision components are extracted, including the chroma component, saturation component and luminance component. By combining chroma, saturation, and local brightness, pixel-by-pixel attribute determination is performed on the converted workpiece image to filter out highlight pixels. Then, the eight-neighbor connected component analysis method is used to perform region clustering to obtain the highlight reflection region. Edge detection is performed on the converted workpiece image to generate a set of structural edges; The image is divided into multiple structural regions using the closed contours formed by edge pixels as segmentation boundaries; For the pixels to be repaired in the highlight reflection area, select pixels that are neither highlight nor edge and belong to the same structural region to form an effective interpolation sample set; By using distance-brightness dual-weighted interpolation, the target brightness of the pixel to be repaired is calculated and replaced to obtain the brightness image of the workpiece.
4. The surface finish detection system based on scale-invariant feature matching according to claim 3, characterized in that, The multi-view workpiece image set includes: A local fitting neighborhood is constructed for the edge pixels, and a quadratic curve is fitted. The local curvature of the edge pixels is calculated using the second derivative of the fitted curve. For non-edge regions, bilinear interpolation is performed using the local curvature and distance weights of edge pixels within the neighborhood window. The maximum local curvature value in the brightness image of the workpiece is selected to normalize the local curvature of all pixels and generate a curvature distribution map of the brightness image of the workpiece. Set a baseline gamma coefficient, and calculate the gamma adjustment coefficient of the corresponding pixel based on the normalized curvature and curvature adjustment coefficient to obtain the adaptive gamma coefficient. Substitute the adaptive gamma coefficients corresponding to each pixel into the gamma transform to perform grayscale correction on the brightness image of the workpiece pixel by pixel. Highlight suppression and adaptive gamma correction are performed on the brightness images of the workpiece from all acquisition perspectives. Combined with the corresponding perspective labels, a multi-view workpiece image set is generated.
5. The surface finish detection system based on scale-invariant feature matching according to claim 4, characterized in that, The steps for constructing a curvature-guided non-uniform multi-scale space include: Based on the SIFT Gaussian pyramid, a set of reference parameters for multi-scale space is set. Traverse the curvature distribution map and map the normalized curvature to the density adjustment factor of the corresponding pixel; Based on the reference sampling interval in the reference parameter set, and combined with the density adjustment factor, the scale sampling interval for each pixel is calculated; By using the first difference of the curvature distribution, the curvature gradient of each pixel is calculated, and pixels with curvature gradients greater than the mutation threshold are identified as mutation pixels, thereby constructing gradient mutation boundaries. Based on the gradient abrupt boundary, the transition zone is expanded to both sides of the boundary to form a transition zone, with the boundary pixels on both sides of the transition zone as the endpoints; For pixels within the transition zone, the distance ratio from the pixel to both endpoints is used as the endpoint interval weight, and linear interpolation is performed by weighted summation. Based on the scale sampling interval of each pixel in the workpiece image, a non-uniform multi-scale space is constructed using Gaussian blur.
6. The surface finish detection system based on scale-invariant feature matching according to claim 5, characterized in that, The steps for constructing a workpiece feature set with curvature attribute labels include: In the non-uniform multi-scale space, initial feature points are extracted through multi-scale neighborhood extremum detection; By scaling the scale, the pixel coordinates of the initial feature points at each scale layer are mapped to the original image coordinate system, and the local curvature of the pixel coordinates of the initial feature points at the current scale layer and the adjacent scale layers above and below is extracted. Calculate the curvature difference between adjacent scale layers and retain only feature points whose curvature difference is lower than the curvature consistency threshold; Based on the descriptor generation rules of the SIFT algorithm, feature points are assigned principal directions, and feature descriptors of corresponding dimensions are generated. The local curvature corresponding to each feature point is used as a curvature attribute label and bound to the feature descriptor, the pixel coordinates of the original image, and the scale parameter to which the feature point belongs, to form a single feature point data. Based on the viewpoint identifier, all feature point data under the same viewpoint are integrated into a feature subset to generate a workpiece feature set.
7. The surface finish detection system based on scale-invariant feature matching according to claim 6, characterized in that, The steps to generate a cross-view matching point pair set include: Based on the Euclidean distance between the camera position of a single viewpoint and the geometric center point of all viewpoints, a reference viewpoint and a viewpoint to be matched are divided, and viewpoint task pairs are constructed. Each viewpoint task pair is traversed sequentially, and a global search is performed in the baseline viewpoint using the Euclidean distance between feature descriptors to obtain the nearest neighbor candidate point and the second nearest neighbor candidate point. Calculate the ratio of the nearest neighbor distance to the second nearest neighbor distance, and use it as the nearest neighbor distance ratio; When the nearest neighbor distance ratio is less than the initial matching threshold, it is determined that the feature point in the view to be matched and the nearest neighbor candidate point constitute an initial matching point pair. If the nearest neighbor distance ratio is not less than the initial matching threshold, then the feature point is determined to be unmatched.
8. The surface finish detection system based on scale-invariant feature matching according to claim 7, characterized in that, The steps for generating a cross-view matching point pair set also include: For each initial pair of matching points, extract the curvature attribute labels corresponding to the two feature points and calculate the relative curvature deviation; Set the curvature matching threshold and perform curvature filtering on the initial matching point pairs; If the relative curvature deviation is less than the curvature matching threshold, the curvature screening is deemed successful; otherwise, the current initial matching point pair is discarded. For the matching point pairs filtered by curvature, the Euclidean distance and relative curvature deviation between feature descriptors are called, and normalized and weighted sum of the reciprocals are performed to obtain the matching confidence. All matching point pairs filtered by curvature are aggregated, categorized and stored according to viewpoint identifiers, and accompanied by corresponding matching confidence scores, generating a cross-viewpoint matching point pair set.
9. The surface finish detection system based on scale-invariant feature matching according to claim 8, characterized in that, The steps for identifying and spatially locating areas with abnormal smoothness include: Extract the geometric datum features of the workpiece model to be tested and the standard workpiece model, solve the rigid body transformation matrix based on the singular value decomposition method, and align the workpiece model to be tested to the coordinate system of the standard workpiece model. With the optimization objective of minimizing the sum of Euclidean distances between corresponding vertices of the workpiece model to be tested and the standard workpiece model, the iterative nearest point algorithm is used to perform global fine registration and establish a spatial mapping relationship. For each mesh vertex of the workpiece model under test, calculate the average curvature value of the vertex, and combine the curvature-standard texture mapping relationship to normalize and correct the original smooth texture of each vertex of the workpiece model under test. From the standard workpiece model and the normalized and calibrated workpiece model under test, the texture under test and the standard texture of each pair of vertices are extracted sequentially, and the smoothness difference of each vertex is calculated. Filter out abnormal candidate vertices, aggregate them into abnormal candidate regions, and remove abnormal candidate regions whose total number of vertices is less than the minimum region threshold; For the retained abnormal areas, attribute information is extracted and annotated on the workpiece model to be tested.
10. A surface finish detection method based on scale-invariant feature matching, used to implement the surface finish detection system based on scale-invariant feature matching as described in any one of claims 1-9, characterized in that, include: Acquire multi-view images of the workpiece; Spectrum attributes are determined for each viewpoint image, specular reflection regions are identified, and neighborhood texture interpolation is performed. The gamma correction coefficients for different regions of the image are adaptively adjusted by the image edge curvature distribution and applied to the corresponding image regions to generate a multi-view workpiece image set. A non-uniform multi-scale space is constructed by setting the scale sampling interval based on the edge curvature of each workpiece image. By extracting SIFT features and verifying the consistency of curvature across scales, a workpiece feature set with curvature attribute labels is constructed. Cross-view feature matching is performed on the feature subsets of each view in the workpiece feature set, and abnormal matching is filtered and eliminated by combining curvature attribute labels to obtain a cross-view matching point pair set. An initial mesh model is constructed by triangulation spatial back projection and surface mesh reconstruction; Texture features of each workpiece image are extracted from the multi-view workpiece image set and mapped to the initial mesh model to generate the workpiece model to be tested; The model of the workpiece under test is globally registered and aligned with the standard workpiece model, and curvature normalization correction is performed. The abnormal smoothness area is determined by the feature difference between the model of the workpiece under test and the standard workpiece model.
Citation Information
Patent Citations
Multispectral extra-high pressure valve hall defect identification method and equipment
CN120852725A
