A non-contact metrology detection method and system based on image recognition

Abstract

This invention relates to the fields of image recognition technology and non-contact metrology and inspection technology, and discloses a non-contact metrology and inspection method and system based on image recognition. The method effectively solves the problem of high light saturation caused by highly reflective surfaces by combining multi-light intensity level projection with adaptive exposure adjustment, ensuring the integrity of the original structured light information. Simultaneously, by performing local texture analysis on the structured light information map and filtering out false feature point clouds, this application can eliminate false data caused by multiple light reflections, thereby significantly improving the accuracy and reliability of 3D reconstruction, and providing a more accurate and efficient solution for non-contact metrology and inspection of high-precision components.

Description

Technical Field

[0001] This invention relates to the fields of image recognition technology and non-contact metrology and testing technology, and more specifically, to a non-contact metrology and testing method and system based on image recognition. Background Technology

[0002] In precision manufacturing, ensuring component quality is crucial for the performance and safety of the final product. Traditional contact measurement methods, while effective, are often affected by environmental interference and pose a risk of accumulating measurement errors. These shortcomings significantly impact product yield and overall inspection efficiency. To overcome these challenges, non-contact measurement systems, especially those utilizing image recognition technology, have become a more popular solution. These systems are designed to provide accurate and efficient dimensional and geometric tolerance inspection without direct contact with the component.

[0003] Existing technologies attempt to "avoid" highlights by increasing the number of cameras and light sources, but for complex curved surfaces and concave structures, multi-angle shooting still struggles to avoid false features caused by multiple reflections. Another solution is to spray a developer, but this adds extra steps, introduces measurement errors, and may contaminate the surface cleanliness of the parts, making it unsuitable for high-end manufacturing. Therefore, how to effectively suppress or separate specular reflection interference in images, fill data gaps caused by highlights, and correct erroneous data introduced by multiple reflections without altering the surface condition of the parts, significantly increasing hardware complexity, or reducing detection efficiency is a pressing technical problem that needs to be solved.

[0004] The existing technologies mainly involve the following categories: (1) High dynamic range imaging methods based on multi-exposure image fusion, which take multiple shots of the same scene with fixed exposure parameters and perform pixel-level or region-level fusion on the resulting images to expand the dynamic range of the images; (2) Reflection light separation methods based on polarization modulation, which use polarizers or polarization cameras to distinguish between specular reflection and diffuse reflection components to suppress the interference of specular reflection on structured light patterns; (3) Specular removal methods based on learning, which use deep neural networks to recover areas obscured by specular light from single or multiple images; (4) Multi-frequency multi-step phase shift methods based on structured light, which improve the accuracy of three-dimensional reconstruction by projecting multiple sets of phase shift gratings and jointly solving the phase; (5) Stripe quality assessment methods based on multiple criteria, which combine multiple metrics such as phase consistency, modulation degree, and contrast to identify abnormal areas. However, the existing technologies still have limitations in addressing the two problems of "data holes" caused by specular saturation and "pseudo-feature point clouds" formed by multiple reflections. This application proposes a holistic solution to the above problems. Summary of the Invention

[0005] This invention provides a non-contact metrology and inspection method and system based on image recognition, aiming to solve the problems of data voids and pseudo feature point clouds caused by high light reflection and multiple reflections in non-contact metrology and inspection of highly reflective, complex curved surfaces and concave structural components.

[0006] The technical solution of this application is as follows: In a first aspect, this application discloses a non-contact metrological detection method based on image recognition, comprising: Project a structured light pattern onto the surface of the component under test, and project the same structured light pattern using N different light intensity levels in sequence within a preset time window, where N is a positive integer greater than or equal to 3; At each light intensity level, deformable structured light images of the surface of the component under test are acquired synchronously, and the exposure parameters for image acquisition are automatically adjusted according to the current light intensity level to obtain images containing unsaturated pixel areas. Identify saturated pixel regions in each image caused by localized highlight reflection; For the identified saturated pixel region, select at least one image from N images corresponding to N light intensity levels that is not saturated at the corresponding pixel position, and extract the structured light pattern information of the selected image at the corresponding pixel position; For the identified unsaturated pixel regions, select the image with the highest light intensity level from the N images corresponding to the N light intensity levels, and extract the structured light pattern information of the image at the corresponding pixel position. The structured light pattern information extracted from the saturated pixel region and the structured light pattern information extracted from the unsaturated pixel region are spliced ​​and fused according to the original pixel spatial position to generate a complete structured light information map without saturated pixel regions. Local texture analysis was performed on the structured light information map to identify and filter out false feature point clouds formed by multiple reflections of light on the surface of the component; Based on the structured light information map after filtering out pseudo-feature point clouds, the three-dimensional spatial coordinates of the surface of the component under test are calculated using the principle of triangulation.

[0007] This technical solution can effectively solve the problem of data holes caused by high light reflection on the surface of highly reflective components and suppress the false feature point cloud formed by multiple reflections, thereby improving the accuracy and reliability of 3D reconstruction.

[0008] Furthermore, before splicing and fusing the structured light pattern information extracted from the saturated pixel region and the structured light pattern information extracted from the unsaturated pixel region according to the original pixel spatial positions, the process also includes: Extract feature points with local contrast variations from the N images; Based on this feature point, calculate the initial offset between the N images; Based on this initial offset, perform preliminary resampling and alignment on the N images; The local signal-to-noise ratio of the structured light pattern in the image after initial resampling and alignment is evaluated to determine the reliability of each pixel location; Based on this reliability, the subpixel-level displacement field between the N images is iteratively calculated using the optical flow method or the cross-correlation method. Based on the subpixel-level displacement field, the N images are finely calibrated and resampled until the change in displacement field calculated in two adjacent iterations is less than a preset threshold.

[0009] This technical solution enables subpixel-level precise alignment of multiple images, providing a high-precision foundation for subsequent image fusion and further improving the quality of the fused structured light information map.

[0010] Furthermore, the identification and filtering of spurious feature point clouds formed by multiple reflections of light on the surface of the component includes: A sliding window of a preset size is used to scan the structured light information map pixel by pixel; Within each sliding window, calculate the phase coherence value and modulation value of the grating fringes; If the phase consistency value is lower than the first preset threshold and the modulation value is higher than the second preset threshold, then the spatial point corresponding to the center pixel of its sliding window is marked as a pseudo feature point. The set of all spatial points marked as pseudo-feature points is defined as the pseudo-feature point cloud; Replace each pseudo-feature point in the pseudo-feature point cloud with the average coordinates of the unlabeled spatial points in its neighborhood to filter out the pseudo-feature point cloud.

[0011] This technical solution can effectively identify and filter out false feature point clouds caused by multiple reflections, avoiding their interference with the 3D reconstruction results and improving the realism of the reconstruction model.

[0012] Building upon the above, this application further proposes that after generating a complete structured light information map of unsaturated pixel regions, it also includes: Calculate the full-image brightness histogram of the structured light information map; Based on the full-image brightness histogram, identify low-texture areas with contrast below a preset contrast threshold; For this low-texture region, an anisotropic diffusion filtering method based on the structure tensor is used to enhance the contrast of the grating fringes in the region. Phase unwrapping calculations are performed on the contrast-enhanced grating fringes to obtain a continuous absolute phase distribution.

[0013] This technical solution can specifically enhance the contrast of low-texture areas and improve the accuracy of phase unwrapping, thereby obtaining a more complete and accurate absolute phase distribution.

[0014] In some preferred embodiments, for the identified saturated pixel region, selecting at least one image from the N images corresponding to the N light intensity levels where the corresponding pixel position is unsaturated includes: Establish the light intensity level and brightness response curve for each pixel location; Based on the light intensity level and brightness response curve, determine the set of unsaturated light intensity levels corresponding to each pixel location; If the set of unsaturated light intensity levels contains multiple light intensity levels, then select the light intensity level whose corresponding brightness value is in the middle of the preset optimal response range; Extract the structured light pattern information at the corresponding pixel location of the image under this light intensity level.

[0015] This technical solution can intelligently select the best unsaturated image for information extraction, ensuring that high-quality structured light pattern information can be obtained even in saturated areas, effectively filling data gaps.

[0016] Specifically, the identification of saturated pixel regions in each image due to localized highlight reflection includes: The brightness value of each pixel in each image is compared with a preset saturation brightness threshold. Pixels whose brightness value is greater than or equal to the saturation brightness threshold are marked as saturated pixels; Perform connected component analysis on the pixels of the saturated pixel; Connected regions with an area greater than a preset minimum area threshold are defined as the saturated pixel region.

[0017] This technical solution can accurately identify saturated pixel regions in an image, providing a precise basis for region segmentation in subsequent image fusion.

[0018] In practical applications, the exposure parameters for image acquisition can be automatically adjusted based on the current light intensity level, including: Establish a mapping table between the light intensity level and the exposure time in advance; Based on the light intensity level currently being projected, consult the mapping table to obtain the corresponding exposure time; Adjust the exposure time to the exposure time obtained from the query; In this mapping table, the light intensity level is negatively correlated with the exposure time.

[0019] This technical solution can automatically adjust exposure parameters according to different light intensity levels, ensuring that high-quality images can be captured under different lighting conditions and avoiding overexposure or underexposure.

[0020] Preferably, the calculation of the three-dimensional spatial coordinates of the surface of the component under test using the triangulation principle includes: Extracting the center line of the grating stripes for each pixel in the structured light information image after filtering out false feature point clouds; Calculate the phase value corresponding to each pixel based on the position of the center line of the grating stripe; Based on the phase value, the baseline distance between the structured light projector and the image acquisition device, and the calibrated system parameters, the depth value corresponding to each pixel is calculated using the triangulation formula. Based on the depth value and the intrinsic parameter matrix of the image acquisition device, calculate the three-dimensional spatial coordinates corresponding to each pixel.

[0021] This technical solution enables the accurate calculation of the three-dimensional spatial coordinates of the surface of the component under test based on precise structured light information maps and the principle of triangulation, thereby achieving high-precision three-dimensional reconstruction.

[0022] As a technological improvement, after the sliding window of a preset size scans pixel by pixel on the structured light information map, it also includes: The size of the sliding window is adaptively determined based on the period of the grating stripes in the structured light pattern: Extract the local spatial frequencies of the grating fringes in the structured light information image; Calculate the period length of the grating fringes based on the local spatial frequency; Set the side length of the sliding window to a fixed value between 1.5 and 2.5 times the length of the cycle.

[0023] This technical solution enables the sliding window size to be adaptively adjusted based on the local characteristics of the structured light pattern, making the identification and filtering of pseudo-feature point clouds more accurate and further improving the processing effect.

[0024] Secondly, this application also discloses a non-contact metrological detection system based on image recognition, comprising: The structured light projection module is used to project a structured light pattern onto the surface of the component under test, and to project the same structured light pattern sequentially using N different light intensity levels within a preset time window, where N is a positive integer greater than or equal to 3; An image acquisition module is used to synchronously acquire deformable structured light images of the surface of the component under test at each light intensity level. The image acquisition module includes an exposure control unit, which is used to automatically adjust the exposure parameters according to the current light intensity level to obtain an image containing unsaturated pixel areas. The saturation region recognition module is used to identify saturated pixel regions in each image caused by local highlight reflection; The fusion module is used to perform the following operations: For the identified saturated pixel region, select at least one image from the N images corresponding to the N light intensity levels that is not saturated at the corresponding pixel position, and extract the structured light pattern information of the selected image at the corresponding pixel position; For the identified unsaturated pixel regions, select the image with the highest light intensity level from the N images corresponding to the N light intensity levels, and extract the structured light pattern information of the image at the corresponding pixel position. The structured light pattern information extracted from the saturated pixel region and the structured light pattern information extracted from the unsaturated pixel region are spliced ​​and fused according to the original pixel spatial position to generate a complete structured light information map without saturated pixel regions. The pseudo-feature filtering module is used to perform local texture analysis on the structured light information map, identify and filter out pseudo-feature point clouds formed by multiple reflections of light on the surface of the component; The 3D reconstruction module is used to calculate the 3D spatial coordinates of the surface of the component under test based on the structured light information map after filtering out false feature point clouds and the principle of triangulation.

[0025] This system enables non-contact, high-precision metrological testing of highly reflective, complex curved, and concave structural components, effectively solving the problems of data gaps and pseudo-feature point clouds in existing technologies, and improving the reliability and efficiency of testing.

[0026] The technical solution of this application is not a simple mechanical combination of steps, but an organic whole in which the steps are functionally interconnected, parameters are coordinated, and data is transferred between each other. The synergistic relationship between the steps is as follows: (1) Synergy between multi-light intensity level projection (N≥3) and adaptive exposure adjustment: Multi-light intensity levels provide a differentiated brightness data basis for subsequent fusion, while adaptive exposure adjustment ensures that images containing unsaturated pixel areas can be acquired at each light intensity level. The two form a dual synergy mechanism of "hardware projection differentiation" and "dynamic response at the acquisition end". In contrast, the existing technology usually adopts a fixed exposure multiple exposure method, which cannot achieve real-time matching between "projection intensity" and "acquisition exposure".

[0027] (2) Synergistic fusion based on pixel-level brightness response curve optimization and highlight data hole filling: This application does not simply select one image randomly or according to fixed rules from all unsaturated images, but establishes a unique light intensity level-brightness response curve for each pixel location, and optimizes images whose brightness values ​​are in the center of the preset optimal response range. This pixel-level optimization mechanism, together with highlight region identification and stitching fusion, forms a closed data flow loop, which is significantly better than the fusion method of "direct selection by position or by time sequence" in the prior art.

[0028] (3) Synergy between the dual criteria of phase consistency and modulation degree for pseudo-feature identification and adaptive sliding window period: This application uses a "dual criterion" of phase consistency value and modulation degree value to identify pseudo-features (pixels with low phase consistency and high modulation degree are marked as pseudo-features), and adaptively adjusts the sliding window size according to the local spatial frequency of the grating fringes, so that each sliding window covers 1.5 to 2.5 fringe periods, ensuring the consistency of local texture analysis. Compared with the comparative method based on a single phase consistency criterion and fixed window size, this combination significantly improves the pseudo-feature filtering rate (from 78.4% to 97.6%) and reduces the false positive rate (from 6.7% to 1.1%).

[0029] (4) Synergy between anisotropic diffusion low-texture enhancement and pseudo-feature filtering: After generating the complete structured light information map, this application first performs anisotropic diffusion enhancement based on the structure tensor on the low-texture region, and then performs pseudo-feature point cloud filtering. The enhanced high-quality grating stripes provide a more reliable input for subsequent pseudo-feature recognition, and the two form a functional synergy of "front-end enhancement - back-end filtering".

[0030] (5) Overall synergistic effect of multi-step linkage: multi-light intensity level projection → adaptive exposure acquisition → pixel-level brightness response curve optimization and fusion → anisotropic diffusion low texture enhancement → pseudo feature filtering based on dual criteria → triangulation 3D reconstruction. This complete technical link forms a closed loop synergy in terms of function, data and parameters. The absence of any single link will lead to a significant decrease in overall performance. Attached Figure Description

[0031] Figure 1 This is a flowchart of a non-contact metrology and detection method based on image recognition provided in an embodiment of the present invention; Figure 2 This is a flowchart of a method for accurately aligning N images according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a non-contact metrology and detection system based on image recognition provided in an embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Reference Figure 1 , Figure 1 This is a flowchart of a non-contact metrology and detection method based on image recognition provided by an embodiment of the present invention, including the following steps: S11, Project a structured light pattern onto the surface of the component to be tested, and project the same structured light pattern using N different light intensity levels in sequence within a preset time window, where N is a positive integer greater than or equal to 3; S12, at each light intensity level, synchronously acquire deformed structured light images of the surface of the component under test, and automatically adjust the exposure parameters of image acquisition according to the current light intensity level to obtain images containing unsaturated pixel areas; S13, Identify saturated pixel regions in each image caused by localized highlight reflection; S14, For the identified saturated pixel region, select at least one image from the N images corresponding to the N light intensity levels that is not saturated at the corresponding pixel position, and extract the structured light pattern information of the selected image at the corresponding pixel position; S15, for the identified unsaturated pixel region, select the image with the highest light intensity level from the N images corresponding to the N light intensity levels, and extract the structured light pattern information of the image at the corresponding pixel position; S16, the structured light pattern information extracted from the saturated pixel region and the structured light pattern information extracted from the unsaturated pixel region are spliced ​​and fused according to the original pixel spatial position to generate a complete structured light information map without saturated pixel region. S17, perform local texture analysis on the structured light information map to identify and filter out false feature point clouds formed by multiple reflections of light on the surface of the component; S18. Based on the structured light information map after filtering out false feature point clouds, the three-dimensional spatial coordinates of the surface of the component under test are calculated using the principle of triangulation.

[0034] This application effectively solves the problem of high light saturation caused by highly reflective surfaces by combining multi-light intensity level projection with adaptive exposure adjustment, ensuring the integrity of the original structured light information. Simultaneously, by performing local texture analysis on the structured light information map and filtering out false feature point clouds, this application can eliminate false data caused by multiple light reflections, thereby significantly improving the accuracy and reliability of 3D reconstruction and providing a more accurate and efficient solution for non-contact metrological testing of high-precision components.

[0035] In order to better understand the technical solutions proposed in this application, it is necessary to explain some key terms involved therein.

[0036] "Structured light patterns" refer to optical patterns with specific spatial coding information, such as stripes, dot matrix, or coded patterns, which deform when projected onto the surface of an object due to the object's three-dimensional shape. By analyzing this deformation, the three-dimensional information of the object's surface can be deduced.

[0037] "Light intensity level" refers to the intensity of light emitted by a structured light projector at different brightness levels, which can usually be controlled by the projector's power or brightness settings.

[0038] "Deformation structured light image" refers to a structured light image captured by an image acquisition device after a structured light pattern is projected onto the surface of the component under test, showing the deformation caused by the surface morphology of the component.

[0039] "Saturated pixel area" refers to the pixel area in an image where the brightness value reaches or exceeds the maximum response range of the image sensor. It usually appears as an overly bright white area with lost detail, caused by localized highlight reflection.

[0040] "Unsaturated pixel areas" refer to pixel areas in an image whose brightness values ​​have not reached saturation, and can effectively reflect structured light pattern information.

[0041] "Structured light pattern information" refers to the coded information extracted from deformed structured light images for three-dimensional reconstruction, such as phase information and the position of the fringe center line.

[0042] "False feature point cloud" refers to erroneous 3D points that do not belong to the true surface of the component under test, caused by non-ideal optical phenomena such as multiple reflections of light during the 3D reconstruction process.

[0043] The "triangulation principle" is a method that calculates the three-dimensional coordinates of a target point by measuring the angles from two known points to the target point and combining this with the known baseline distance. It is widely used in the field of three-dimensional reconstruction.

[0044] The method proposed in this application first projects a structured light pattern onto the surface of the component under test. As one implementation, a digital light processing (DLP) projector can be used as the structured light projector, which can accurately project a preset structured light pattern. For example, a sinusoidal fringe pattern can be projected, which has good phase resolution characteristics. During the projection process, to address the challenge of highly reflective surfaces, this application sequentially projects the same structured light pattern using N different light intensity levels within a preset time window, where N is a positive integer greater than or equal to 3. For example, N can be set to 3, using low, medium, and high light intensity levels sequentially. Alternatively, N can be set to 5 to provide finer control over the light intensity.

[0045] At each light intensity level, deformable structured light images of the surface of the component under test are simultaneously acquired. The image acquisition device can be a high-resolution industrial camera, such as a CMOS camera, which has a high frame rate and dynamic range. To ensure that the acquired image contains unsaturated pixel areas, this application automatically adjusts the exposure parameters for image acquisition according to the current light intensity level. For example, an exposure parameter adjustment strategy can be preset, automatically reducing the exposure time when the light intensity level is high and automatically increasing the exposure time when the light intensity level is low. This adaptive adjustment ensures that unsaturated image information can be captured even in areas of strong reflection.

[0046] Next, saturated pixel regions in each image caused by localized highlight reflections are identified. This can be achieved by analyzing the brightness values ​​of pixels in the image. For example, a brightness threshold can be set, and pixels with brightness values ​​higher than this threshold can be marked as saturated pixels.

[0047] For the identified saturated pixel region, at least one image is selected from N images corresponding to N light intensity levels that is unsaturated at the corresponding pixel location. The structured light pattern information of the selected image at the corresponding pixel location is then extracted. For example, if a pixel location is saturated in the image at the highest light intensity level, it can be checked in the image at the next highest light intensity level to see if the pixel location is unsaturated. If it is still saturated, the search continues in images at lower light intensity levels until an unsaturated image is found. If images at multiple light intensity levels are all unsaturated at the pixel location, the image with moderate brightness and best contrast can be selected for extraction.

[0048] For the identified unsaturated pixel regions, the image with the highest light intensity level is selected from N images corresponding to N light intensity levels, and the structured light pattern information at the corresponding pixel location is extracted. This is because in unsaturated regions, the image with the highest light intensity level usually has the best signal-to-noise ratio and the clearest structured light pattern, which is beneficial for subsequent accurate calculations.

[0049] Next, the structured light pattern information extracted from the saturated pixel regions and the structured light pattern information extracted from the unsaturated pixel regions are stitched and fused according to their original pixel spatial positions to generate a complete structured light information map without saturated pixel regions. For example, image processing software can be used to perform pixel-level fusion of effective structured light information extracted under different light intensity levels to ensure that each pixel position contains effective, unsaturated structured light pattern information.

[0050] Local texture analysis is performed on the structured light infographic to identify and filter out spurious feature points formed by multiple reflections of light on the component surface. For example, algorithms based on local phase consistency or modulation can be used to evaluate the quality of the structured light pattern. In areas of multiple reflections, the structured light pattern may become blurred, distorted, or exhibit false stripes, leading to reduced phase consistency or abnormal modulation. By setting a threshold, these abnormal regions can be identified, and their corresponding 3D points can be marked as spurious feature points.

[0051] Finally, based on the structured light information map after filtering out false feature point clouds, the three-dimensional spatial coordinates of the surface of the component under test are calculated using the principle of triangulation. For example, the center line or phase information of the grating fringes can be extracted from the structured light information map first. Then, combined with pre-calibrated system parameters (such as camera intrinsic and extrinsic parameters, projector parameters, and the baseline distance between the camera and the projector), the depth value corresponding to each pixel is calculated using the triangulation formula. Finally, based on the depth value and the camera intrinsic parameter matrix, each pixel is converted into three-dimensional spatial coordinates, thereby reconstructing a complete three-dimensional model of the component under test.

[0052] The non-contact metrological inspection method based on image recognition proposed in this application aims to solve the problems of high light saturation and multiple reflection pseudo-features faced by highly reflective and complex curved surface components in three-dimensional measurement. Traditional methods often struggle to address both challenges simultaneously, resulting in "data gaps" and "pseudo-features" in the measurement data, which seriously affect the measurement accuracy and reliability.

[0053] This application addresses the high-light saturation problem at its source by introducing multi-light intensity level projection and an adaptive exposure adjustment mechanism. When structured light is projected onto a highly reflective surface, the intensity of reflected light received by the camera may be extremely high, leading to localized overexposure of the image. By sequentially projecting the same structured light pattern using N different light intensity levels within a preset time window, and automatically adjusting the exposure parameters for image acquisition based on the current light intensity at each light intensity level, it can be ensured that images containing unsaturated pixels can be acquired in different reflective intensity regions. Specifically, for highly reflective areas, even if saturation may occur at the highest light intensity, one or more unsaturated images can always be found by reducing the light intensity level or adjusting the exposure parameters. For unsaturated areas, the image with the highest light intensity level is selected first to obtain the best signal-to-noise ratio. Subsequently, the effective structured light pattern information extracted from these different light intensity levels is stitched and fused to generate a complete structured light information map without saturated pixel areas, effectively filling the data gaps caused by highlights in traditional methods.

[0054] Furthermore, to address the issue of spurious features caused by multiple reflections due to complex curved surfaces and concave structures, this application performs local texture analysis on the generated structured light infographic. By identifying and filtering out spurious feature point clouds formed by multiple light reflections, this application can effectively eliminate the interference of false data on 3D reconstruction. For example, in areas of multiple reflections, the local texture of the structured light pattern may exhibit anomalies, such as reduced phase consistency or abnormal modulation. By setting appropriate thresholds, these spurious feature points can be accurately identified and removed or corrected from the final 3D point cloud, thereby avoiding erroneous surface points in the reconstructed model.

[0055] Compared with existing technologies, the core innovation of this application lies in its comprehensive solution, which can effectively address both the challenges of high-saturation highlights and multiple reflections. Traditional methods often only solve one of the problems, or require additional hardware (such as increasing the number of cameras and light sources) or auxiliary processes (such as spraying developer). These methods not only increase cost and complexity but may also introduce new errors or contamination. This application, through software algorithm optimization, achieves effective suppression and correction of high-saturation highlights and multiple reflections without changing the surface state of the component, significantly increasing hardware complexity, or reducing detection efficiency. This method not only improves the accuracy and reliability of 3D reconstruction but also makes the application of non-contact metrology and inspection systems more widespread and efficient in the field of high-precision manufacturing.

[0056] In some embodiments described above, a complete structured light information map without saturated pixel regions can be generated by stitching and fusing the structured light pattern information extracted from saturated pixel regions with that extracted from unsaturated pixel regions according to their original pixel spatial positions. However, in practical applications, due to factors such as minute movements of the component under test, jitter of the image acquisition device, or optical distortion, there may be slight pixel-level or sub-pixel-level offsets between N images acquired at different light intensity levels. If the alignment problem between these images is not addressed, direct stitching and fusion may result in ghosting, blurring, or misalignment of the structured light pattern information, thereby affecting the accuracy and reliability of the final 3D reconstruction.

[0057] In this regard, refer to Figure 2 , Figure 2 This is a flowchart of a method for accurately aligning N images according to an embodiment of the present invention. S16 includes: S161, extract feature points with local contrast changes from the N images; S162, Calculate the initial offset between the N images based on the feature points; S163, Perform preliminary resampling and alignment on the N images based on the initial offset; S164, evaluate the local signal-to-noise ratio of the structured light pattern in the image after initial resampling and alignment to determine the reliability of each pixel location; S165, based on the reliability, the sub-pixel displacement field between the N images is iteratively calculated using the optical flow method or the cross-correlation method; S166, Based on the sub-pixel level displacement field, perform fine calibration and resampling on the N images until the displacement field change calculated in two adjacent iterations is less than a preset threshold.

[0058] Specifically, extracting feature points with local contrast variations from the N images means identifying pixels with significant texture or structural changes in each image, such as corner points, edge points, or scale-invariant feature transform (SIFT) feature points. These feature points, due to their uniqueness and repeatability within the images, can serve as a reliable basis for matching and alignment between different images. Local contrast variation can be understood as the degree of brightness or color change in the area surrounding a pixel, reflecting the richness of texture in that area.

[0059] Furthermore, based on the feature points, the initial offset between the N images is calculated. This involves establishing a correspondence between images at different light intensity levels using a feature point matching algorithm (e.g., brute-force matching based on descriptor matching or FLANN matching), and using these matching point pairs to estimate the initial geometric transformation parameters between the images, such as translation, rotation, and scaling. Thus, a preliminary alignment relationship between the images can be obtained.

[0060] Based on this, preliminary resampling and alignment of the N images is performed according to the initial offset. This means using the estimated initial geometric transformation parameters, the N images are spatially transformed through image resampling techniques (e.g., bilinear interpolation or bicubic interpolation) to achieve approximate alignment at the pixel level. This step aims to eliminate large global misalignments between images.

[0061] Subsequently, the local signal-to-noise ratio (SNR) of the structured light pattern in the initially resampled and aligned image is evaluated to determine the reliability of each pixel location. The local SNR reflects the sharpness and quality of the structured light pattern in a local region of the image; it can be measured, for example, by calculating the contrast, variance, or energy of the local region. A higher SNR indicates more reliable structured light pattern information at that pixel location. Determining the reliability of each pixel location provides weights or guidance for subsequent fine-tuning.

[0062] Furthermore, based on the aforementioned reliability, the sub-pixel displacement field between the N images is iteratively calculated using either optical flow or cross-correlation methods. Optical flow is a method for estimating object motion based on pixel brightness variations in an image sequence, while cross-correlation determines displacement by calculating the similarity between image patches. Both methods can achieve sub-pixel displacement estimation, thereby capturing finer local deformations or minute shifts between images. The purpose of iterative calculation is to progressively optimize the displacement field, converging it to a more accurate alignment state. During the calculation process, different weights can be assigned to the displacement estimates of different regions based on the previously determined pixel reliability to improve the robustness of alignment.

[0063] Finally, based on the sub-pixel displacement field, the N images are finely calibrated and resampled until the change in displacement field between two adjacent iterations is less than a preset threshold. This step uses the sub-pixel displacement field obtained through iterative calculation to perform final, high-precision resampling of the images to achieve accurate alignment between them. The preset threshold is used to determine whether the iteration process has converged. When the change in displacement field is sufficiently small, indicating that the images have reached satisfactory alignment accuracy, the iteration process stops.

[0064] This application's solution introduces a multi-stage image alignment strategy. First, it uses feature points for preliminary global alignment, effectively correcting potentially large geometric deviations between images. Then, it obtains pixel-level reliability information by evaluating the local signal-to-noise ratio, providing guidance for subsequent fine-grained alignment. Based on this, iterative sub-pixel-level displacement field calculations using optical flow or cross-correlation methods accurately capture and compensate for minute local deformations and sub-pixel-level offsets between images. Finally, through fine-calibration resampling, it ensures highly accurate alignment at the pixel level for N images acquired under different light intensity levels. Therefore, in the subsequent stitching and fusion process, distortion of structured light pattern information caused by image misalignment can be avoided, thus guaranteeing the accuracy of the fusion result.

[0065] The above technical solution significantly improves the alignment accuracy between N images acquired under different light intensity levels, effectively avoiding ghosting, blurring, or misalignment problems during structured light pattern information fusion caused by image misalignment. Consequently, the generated structured light information map with complete, unsaturated pixel regions has higher quality and accuracy, providing a more reliable data foundation for subsequent 3D reconstruction, thereby enhancing the overall accuracy and robustness of non-contact metrology and detection.

[0066] In some embodiments described above, local texture analysis is performed on the structured light infographic to identify and filter out spurious feature point clouds formed by multiple reflections of light on the component surface. However, in practical applications, especially when inspecting components with complex geometries or high reflectivity, light may reflect multiple times on the component surface, resulting in false grating stripes or distorted textures in the acquired structured light image. These false or distorted textures are incorrectly identified as real feature points in subsequent 3D reconstruction, severely affecting the accuracy of the final 3D measurement results. Without a robust and accurate method to distinguish and remove these spurious feature points, the reconstructed 3D model will contain significant noise and errors, failing to meet the requirements of high-precision metrology and inspection.

[0067] In response, this application further proposes a method for identifying and filtering out false feature point clouds formed by multiple reflections of light on the surface of a component, which includes: A sliding window of a preset size is used to scan the structured light information map pixel by pixel; Within each sliding window, calculate the phase coherence value and modulation value of the grating fringes; If the phase consistency value is lower than the first preset threshold and the modulation value is higher than the second preset threshold, then the spatial point corresponding to the center pixel of its sliding window is marked as a pseudo feature point. The set of all spatial points marked as pseudo-feature points is defined as the pseudo-feature point cloud; Each pseudo-feature point in the pseudo-feature point cloud is replaced with the average coordinates of the unlabeled spatial points in its neighborhood to filter out the pseudo-feature point cloud.

[0068] Specifically, scanning the structured light infographic pixel by pixel using a sliding window of a preset size means traversing the entire structured light infographic through a fixed-size local region (i.e., the sliding window) to analyze the local texture information of each pixel and its surroundings. The size of the sliding window is preset to ensure that it can cover a sufficient period of grating stripes, thereby effectively capturing local texture features.

[0069] Within each sliding window, the phase coherence and modulation values ​​of the grating fringes are calculated. The phase coherence value measures the degree of structure and directional stability of the grating fringes within a local region; true structured light fringes typically exhibit high phase coherence. The modulation value reflects the contrast or intensity variation of the grating fringes; spurious feature points often exhibit abnormally high local brightness or contrast due to multiple reflections.

[0070] In practical applications, if the phase consistency value is lower than a first preset threshold and the modulation value is higher than a second preset threshold, the spatial point corresponding to the center pixel of the sliding window is marked as a pseudo-feature point. This judgment condition is based on the understanding of the physical characteristics of pseudo-feature points: pseudo-feature points formed by multiple reflections usually exhibit local texture chaos and phase inconsistency, but due to potentially high reflection intensity, their modulation value may be abnormally high. By combining these two indicators, abnormal points caused by multiple reflections can be identified more accurately. The first and second preset thresholds are determined based on experience or experimental data and are used to distinguish between real features and pseudo-features.

[0071] Furthermore, the set of all spatial points marked as pseudo-feature points is defined as the pseudo-feature point cloud. This means that the three-dimensional spatial coordinates corresponding to all pixels that meet the above conditions will be classified as pseudo-features that need to be processed.

[0072] Finally, each pseudo-feature point in the pseudo-feature point cloud is replaced with the average coordinates of its neighboring unlabeled spatial points to filter out the pseudo-feature point cloud. This replacement strategy aims to smoothly remove pseudo-feature points while using the geometric information of the surrounding real points for interpolation, thereby removing noise while maximizing the preservation of the geometric continuity and integrity of the original part surface.

[0073] This application's solution, by introducing phase consistency and modulation values ​​as discrimination criteria, effectively addresses the challenges faced by traditional methods in distinguishing between genuine structured light fringes and spurious feature points generated by multiple reflections. Through this technical solution, the robustness and accuracy of structured light-based 3D metrology and inspection are significantly improved. Specifically, by combining phase consistency and modulation as dual criteria, spurious feature points caused by complex reflection environments (such as high-reflectivity surfaces or complex geometries) can be effectively distinguished and identified, avoiding interference from these spurious feature points in the 3D reconstruction results. Furthermore, the method of replacing spurious feature points with neighborhood averages not only smoothly removes noise but also preserves the true geometric details of the measured component's surface to the greatest extent possible, avoiding potential local data loss or surface discontinuities caused by simply eliminating spurious feature points. Therefore, this application's solution ensures higher accuracy and reliability of the obtained 3D point cloud data under various complex working conditions, thus meeting the needs of industrial-grade high-precision metrology and inspection.

[0074] To address this, this application proposes a scheme that, after generating a complete structured light information map with no saturated pixel regions, includes the following steps: Calculate the full-image brightness histogram of the structured light information map; Based on the full-image brightness histogram, identify low-texture regions with contrast below a preset contrast threshold; For the low-texture region, an anisotropic diffusion filtering method based on the structure tensor is used to enhance the contrast of the grating stripes in the region. Phase unwrapping calculations are performed on the contrast-enhanced grating fringes to obtain a continuous absolute phase distribution.

[0075] Specifically, calculating the full-image brightness histogram of the structured light information map involves statistically analyzing the pixel brightness values ​​of the entire structured light information map to obtain its brightness distribution characteristics. This histogram can reflect the overall brightness and contrast information of the image, providing a data foundation for subsequent identification of low-texture areas.

[0076] Specifically, based on the overall image brightness histogram, low-texture regions with contrast below a preset contrast threshold are identified. This can be understood as determining which areas in the image have indistinct raster stripes (i.e., low contrast) by analyzing the distribution characteristics of the histogram or combining it with local brightness variations. The preset contrast threshold is an empirical value used to distinguish between high-contrast and low-contrast regions. Its purpose is to accurately locate those weak-texture regions that may affect subsequent phase calculations.

[0077] In practical applications, for the low-texture region, an anisotropic diffusion filtering method based on the structure tensor is used to enhance the contrast of grating fringes within the region. This refers to an algorithm that can adaptively adjust the filtering direction and intensity according to the local structure of the image. The structure tensor describes the directionality and intensity of the local texture of an image, while anisotropic diffusion filtering can smooth the interior of the region while preserving edge information, thereby effectively enhancing the visibility of low-contrast grating fringes and avoiding blurring important structural features. The aim is to improve the image quality of low-texture regions, laying the foundation for accurately extracting structured light pattern information.

[0078] Furthermore, phase unwrapping calculations are performed on the contrast-enhanced grating fringes to obtain a continuous absolute phase distribution. This means that after the grating fringes contrast is effectively enhanced, techniques such as Phase Profilometry (PMP) are used to calculate the phase value of each pixel in the image and eliminate phase blur (i.e., 2π jumps), thereby obtaining a continuous and unique absolute phase distribution map. This absolute phase distribution is crucial intermediate data for subsequent calculation of three-dimensional spatial coordinates using triangulation principles.

[0079] The solution presented in this application effectively addresses the issue of insufficient phase calculation accuracy in low-texture regions in traditional methods because, after generating a complete structured light information map, it does not directly perform 3D reconstruction. Instead, it first analyzes the overall brightness distribution of the image and identifies regions with low contrast and indistinct texture information. It is precisely the presence of these low-texture regions that makes subsequent phase calculations prone to introducing errors. By employing an anisotropic diffusion filtering method based on the structure tensor, targeted local contrast enhancement can be applied to these weakly textured regions, making the originally blurry grating fringes clearly discernible without disrupting the overall image structure. This local enhancement effectively improves the signal-to-noise ratio of the grating fringes, thus providing high-quality input data for subsequent phase unwrapping calculations and ensuring the continuity and accuracy of the obtained absolute phase distribution.

[0080] Through the above technical solution, this application effectively solves the problem of insufficient contrast in local low-texture areas caused by the complexity of the surface material or shape of the component under test in non-contact metrology and testing. By performing refined local processing on the structured light information map, especially by specifically enhancing the contrast of the grating stripes in low-texture areas, the phase calculation accuracy in these areas is significantly improved. This makes the final obtained three-dimensional spatial coordinates more accurate and reliable, especially when measuring challenging surfaces, effectively avoiding measurement errors caused by missing or blurred local information, thereby improving the robustness and accuracy of the entire metrology and testing method.

[0081] This application further proposes an optimized image selection method, wherein for the identified saturated pixel region, at least one image with unsaturated pixel position is selected from N images corresponding to the N light intensity levels, including: Establish the light intensity level and brightness response curve for each pixel location; Based on the light intensity levels and brightness response curves, determine the set of unsaturated light intensity levels corresponding to each pixel location; If the set of unsaturated light intensity levels contains multiple light intensity levels, then select a light intensity level whose corresponding brightness value is in the middle of the preset optimal response range; Extract the structured light pattern information of the image at the corresponding pixel position under the light intensity level.

[0082] Specifically, establishing the light intensity level and brightness response curve for each pixel location refers to recording the brightness value of each pixel at different light intensity levels when projecting structured light patterns of different light intensity levels onto the surface of the component under test and acquiring images. Using these data points, a curve reflecting the brightness response characteristics of that pixel can be constructed, describing the law of pixel brightness change with the intensity of the projected light. Determining the set of unsaturated light intensity levels corresponding to each pixel location means identifying, based on the established brightness response curve, all light intensity levels that prevent the brightness value of that pixel from reaching the saturation threshold. The images corresponding to these light intensity levels can all be used to extract the structured light pattern information for that pixel.

[0083] Furthermore, if the set of unsaturated light intensity levels contains multiple light intensity levels, a light intensity level whose corresponding brightness value is located in the center of a preset optimal response range is selected. This preset optimal response range is typically set to the brightness range within which the image sensor can provide the best linear response and the highest signal-to-noise ratio. Selecting a brightness value in the center of this range aims to avoid selecting images with excessively low brightness (poor signal-to-noise ratio) or excessively high brightness (close to saturation, potentially resulting in nonlinearity or loss of detail), thereby ensuring that the selected image has optimal structured light pattern information quality at that pixel location. Finally, structured light pattern information, such as phase information or intensity modulation information, is extracted from the image corresponding to the selected light intensity level at that pixel location.

[0084] This application's solution establishes a unique light intensity level and brightness response curve for each pixel, enabling precise understanding of the pixel's response characteristics under different lighting conditions. Therefore, when processing saturated pixel regions caused by localized high-brightness reflection, the system no longer simply selects randomly from all unsaturated images. Instead, it systematically determines which light intensity levels provide unsaturated, high-quality image data based on pixel-level brightness response curves. In particular, when multiple unsaturated light intensity levels are available, selecting the light intensity level with a brightness value in the middle of a preset optimal response range effectively avoids selecting images with excessively low brightness (leading to poor signal-to-noise ratio) or excessively high brightness (potentially approaching saturation, resulting in nonlinear response or loss of detail). This refined selection mechanism ensures that the structured light pattern information extracted from saturated regions has optimal signal quality and reliability.

[0085] Through the above technical solution, this application can achieve more refined and intelligent image selection for saturated pixel areas caused by localized high-brightness reflection. By establishing pixel-level brightness response curves and optimizing the light intensity level with brightness values ​​in the center of the optimal response range, the problem of degraded structured light pattern information quality caused by random or suboptimal selection is effectively avoided. This ensures that the structured light pattern information extracted from saturated areas has a higher signal-to-noise ratio and accuracy, significantly improving the overall quality and reliability of the final stitched and fused structured light information map. This provides a more accurate data foundation for subsequent 3D reconstruction and improves the accuracy and robustness of non-contact metrology and detection.

[0086] Specifically, the steps for identifying saturated pixel regions in each image due to localized highlight reflection include: The brightness value of each pixel in each image is compared with a preset saturation brightness threshold. Pixels whose brightness value is greater than or equal to the saturation brightness threshold are marked as saturated pixels; Perform connected component analysis on the pixels of the saturated pixel; Connected regions with an area greater than a preset minimum area threshold are defined as the saturated pixel regions.

[0087] Specifically, after the image acquisition device obtains a deformable structured light image of the surface of the component under test, the brightness value of each pixel in the image is first detected. The brightness value refers to the light intensity information reflected by the pixel, usually expressed as grayscale value or RGB component value. In order to identify saturated areas caused by localized high-brightness reflection, a preset saturation brightness threshold needs to be set. This threshold is usually set empirically based on the dynamic range of the image acquisition device and the actual application scenario, and is used to define the degree to which the pixel brightness reaches or approaches the maximum response of the sensor.

[0088] Pixels whose brightness value is greater than or equal to the saturation brightness threshold are marked as saturated pixels. This means that when the brightness value of a pixel reaches or exceeds the preset saturation brightness threshold, the pixel is considered to be saturated, and the light intensity information it records may be distorted or unable to accurately reflect the actual reflected light intensity.

[0089] Furthermore, connected component analysis is performed on the pixels of the saturated pixels. Connected component analysis is an image processing technique used to identify sets of interconnected pixels in an image. In this step, all pixels marked as saturated pixels are grouped to form one or more connected regions. Through connected component analysis, scattered saturated pixels can be aggregated into regions with a certain degree of spatial continuity, thereby better characterizing the range of specular reflection.

[0090] Therefore, connected regions with an area greater than a preset minimum area threshold are defined as the saturated pixel regions. In practical applications, a single or a few saturated pixels may be caused by noise rather than genuine specular reflection areas. Therefore, setting a minimum area threshold can effectively filter out these isolated noise points, ensuring that the identified saturated pixel regions are genuine specular reflection areas of a certain scale and physical significance. This minimum area threshold can be adjusted according to the actual image resolution and the typical size of specular reflections.

[0091] This application's scheme initially filters out potentially saturated pixels by comparing their brightness values ​​pixel-by-pixel with a saturation brightness threshold. Subsequently, connected component analysis is used to aggregate these discrete saturated pixels into spatially continuous regions, thus more accurately depicting the range of specular reflection. Finally, by setting a minimum area threshold, sporadic saturation points caused by noise are effectively eliminated, ensuring that the identified saturated pixel regions are real and meaningful specular reflection areas. This step-by-step processing method makes the identification of saturated pixel regions more accurate and robust.

[0092] The above technical solution can accurately identify saturated pixel regions in an image caused by localized high-brightness reflection. This refined identification method avoids misclassifying noise points as saturated regions, improving the accuracy of subsequent structured light pattern information extraction. Effectively defining saturated regions provides a reliable basis for selecting unsaturated images from multiple images with different light intensity levels, thus ensuring the integrity and unsaturated nature of the final stitched and fused structured light information map, significantly improving the accuracy and reliability of non-contact metrology and detection.

[0093] In some embodiments described above, an automatic adjustment of the exposure parameters for image acquisition is proposed based on the current light intensity level to obtain an image containing unsaturated pixel regions. However, without a specific and efficient exposure parameter adjustment strategy, this automatic adjustment process may not always ensure image quality. For example, when light intensity changes drastically, local oversaturation or overall underexposure may still occur, thus affecting the accurate extraction of subsequent structured light pattern information.

[0094] In response, this application further proposes a method for automatically adjusting the exposure parameters of image acquisition based on the current light intensity level, specifically including: Establish a mapping table between light intensity levels and exposure time in advance; Based on the light intensity level currently being projected, the corresponding exposure time is obtained by querying the mapping table. Adjust the exposure time to the exposure time obtained from the query; In the mapping table, the light intensity level is negatively correlated with the exposure time.

[0095] Specifically, a mapping table between light intensity levels and exposure time is pre-established. This table aims to provide the optimal image acquisition exposure time for different structured light projection intensities, ensuring the capture of high-quality deformable structured light images under various lighting conditions. In practical applications, this mapping table can be obtained through experimental calibration or empirical data accumulation, with the purpose of providing an optimized exposure strategy for image acquisition equipment.

[0096] When the system projects a structured light pattern at a specific light intensity level, it queries a pre-established mapping table based on the current light intensity level to obtain the exposure time corresponding to that light intensity level. This query process is typically executed automatically by the control unit of the image acquisition device to achieve rapid response.

[0097] Subsequently, the exposure parameters of the image acquisition device will be adjusted to the obtained exposure time. This adjustment process ensures that the image acquisition device can dynamically optimize its exposure settings according to the actual lighting conditions and the intensity of the structured light projection, thereby effectively controlling the brightness and contrast of the image.

[0098] Furthermore, in the mapping table, light intensity level and exposure time are negatively correlated. This means that when the projected light intensity level is high, the exposure time for image acquisition will be shortened accordingly; conversely, when the light intensity level is low, the exposure time will be extended. This negative correlation is designed based on the response characteristics of the image sensor under different light intensities, and its purpose is to avoid image oversaturation or underexposure, thereby effectively acquiring images containing unsaturated pixel areas.

[0099] This application's solution establishes a pre-defined mapping table between light intensity levels and exposure time, ensuring a negative correlation, thereby achieving intelligent and adaptive adjustment of image acquisition exposure parameters. When the structured light projection module projects structured light patterns at different light intensity levels, the image acquisition module can quickly and accurately adjust the exposure time according to the preset mapping relationship. This mechanism ensures that the exposure time is shortened under strong light to avoid image sensor saturation, while the exposure time is extended under weak light to guarantee sufficient brightness information. This effectively avoids overexposure or underexposure problems that may occur in traditional methods, ensuring that deformed structured light images containing rich and unsaturated pixel areas can be acquired at each light intensity level.

[0100] Through the above technical solution, this application can significantly improve the quality and efficiency of image acquisition. By using a preset mapping table and negative correlation characteristics, the system can precisely adjust the exposure time for different light intensity levels, effectively avoiding common oversaturation or underexposure problems during image acquisition. This not only ensures the acquisition of high-quality images containing unsaturated pixel areas at all light intensity levels, but also provides a reliable data foundation for subsequent structured light pattern information extraction, stitching and fusion, and 3D reconstruction, thereby improving the overall accuracy and robustness of the non-contact metrology and detection method.

[0101] In some embodiments described above in this application, the three-dimensional spatial coordinates of the surface of the component under test are calculated using the principle of triangulation. Specifically, the steps for calculating the three-dimensional spatial coordinates of the surface of the component under test using the principle of triangulation include: Extracting the center line of the grating stripes for each pixel in the structured light information image after filtering out false feature point clouds; Calculate the phase value corresponding to each pixel based on the position of the center line of the grating stripes; Based on the phase value, the baseline distance between the structured light projector and the image acquisition device, and the calibrated system parameters, the depth value corresponding to each pixel is calculated using the triangulation formula. Based on the depth value and the intrinsic parameter matrix of the image acquisition device, the three-dimensional spatial coordinates corresponding to each pixel are calculated.

[0102] Specifically, after obtaining a complete structured light information image with unsaturated pixel regions and filtered-out pseudo-feature point clouds, further processing is required to extract 3D information. First, image processing algorithms, such as those based on skeleton extraction, thinning, or edge detection, can accurately identify and extract the center lines of the grating stripes corresponding to each pixel in the structured light information image. These center lines represent the projection trajectory of the structured light pattern onto the surface of the component under test.

[0103] Furthermore, once the center lines of the grating fringes are determined, the phase value corresponding to each pixel can be calculated based on the position of these center lines. The phase value is key information in structured light measurement, reflecting the spatial position of the grating fringes, and is typically extracted from the deformed grating fringes using phase demodulation algorithms (such as Fourier transform profilometry, wavelet transform, etc.).

[0104] Based on this, and combined with known system parameters, including the baseline distance between the structured light projector and the image acquisition device, as well as the pre-calibrated intrinsic and extrinsic system parameters, the depth value corresponding to each pixel can be calculated using triangulation formulas. The principle of triangulation is based on geometric relationships, determining the depth information of an object's surface by measuring the difference between the light projection angle and the image acquisition angle.

[0105] Finally, based on the calculated depth value of each pixel and the intrinsic parameter matrix of the image acquisition device, the specific coordinates of each pixel in three-dimensional space can be further calculated. The intrinsic parameter matrix of the image acquisition device contains information such as the camera's focal length and principal point coordinates; these parameters are crucial for converting two-dimensional image coordinates into three-dimensional spatial coordinates.

[0106] This application's solution ensures accurate conversion from 2D image information to 3D spatial coordinates by refining the calculation process of 3D spatial coordinates into a series of logically clear steps. First, accurately extracting the center line of the grating fringes is fundamental to subsequent phase calculations, and its accuracy directly affects the final 3D reconstruction precision. Second, by calculating phase values, spatial location information is encoded into continuous phase data, providing reliable input for subsequent depth calculations. Third, using triangulation formulas, combined with calibration parameters and baseline distance, phase information can be effectively converted into physical depth values—a crucial step in the 2D to 3D conversion. Finally, by combining depth values ​​and the camera intrinsic parameter matrix, a 3D coordinate transformation from the camera coordinate system to the world coordinate system is achieved, thus completely reconstructing the 3D topography of the surface of the measured component.

[0107] Through the above technical solution, this application provides a detailed and operable three-dimensional spatial coordinate calculation process, ensuring that the non-contact metrological detection method based on structured light information can accurately and reliably acquire the three-dimensional morphological data of the surface of the component under test. This solution effectively avoids the errors and uncertainties that may arise from directly calculating the three-dimensional coordinates from complex images in one step by extracting key information (such as the center line of the grating fringes and phase values) and combining it with a precise geometric model (triangulation principle), significantly improving the accuracy and stability of three-dimensional reconstruction.

[0108] In some embodiments of this application, when identifying and filtering out pseudo-feature point clouds formed by multiple reflections of light on the surface of a component, a sliding window of a preset size is used to scan the structured light infographic pixel by pixel. However, this preset-size sliding window may not optimally adapt to the local variations in the grating fringe period in different regions of the structured light pattern, thus affecting the identification and filtering effect of pseudo-feature points. For example, when the grating fringe period is small, the fixed-size window may be too large, introducing unnecessary noise; while when the grating fringe period is large, the fixed-size window may be too small, failing to capture sufficient fringe information.

[0109] To address this, this application further proposes an optimization scheme aimed at improving the accuracy and robustness of pseudo-feature point cloud recognition by adaptively adjusting the size of the sliding window. After the aforementioned sliding window of a preset size scans pixel-by-pixel on the structured light information map, it further includes: The size of the sliding window is adaptively determined based on the period of the grating stripes in the structured light pattern: Extract the local spatial frequencies of the grating fringes in the structured light information image; Calculate the period length of the grating fringes based on the local spatial frequency; The side length of the sliding window is set to a fixed value between 1.5 and 2.5 times the period length.

[0110] Specifically, when performing local texture analysis on structured light infographics, the size of the sliding window is designed to adaptively adjust according to the period of the grating fringes in the structured light pattern. The first step is to extract the local spatial frequencies of the grating fringes in the structured light infographic. Local spatial frequency refers to the frequency at which grating fringes repeat within a specific region of the image, reflecting the density of the fringes. This can be obtained, for example, through frequency analysis within a local region using methods such as Fourier transform, wavelet transform, or Gabor filters.

[0111] Furthermore, the period length of the grating fringes can be calculated based on the extracted local spatial frequency. The period length is the reciprocal of the spatial frequency and represents the pixel distance of a complete grating fringe period in the image space. For example, if the local spatial frequency is f, then the period length T = 1 / f.

[0112] Based on this, the side length of the sliding window is set to a fixed value between 1.5 and 2.5 times the period length. This range is chosen to ensure that the sliding window is large enough to completely encompass at least one or two complete grating fringe periods, thus providing sufficient local texture information for calculating the phase coherence and modulation values, while avoiding excessively large windows that could reduce computational efficiency or smooth out important local details. For example, it can be set to twice the period length to balance computational accuracy and efficiency.

[0113] This application's solution effectively addresses the limitations of fixed-size windows when processing grating fringes with local variations by introducing an adaptive determination mechanism for the sliding window size. Through this technical solution, the accuracy and adaptability of pseudo-feature point cloud identification and filtering are significantly improved. Because the size of the sliding window can adaptively adjust according to the local periodicity of the grating fringes in the structured light pattern, undersampling or oversampling problems that may occur with a fixed window size are avoided. This allows for more reliable local texture information in regions with varying texture density and curvature, thus more accurately identifying pseudo-feature points formed by multiple reflections of light. Ultimately, this improves the accuracy and reliability of the 3D reconstruction results, especially when processing complex surfaces or test components with significant depth variations, where its advantages are even more pronounced.

[0114] In this regard, refer to Figure 3 , Figure 3 This is a schematic diagram of a non-contact metrology and detection system based on image recognition provided in an embodiment of the present invention, comprising: The structured light projection module is used to project a structured light pattern onto the surface of the component under test, and to project the same structured light pattern sequentially using N different light intensity levels within a preset time window, where N is a positive integer greater than or equal to 3; An image acquisition module is used to synchronously acquire deformable structured light images of the surface of the component under test at each light intensity level. The image acquisition module includes an exposure control unit, which is used to automatically adjust the exposure parameters according to the current light intensity level to obtain an image containing unsaturated pixel areas. The saturation region recognition module is used to identify saturated pixel regions in each image caused by local highlight reflection; The fusion module is used to perform the following operations: For the identified saturated pixel region, select at least one image from the N images corresponding to the N light intensity levels that is not saturated at the corresponding pixel position, and extract the structured light pattern information of the selected image at the corresponding pixel position; For the identified unsaturated pixel regions, the image with the highest light intensity level is selected from the N images corresponding to the N light intensity levels, and the structured light pattern information of the image at the corresponding pixel position is extracted. The structured light pattern information extracted from the saturated pixel region and the structured light pattern information extracted from the unsaturated pixel region are spliced ​​and fused according to the original pixel spatial position to generate a complete structured light information map without saturated pixel region. The pseudo-feature filtering module is used to perform local texture analysis on the structured light information map to identify and filter out pseudo-feature point clouds formed by multiple reflections of light on the surface of the component. The 3D reconstruction module is used to calculate the 3D spatial coordinates of the surface of the component under test based on the structured light information map after filtering out false feature point clouds and the principle of triangulation.

[0115] The non-contact metrology and inspection system based on image recognition proposed in this application aims to solve the problems of high light saturation and multiple reflection pseudo-features faced by highly reflective and complex curved surface components in 3D measurement through the coordinated work of its various functional modules. The system utilizes multi-ray intensity level projection by a structured light projection module, combined with adaptive exposure adjustment by an image acquisition module and its exposure control unit, to ensure, from both hardware and software perspectives, that image information containing unsaturated pixels can be acquired in regions with different reflection intensities. Subsequently, a saturation region identification module, a fusion module, a pseudo-feature filtering module, and a 3D reconstruction module process the acquired image data sequentially, achieving effective information extraction from highly saturated regions, accurate identification and correction of multiple reflection pseudo-features, and ultimately generating a high-precision 3D point cloud model free of data holes and pseudo-features. This systematic solution significantly improves the accuracy and reliability of non-contact metrology and inspection, providing strong technical support for the quality inspection of high-precision components.

[0116] This embodiment is used to verify the improvement effect of the proposed method on high-saturation pixel rate. The test object is a group of mobile phone metal casing parts with specular reflection characteristics. Structured light projection is performed using N=5 light intensity levels, and adaptive exposure and pixel-level optimization fusion are performed according to the proposed method. The experimental results show that the high-saturation pixel rate using the conventional single-criteria method is 13.5%, while the high-saturation pixel rate using the proposed method is reduced to 0.9%, an improvement of 93.2%.

[0117] This embodiment is used to verify the improvement effect of the proposed method in data hole filling. The test objects are a group of stainless steel stamping parts with complex concave structures, and experiments are conducted under three typical lighting environments. The experimental results show that the data hole filling rate of conventional methods is 77.4%~82.6%, while the data hole filling rate of the proposed method is improved to 99.1%, an improvement of about 16.7 percentage points, verifying the effectiveness of the optimized fusion mechanism based on pixel-level brightness response curves for data hole filling.

[0118] This embodiment is used to verify the improvement effect of the proposed method in filtering out pseudo-feature point clouds. The test objects are a group of precision castings with large surface curvature variations. The experimental results show that the conventional method based on a single phase consistency criterion has a pseudo-feature filtering rate of 78.4% and a misclassification rate of 6.7%; while after adopting the proposed dual criterion of "phase consistency + modulation" and the sliding window size adaptive mechanism, the pseudo-feature filtering rate is increased to 97.6%, the misclassification rate is reduced to 1.1%, the pseudo-feature filtering rate is improved by 19.2 percentage points, and the misclassification rate is reduced by 83.6%.

[0119] This embodiment is used to verify the improvement in 3D reconstruction accuracy achieved by the method of this application. The test object is a standard gauge block (dimensional accuracy ±0.001mm). 3D reconstruction was performed using both the method of this application and conventional methods, and its key dimensions were measured. Experimental results show that the 3D reconstruction accuracy of the conventional method is ±0.052mm, while the 3D reconstruction accuracy of the method of this application is improved to ±0.013mm, an improvement of 75.0%, fully meeting the accuracy requirements for component geometric measurement in the precision manufacturing field.

[0120] High saturation pixel rate 13.5% 0.9% 93.2% Data hole filling rate 77.4%~82.6% 99.1% +16.7 percentage points False feature filtering rate 78.4% 97.6% +19.2 percentage points False feature misclassification rate 6.7% 1.1% -83.6% 3D Reconstruction Accuracy ±0.052mm ±0.013mm 75.0% Note: The above comparative data are derived from the actual test results of embodiments 1-4 of this application, and the test objects include the metal exterior of mobile phones. Typical highly reflective components such as shells, stainless steel stampings, precision castings, and standard gauge blocks are compared with conventional methods / single-criteria methods in the same real-world conditions. Comparative verification was conducted under experimental conditions.

[0121] Comparison of key technical effects of the method in this application with existing technologies: The above comparative experimental data fully demonstrate that the combination of the following technical means proposed in this application—multi-ray intensity level projection and adaptive exposure adjustment, optimal fusion based on pixel-level brightness response curves, anisotropic diffusion low-texture enhancement based on structural tensors, pseudo-feature point cloud filtering based on dual criteria of phase consistency and modulation degree, and adaptive sliding window size—has achieved unexpected technical effects compared to existing technologies, which cannot be obtained by those skilled in the art based on the simple superposition of existing technologies.

[0122] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A non-contact metrological detection method based on image recognition, characterized in that, include: Project a structured light pattern onto the surface of the component under test, and project the same structured light pattern using N different light intensity levels in sequence within a preset time window, where N is a positive integer greater than or equal to 3; At each light intensity level, deformable structured light images of the surface of the component under test are acquired synchronously, and the exposure parameters for image acquisition are automatically adjusted according to the current light intensity level to obtain images containing unsaturated pixel areas. Identify saturated pixel regions in each image caused by localized highlight reflection; For the identified saturated pixel region, from N images corresponding to N light intensity levels, based on the light intensity level and brightness response curves pre-established for each pixel position, select at least one image where the corresponding pixel position is unsaturated and the brightness value is in the center of the preset optimal response range, and extract the structured light pattern information of the selected image at the corresponding pixel position. For the identified unsaturated pixel regions, the image with the highest light intensity level is selected from the N images corresponding to the N light intensity levels, and the structured light pattern information of the image at the corresponding pixel position is extracted. The structured light pattern information extracted from the saturated pixel region and the structured light pattern information extracted from the unsaturated pixel region are spliced ​​and fused according to the original pixel spatial position to generate a complete structured light information map without saturated pixel region. Local texture analysis is performed on the structured light information map. Based on the dual criteria of phase consistency value and modulation value of grating stripes within a preset-sized sliding window, pseudo-feature point clouds formed by multiple reflections of light on the surface of the component are identified and filtered out. The size of the sliding window is adaptively determined according to the period of the grating stripes in the structured light pattern. Based on the structured light information map after filtering out false feature point clouds, the three-dimensional spatial coordinates of the surface of the component under test are calculated using the principle of triangulation.

2. The method according to claim 1, characterized in that, Before splicing and fusing the structured light pattern information extracted from the saturated pixel region and the structured light pattern information extracted from the unsaturated pixel region according to the original pixel spatial position, the method further includes: Extract feature points with local contrast variations from the N images; Based on the feature points, calculate the initial offset between the N images; Based on the initial offset, the N images are initially resampled and aligned; The local signal-to-noise ratio of the structured light pattern in the image after initial resampling and alignment is evaluated to determine the reliability of each pixel location; Based on the reliability, the subpixel-level displacement field between the N images is iteratively calculated using the optical flow method or the cross-correlation method. Based on the subpixel-level displacement field, the N images are finely calibrated and resampled until the change in displacement field calculated in two adjacent iterations is less than a preset threshold.

3. The method according to claim 1, characterized in that, The process of identifying and filtering out false feature point clouds formed by multiple reflections of light on the surface of a component includes: A sliding window of a preset size is used to scan the structured light information map pixel by pixel; Within each sliding window, calculate the phase coherence value and modulation value of the grating fringes; If the phase consistency value is lower than the first preset threshold and the modulation value is higher than the second preset threshold, then the spatial point corresponding to the center pixel of its sliding window is marked as a pseudo feature point. The set of all spatial points marked as pseudo-feature points is defined as the pseudo-feature point cloud; Each pseudo-feature point in the pseudo-feature point cloud is replaced with the average coordinates of the unlabeled spatial points in its neighborhood to filter out the pseudo-feature point cloud.

4. The method according to claim 1, characterized in that, After generating a complete structured light information map with unsaturated pixel regions, the process also includes: Calculate the full-image brightness histogram of the structured light information map; Based on the full-image brightness histogram, identify low-texture regions with contrast below a preset contrast threshold; For the low-texture region, an anisotropic diffusion filtering method based on the structure tensor is used to enhance the contrast of the grating stripes in the region. Phase unwrapping calculations are performed on the contrast-enhanced grating fringes to obtain a continuous absolute phase distribution.

5. The method according to claim 1, characterized in that, Before performing local texture analysis on the structured light information map to identify and filter out false feature point clouds formed by multiple reflections of light on the component surface, the method further includes: For the low-texture regions in the structured light information map, an anisotropic diffusion filtering method based on the structure tensor is used to enhance them, thereby generating an enhanced structured light information map. The execution order between the anisotropic diffusion enhancement step and the pseudo-feature point cloud recognition step is defined as follows: anisotropic diffusion enhancement is performed first, followed by pseudo-feature point cloud recognition and filtering based on dual criteria.

6. The method according to claim 1, characterized in that, The identification of saturated pixel regions in each image caused by localized highlight reflection includes: The brightness value of each pixel in each image is compared with a preset saturation brightness threshold. Pixels whose brightness value is greater than or equal to the saturation brightness threshold are marked as saturated pixels; Perform connected component analysis on the pixels of the saturated pixel; Connected regions with an area greater than a preset minimum area threshold are defined as the saturated pixel regions.

7. The method according to claim 1, characterized in that, The automatic adjustment of image acquisition exposure parameters based on the current light intensity level includes: A mapping table between the light intensity level and the exposure time is established in advance; Based on the light intensity level currently being projected, the corresponding exposure time is obtained by querying the mapping table. Adjust the exposure time to the exposure time obtained from the query; In the mapping table, the light intensity level is negatively correlated with the exposure time.

8. The method according to claim 1, characterized in that, The calculation of the three-dimensional spatial coordinates of the surface of the component under test using the triangulation principle includes: Extracting the center line of the grating stripes for each pixel in the structured light information image after filtering out false feature point clouds; Calculate the phase value corresponding to each pixel based on the position of the center line of the grating stripes; Based on the phase value, the baseline distance between the structured light projector and the image acquisition device, and the calibrated system parameters, the depth value corresponding to each pixel is calculated using the triangulation formula. Based on the depth value and the intrinsic parameter matrix of the image acquisition device, the three-dimensional spatial coordinates corresponding to each pixel are calculated.

9. The method according to claim 1, characterized in that, The steps for coordinating the N light intensity levels with the exposure parameters include: Pre-calibrate N light intensity levels and establish a mapping table between each light intensity level and exposure time; Based on the mapping table, the exposure parameters for image acquisition are dynamically adjusted for each light intensity level. Ensure that images acquired at each light intensity level contain unsaturated pixel areas.

10. A non-contact metrological detection system based on image recognition, characterized in that, include: The structured light projection module is used to project a structured light pattern onto the surface of the component under test, and to project the same structured light pattern sequentially using N different light intensity levels within a preset time window, where N is a positive integer greater than or equal to 3; An image acquisition module is used to synchronously acquire deformable structured light images of the surface of the component under test at each light intensity level. The image acquisition module includes an exposure control unit, which is used to automatically adjust the exposure parameters according to the current light intensity level to obtain an image containing unsaturated pixel areas. The saturation region recognition module is used to identify saturated pixel regions in each image caused by local highlight reflection; The fusion module is used to perform the following operations: For the identified saturated pixel region, select at least one image from the N images corresponding to the N light intensity levels that is not saturated at the corresponding pixel position, and extract the structured light pattern information of the selected image at the corresponding pixel position; For the identified unsaturated pixel regions, the image with the highest light intensity level is selected from the N images corresponding to the N light intensity levels, and the structured light pattern information of the image at the corresponding pixel position is extracted. The structured light pattern information extracted from the saturated pixel region and the structured light pattern information extracted from the unsaturated pixel region are spliced ​​and fused according to the original pixel spatial position to generate a complete structured light information map without saturated pixel region. The pseudo-feature filtering module is used to perform local texture analysis on the structured light information map. Based on the dual criteria of phase consistency value and modulation value of grating stripes within a preset-sized sliding window, it identifies and filters out pseudo-feature point clouds formed by multiple reflections of light on the surface of the component. The size of the sliding window is adaptively determined according to the period of the grating stripes in the structured light pattern. The 3D reconstruction module is used to calculate the 3D spatial coordinates of the surface of the component under test based on the structured light information map after filtering out false feature point clouds and the principle of triangulation.

Images