A method and system for specular removal based on metasurfaces

Abstract

This invention discloses a method and system for specular highlight elimination based on metasurfaces, belonging to the field of optical 3D measurement technology. The specular highlight elimination method disclosed in this invention actively modulates the polarization state of the projected light field using a tunable metasurface device, maximizing the difference in polarization states between the specular and non-spectral components after reflection from the surface of the object being measured. Then, a polarizer at the imaging end filters out the specular component reflected from the mirror while ensuring no loss in the intensity of the non-spectral component, thereby achieving specular highlight elimination in optical 3D measurement and improving measurement accuracy. The pixel-by-pixel projection light field polarization state adaptive modulation method of this invention achieves specular highlight elimination without weakening the intensity of the non-spectral component, effectively solving the problem of decreased accuracy or even failure in optical 3D measurement of high dynamic range surfaces.

Description

Technical Field

[0001] This invention belongs to the field of optical three-dimensional measurement technology, specifically relating to a method and system for eliminating specular highlights based on metasurfaces. Background Technology

[0002] With the improvement of industrial manufacturing capabilities, products are becoming more diversified, complex, and intelligent, which places higher demands on product quality and inspection. Currently, enterprises mainly rely on measuring tools and coordinate measuring machines (CMMs) for inspection. However, for complex contours, such as turbine blades, the cost of manufacturing specialized measuring tools is high, they introduce human factors, and they can only provide qualitative results. While CMMs offer high precision, their probe-based contact measurement method results in low measurement efficiency. Therefore, these methods cannot meet the high-speed, high-precision inspection requirements of products. Optical 3D measurement technology, due to its non-contact inspection characteristics and the advantages of both high speed and high precision, is increasingly being researched and recognized by scholars and enterprises, and is now widely used in medicine, industry, agriculture, transportation, and many other fields.

[0003] For complex metal surfaces, such as turbine blades, when using optical 3D measurement methods to measure their surface geometry, the influence of specular highlights can sometimes prevent the acquisition of measurement data for the highlighted areas, affecting the integrity of the measurement and reducing its accuracy. Existing specular highlight removal methods mainly include polarization elimination, adaptive projection, multi-exposure fusion, and data restoration techniques. The polarization elimination method analyzes the reflection model and considers that the appearance of specular highlights is mainly caused by specular reflection components. It utilizes the polarization characteristics of the specular reflection components and uses polarization devices, such as polarizers, to eliminate the specular reflection components while retaining the diffuse reflection components, thereby reducing or eliminating the influence of specular highlights. (See attached image) Figure 3As shown, polarization elimination methods also filter out reflection components in non-highlight areas, resulting in darker camera image brightness and poorer fringe quality in these areas. Furthermore, when the polarizer is at a non-optimal polarization angle, it cannot completely eliminate specular reflection components in highlight areas, reducing measurement accuracy. Adaptive projection methods establish a response relationship between the projector's projected light intensity and the camera's acquired light intensity, finding the optimal grayscale value for each pixel of the projector and projecting it, thereby reducing or eliminating the influence of highlights. However, in some surface areas with strong highlights, simply adjusting the projection brightness using adaptive projection methods cannot effectively reduce or eliminate the influence of highlights. Multi-exposure fusion methods control the camera to acquire images with multiple exposures, selecting the optimal grayscale value for each pixel under its optimal exposure, thereby reducing or eliminating the influence of highlights. Since multi-exposure fusion methods require acquiring multiple camera images, measurement efficiency is significantly reduced. Data restoration methods distinguish between highlight and non-highlight areas, acquire measurement data from non-highlight areas, and restore and compensate for the data in highlight areas based on properties such as data continuity. Due to the nature of data continuity, data restoration methods are not applicable when there are large areas of surface abrupt changes and highlight areas.

[0004] Considering the above problems, it is necessary to study a highlight removal method that can improve or solve these problems. Summary of the Invention

[0005] The purpose of this invention is to improve or solve the above-mentioned shortcomings. This invention proposes a method and system for eliminating specular highlights based on metasurfaces. Combining the characteristic of tunable metasurfaces that can adaptively control the light field information, and based on polarization elimination and adaptive projection technology, the influence of specular highlights is effectively reduced or eliminated, thereby improving the integrity and accuracy of measurement data.

[0006] To achieve the above objectives, the following steps are included:

[0007] In a first aspect, the present invention provides a method for specular removal based on metasurfaces, comprising the following steps:

[0008] Project white and black images onto the object under test to obtain the contour of the imaging saturation region and the maximum projection brightness value. Project the low-brightness stripe pattern under the maximum projection brightness onto the surface of the object under test to establish the pixel coordinate mapping between the projection end and the imaging end.

[0009] Based on the pixel coordinate mapping relationship between the projection end and the imaging end, the area to be modulated on the projection end image plane is derived from the contour pixels of the saturated area of ​​the camera image plane. Then, based on the position of the highlight area in the object being measured, the angle relationship between the projection end and the imaging end of the highlight area and the polarization state of the corresponding projected light are derived.

[0010] For the aperture projection pixels in the highlight region of the projection end that do not have a projection-camera mapping relationship, interpolation is performed to fill the gaps by combining the projection polarization state of the neighboring pixels of the aperture projection pixels; a polarization state transition interval is added between the saturated and unsaturated regions of the image plane at the projection end, and the polarization state of the projected light rays of the pixels in the polarization state transition interval is smoothed to determine the optimal projection polarization state distribution of the entire light field.

[0011] By using the results of high-light detection and polarization state inversion at the projection end, the polarization state of the light emitted from the tunable metasurface at the projection end is controlled, so that it adapts to changes in the measured surface and the measurement angle.

[0012] After the polarization state of the projected light field is adjusted pixel by pixel, an adaptive projection fringe pattern is generated and projected onto the surface of the object under test. At the imaging end, the transmission direction of the polarizer is adjusted to filter out the specular reflection component. Then, the three-dimensional point cloud data of the object under test is calculated by the fringe imaging pattern after filtering out the specular reflection.

[0013] As a further improvement of the present invention, the step of projecting a white image and a black image onto the object under test to obtain the contour of the imaging saturation region and the maximum projection brightness value, projecting a low-brightness stripe pattern under the maximum projection brightness onto the surface of the object under test, and establishing a pixel coordinate mapping between the projection end and the imaging end includes:

[0014] A white and black image is projected onto the surface of the object under test. The pixel coordinates of the saturated region contour in the camera image are found based on the specular grayscale threshold. Based on the projected white and black images, a grayscale mapping model of projection-imaging is established. The maximum unsaturated imaging grayscale, i.e., the specular grayscale threshold, is used as input. The maximum projection brightness value of the projector is obtained according to the mapping model. Finally, the low-brightness fringe pattern under the maximum projection brightness is projected onto the surface of the object under test. The pixel coordinate mapping between the projection end and the imaging end is established based on the phase information of the fringe pattern or the speckle information of the speckle pattern.

[0015] As a further improvement of the present invention, the step of retrieving the angular relationship between the projection end and the imaging end of the highlight region and the polarization state of the corresponding projected light rays based on the position of the highlight region in the object under test includes:

[0016] Based on the spatial pose relationship between the projection end, the imaging end, and the object, the angle between the projected light and the reflected light in the highlight region of the measured object is calculated. Then, based on the area and divergence angle of the projection light source, the surface contour, reflectivity, roughness, processing texture of the measured object, and the correlation between the projection and imaging spatial pose, combined with the Fresnel reflection model of the dielectric surface and the angle between projection and imaging, the change in polarization state of the light after reflection from the surface is calculated. Furthermore, by maximizing the difference in polarization state between the reflected light from the highlight region and the reflected light from the non-highlight region after reflection from the surface of the measured object, the polarization state of the projected light in the highlight region is calculated in reverse.

[0017] As a further improvement of the present invention, the interpolation filling based on the projection polarization state of the neighboring pixels of the hole projection pixel includes:

[0018] For the hole projection pixels in the highlight area of ​​the projection end that have no projection-camera mapping relationship, polarization state interpolation is performed to fill in the polarization state. That is, based on surface continuity, the polarization state of the hole pixel at the projection end is determined by the polarization mean of the pixels in its eight neighborhoods using an interpolation method to determine the polarization state of the projection light of that pixel.

[0019] As a further improvement of the present invention, the step of adding a polarization state transition interval between the saturated and unsaturated regions of the image plane at the projection end, and smoothing the polarization state of the projected light rays of the pixels within the polarization state transition interval to determine the optimal projection polarization state distribution of the entire light field includes:

[0020] A polarization state transition interval is added between the saturated and unsaturated regions of the projection. The saturated region of the projection end image plane is expanded to generate the transition interval. Within the transition interval, the polarization state of the projected light rays from the non-highlight region smoothly transitions to the polarization state of the highlight region, and finally the optimal projection polarization state distribution of the entire projected light field is determined.

[0021] As a further improvement of the present invention, the step of controlling the polarization state of the light emitted from the tunable metasurface at the projection end through the results of high-light detection and polarization state inversion at the projection end, so that it adaptively changes with the measured surface and the measurement angle, includes:

[0022] The polarization state of the projected light is modulated by a tunable metasurface device. By utilizing the flexible light field modulation capability of the tunable metasurface, the metaatoms of the tunable metasurface are controlled so that the polarization state of each outgoing light after the incident light passes through the tunable metasurface changes accordingly with the measured surface, projection, and imaging angle. That is, the polarization state of the projected light corresponding to the highlight area is actively adjusted to elliptically polarized light by the tunable metasurface, so that it becomes linearly polarized light after being reflected by the surface of the measured object, and has the greatest difference from the linearly polarized light of other non-highlight components.

[0023] As a further improvement of the present invention, the transmission direction of the polarizer is adjusted at the imaging end to filter out the specular reflection component, and then the three-dimensional point cloud data of the object under test is calculated from the stripe imaging pattern after filtering out the specular reflection, including:

[0024] Projected fringes are generated based on the polarization state distribution of the projected light field, and the high-light reflection component is filtered out at the imaging end by adjusting the light transmission direction of the polarizer. Projected fringes are generated by a programmable light source array, and after passing through a tunable metasurface device, they form variable polarization state projected fringes with a preset polarization state distribution of the projected light field. These fringes are projected onto the surface of the object under test, and the high-light reflection component is filtered out at the imaging end by adjusting the light transmission direction of the polarizer. Then, the three-dimensional point cloud data of the object under test is calculated from the fringe imaging diagram after filtering out the high light.

[0025] Secondly, the present invention provides a specular removal system based on metasurfaces, comprising:

[0026] A module is established to project white and black images onto the object under test, obtain the contour of the imaging saturation region and the maximum projection brightness value, project the low brightness stripe pattern under the maximum projection brightness onto the surface of the object under test, and establish the pixel coordinate mapping between the projection end and the imaging end.

[0027] The inversion module is used to invert the outline pixels of the saturated area of ​​the camera image plane to the area that needs to be modulated on the image plane of the projection end based on the pixel coordinate mapping relationship between the projection end and the imaging end. Then, based on the position of the highlight area in the object under test, it inverts the angular relationship between the projection end and the imaging end of the highlight area and the polarization state of the corresponding projected light.

[0028] The determination module is used to count the number of hole projection pixels in the highlight area of ​​the projection end that do not have a projection-camera mapping relationship, and to interpolate and fill the hole projection pixels by combining the projection polarization state of the neighboring pixels; a polarization state transition interval is added between the saturated and unsaturated regions of the image plane at the projection end, and the polarization state of the projected light rays of the pixels in the polarization state transition interval is smoothed to determine the optimal projection polarization state distribution of the entire light field.

[0029] The control module is used to control the polarization state of the light emitted from the tunable metasurface at the projection end by using the results of high-light detection and polarization state inversion at the projection end, so that it adapts to changes in the measured surface and the measurement angle.

[0030] The calculation module is used to generate an adaptive projection fringe pattern after the polarization state of the projected light field is adjusted pixel by pixel, and then projected onto the surface of the object under test. At the imaging end, the transmission direction of the polarizer is adjusted to filter out the specular reflection component, and then the three-dimensional point cloud data of the object under test is calculated from the fringe imaging pattern after the specular reflection is filtered out.

[0031] Thirdly, the present invention provides a specular removal device based on metasurface, comprising: an array programmable light source, a tunable metasurface device, a camera, a polarizer, and a data processing terminal;

[0032] A tunable metasurface device is arranged in front of a programmable light source array for pixel-by-pixel modulation of the polarization information of the projected light field; a polarizer is arranged in front of a camera for filtering out the specular component reflected from the surface of the object under test; a data processing terminal is used for controlling each device, calculating the polarization state of the projected light field, and processing and analyzing measurement data, and the data processing terminal executes the specular elimination method based on metasurface.

[0033] Compared with existing specular highlight elimination methods, this invention uses a tunable metasurface to adaptively control the light field information at the projection end, making the polarization direction of the specular reflection components of the projected light after reflection from the surface of the object under test perpendicular to the highlight and non-highlight regions, maximizing the difference. This allows the polarizer in front of the camera to eliminate the specular reflection components in the highlight regions while retaining the reflection components in the non-highlight regions, thereby eliminating the influence of highlights and avoiding the reduction in brightness and stripe contrast in the non-highlight regions, thus improving measurement accuracy. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the measurement system of the present invention.

[0035] Figure 2 This is a schematic diagram of projection-imaging and polarization state inversion.

[0036] Figure 3 This is a schematic diagram of data acquisition by a polarization camera on a steam turbine blade. Detailed Implementation

[0037] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0038] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0039] The terms "first," "second," "third," "fourth," etc., used in this application's specification and the aforementioned drawings, if present, are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0040] It should be understood that in this application, "at least one item" refers to one or more items, and "more than one item" refers to two or more items. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of A, B, or C can represent: A, B, C, "A and B", "A and C", "B and C", or "A and B and C", where A, B, and C can be single or multiple.

[0041] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0042] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0043] The first objective of this invention is to provide a method for specular removal based on metasurfaces, comprising the following steps:

[0044] Step 1: Project white and black images onto the object under test to obtain the maximum projection brightness value. Based on the high-light grayscale threshold, obtain the contour of the imaging saturation region. Project the low-brightness stripe pattern (or speckle pattern) under the maximum projection brightness onto the surface of the object under test to form an image. Based on the phase information of the stripe pattern or the speckle information of the speckle pattern, establish the pixel coordinate mapping between the projection end and the imaging end.

[0045] Step 2: Based on the pixel coordinate mapping relationship between the projection end and the imaging end, the area to be modulated on the projection end image plane is derived from the outline pixels of the saturated area of ​​the camera image plane. Then, based on the position of the highlight area in the object being measured, the angle relationship between the projection end and the imaging end of the highlight area and the polarization state of the corresponding projected light are derived.

[0046] Step 3: Count the hole projection pixels in the highlight area of ​​the projection end that do not have a projection-camera mapping relationship, and fill them by interpolation based on the projection polarization state of their neighboring pixels; add a polarization state transition interval between the saturated and unsaturated areas of the image plane at the projection end, and smooth the polarization state of the projected light of the pixels in the transition interval, and finally determine the optimal projection polarization state distribution of the entire light field.

[0047] Step four: Based on the results of high-light detection and polarization state inversion at the projection end, control the polarization state of the emitted light from each "meta-atom" on the tunable metasurface at the projection end, so that it adapts to changes in the measured surface and the measurement angle, so that the reflected light projected onto the high-light reflection area becomes perpendicular to the polarization direction of the light in the non-high-light area after being reflected by the metal surface.

[0048] Step 5: After the polarization state of the projected light field is adjusted pixel by pixel, an adaptive projection fringe pattern (or projection speckle pattern) is generated and projected onto the surface of the object under test. At the imaging end, the transmission direction of the polarizer is adjusted to filter out the specular reflection component. Then, by combining the fringe image (or speckle image) after filtering out the specular reflection with the system calibration parameters, the three-dimensional point cloud data of the object under test is calculated.

[0049] This invention utilizes a tunable metasurface device to actively modulate the polarization state of the projected light field, maximizing the difference in polarization states between the highlight and non-highlight components after reflection from the surface of the object being measured. A polarizer at the imaging end then filters out the specularly reflected highlight component while ensuring no loss of intensity in the non-highlight component, thereby achieving highlight elimination in optical 3D measurement and improving measurement accuracy. This pixel-by-pixel adaptive modulation method of the projected light field polarization state achieves highlight elimination without weakening the intensity of the non-highlight component, effectively solving the problem of decreased accuracy or even failure in optical 3D measurement of high dynamic range surfaces.

[0050] The underlying principles of each step in this process are analyzed as follows:

[0051] Initial settings for projection and imaging: By projecting white and black images onto the object under test, the outline of the imaging saturation region and the maximum projected brightness value can be determined. This is to determine the optimal operating parameters of the projection and imaging system.

[0052] By projecting a low-brightness fringe pattern at maximum projection brightness onto the surface of the object under test, a pixel coordinate mapping relationship can be established between the projection end and the imaging end. This is the basis for subsequent 3D measurements.

[0053] Processing of highlight areas: Based on the pixel coordinate mapping relationship, the area of ​​the image plane that needs to be modulated at the projection end, as well as the angular relationship of the highlight area between the projection end and the imaging end, can be determined. Highlight areas often cause imaging distortion due to excessive surface reflection. By determining the angular relationship and polarization state of the highlight area, the polarization state of the projected light can be adjusted to reduce the impact of highlights.

[0054] Interpolation filling of holes: In the highlight area of ​​the projection end, there may be projected pixels with holes where no projection-camera mapping relationship has been established. These holes will cause missing 3D data. By interpolating the projection polarization state of the neighboring pixels of the hole projected pixels, the polarization state of the hole region can be estimated, thereby completing the 3D data.

[0055] Smooth polarization transition: Adding a polarization transition interval between the saturated and unsaturated regions of the image plane at the projection end ensures a smooth transition of the projected light field, avoiding imaging distortion caused by abrupt changes in polarization state. By smoothing the polarization state of the projected rays of the pixels within the polarization transition interval, the optimal projection polarization state distribution of the entire light field can be determined.

[0056] Adaptive Projection and Imaging: Based on the results of specular detection and polarization state inversion at the projection end, the polarization state of the light emitted from the tunable metasurface at the projection end is controlled. This allows the projected light field to adaptively change with variations in the measured surface and the measurement angle, reducing the impact of specular reflection. After pixel-by-pixel polarization state adjustment, an adaptive projection fringe pattern is generated and projected onto the surface of the object under test. At the imaging end, by adjusting the transmission direction of the polarizer, the specular reflection component can be filtered out, resulting in a more accurate fringe image.

[0057] Solving 3D point cloud data: By filtering out the specular highlights from the fringe image, the 3D point cloud data of the object under test can be calculated. This data can be used for subsequent applications such as surface reconstruction and dimensional measurement.

[0058] The key to this process lies in establishing precise pixel coordinate mapping through projection and imaging techniques, and in reducing the impact of specular reflection by adjusting the polarization state of the projected light. This enables the system to achieve high-precision 3D measurement of complex curved surfaces.

[0059] The measurement elements of the system of this invention mainly consist of a programmable light source array, a tunable metasurface device, a camera, a polarizer, and a data processing terminal. The tunable metasurface device is arranged in front of the programmable light source array for pixel-by-pixel modulation of the polarization information of the projected light field. The polarizer is arranged in front of the camera to filter out the high-light component reflected from the surface of the object being measured. The data processing terminal is used for controlling each device, calculating the polarization state of the projected light field, and processing and analyzing the measurement data.

[0060] The invention will now be further described with reference to the accompanying drawings.

[0061] This invention includes the following steps:

[0062] Step 1: First, project a white image and a black image onto the surface of the object being measured. Based on the specular grayscale threshold (250), find the pixel coordinates of the saturated region outline in the camera image. Simultaneously, based on the projected white and black images, establish a grayscale mapping model between projection and imaging. Using the maximum unsaturated imaging grayscale, i.e., the specular grayscale threshold (250), as input, calculate the maximum projection brightness value of the projector according to this mapping model. Finally, project the low-brightness fringe pattern (or speckle pattern) under the maximum projection brightness onto the surface of the object being measured. Based on the phase information of the fringe pattern or the speckle information of the speckle pattern, establish a pixel coordinate mapping between the projection end and the imaging end. Considering the inconsistency between the camera image coordinates and the projector image coordinates, the pixel coordinates need to be normalized or interpolated when establishing the pixel coordinate mapping.

[0063] Step two, first, as shown in the attached document. Figure 2 As shown, based on the pixel coordinate mapping relationship between the projection end and the imaging end, the pixel coordinates of the area to be modulated on the projection end image plane are derived from the pixel coordinates of the saturated area contour of the camera image plane. Secondly, based on the spatial pose relationship between the projection end, the imaging end, and the object, the angle between the projected light and the reflected light in the highlight area of ​​the measured object is calculated. Then, based on the correlation between the area and divergence angle of the projected light source, the surface contour, reflectivity, roughness, processing texture, and spatial pose of the projection and imaging, the change in polarization state of the light after surface reflection is calculated by combining the Fresnel reflection model of the dielectric surface and the angle between projection and imaging. Furthermore, by maximizing the difference in polarization state between the highlight reflected light and the reflected light in the non-highlight area after reflection from the surface of the measured object, the polarization state of the projected light in the highlight area is deduced in reverse.

[0064] Step 3: First, identify the hole projection pixels in the highlight region of the projection end that do not have a projection-camera mapping relationship. Based on surface continuity, use interpolation to fill in the projection polarization state of the hole projection pixels by using the polarization mean of the pixels in their eight neighborhoods. Second, add a polarization state transition interval between the saturated and unsaturated regions of the image surface at the projection end, and smooth the polarization state of the projected light rays of the pixels in the transition interval. Finally, determine the optimal projection polarization state distribution of the entire light field.

[0065] Step four: First, using the results of polarization state inversion at the high-light detection and projection end, the flexible light field modulation capability of the tunable metasurface is utilized to control each "meta-atom" of the tunable metasurface. This causes the polarization state of each outgoing ray after the incident light passes through the tunable metasurface to change accordingly with the measured surface, projection, and imaging angle. In other words, the tunable metasurface actively adjusts the polarization state of the projected light corresponding to the high-light region to elliptically polarized light, which then becomes linearly polarized light after reflection from the surface of the measured object. This makes the polarization direction of the reflected light projected onto the high-light reflection region perpendicular to the polarization direction of the light in the non-high-light region, thus maximizing the difference between the linearly polarized light and other non-high-light components.

[0066] Step 5: First, based on the circular array and the guide rail, the calibration parameters of the system are solved. Second, after the polarization state of the projected light field is adjusted pixel by pixel, an adaptive projection fringe pattern (or projection speckle pattern) is generated and projected onto the surface of the object under test. At the imaging end, the specular reflection component of the highlight area is filtered out by adjusting the transmission direction of the polarizer, while the reflection component of the non-highlight area is retained. Finally, the measurement point cloud data of the surface of the object under test is solved by combining the fringe image (or speckle image) after filtering out the highlights with the system calibration parameters.

[0067] This invention is applied to the field of acquiring three-dimensional geometric contour information, such as industrial production measurement, and is particularly suitable for measuring high dynamic range surfaces with local high reflectivity.

[0068] This invention can be applied to the measurement of high dynamic range surfaces with localized high reflectivity, providing a new high-precision measurement method for the geometric morphology inspection of reflective surfaces, such as metal parts and smooth ceramic parts, in industrial inspection. Furthermore, the measurement cost is controllable, eliminating the need for pre-measurement spraying of diffuse reflection powder or significantly increasing the measurement angle and exposure times, resulting in lower measurement time costs. It has broad application prospects and considerable economic benefits.

[0069] The pixel-by-pixel projection light field polarization state adaptive modulation method of the present invention achieves high light elimination without weakening the intensity of non-high light components, effectively solving the problem of decreased or even failed accuracy of optical three-dimensional measurement of high dynamic range surfaces.

[0070] A second objective of this invention is to provide a metasurface-based specular removal system, comprising:

[0071] A module is established to project white and black images onto the object under test, obtain the contour of the imaging saturation region and the maximum projection brightness value, project the low brightness stripe pattern under the maximum projection brightness onto the surface of the object under test, and establish the pixel coordinate mapping between the projection end and the imaging end.

[0072] The inversion module is used to invert the outline pixels of the saturated area of ​​the camera image plane to the area that needs to be modulated on the image plane of the projection end based on the pixel coordinate mapping relationship between the projection end and the imaging end. Then, based on the position of the highlight area in the object under test, it inverts the angular relationship between the projection end and the imaging end of the highlight area and the polarization state of the corresponding projected light.

[0073] The determination module is used to count the number of hole projection pixels in the highlight area of ​​the projection end that do not have a projection-camera mapping relationship, and to interpolate and fill the hole projection pixels by combining the projection polarization state of the neighboring pixels; a polarization state transition interval is added between the saturated and unsaturated regions of the image plane at the projection end, and the polarization state of the projected light rays of the pixels in the polarization state transition interval is smoothed to determine the optimal projection polarization state distribution of the entire light field.

[0074] The control module is used to control the polarization state of the light emitted from the tunable metasurface at the projection end by using the results of high-light detection and polarization state inversion at the projection end, so that it adapts to changes in the measured surface and the measurement angle.

[0075] The calculation module is used to generate an adaptive projection fringe pattern after the polarization state of the projected light field is adjusted pixel by pixel, and then projected onto the surface of the object under test. At the imaging end, the transmission direction of the polarizer is adjusted to filter out the specular reflection component, and then the three-dimensional point cloud data of the object under test is calculated from the fringe imaging pattern after the specular reflection is filtered out.

[0076] The third objective of this invention is to provide a specular removal device based on metasurfaces, comprising: an array programmable light source, a tunable metasurface device, a camera, a polarizer, and a data processing terminal;

[0077] A tunable metasurface device is arranged in front of a programmable light source array for pixel-by-pixel modulation of the polarization information of the projected light field; a polarizer is arranged in front of a camera for filtering out the specular component reflected from the surface of the object under test; a data processing terminal is used for controlling each device, calculating the polarization state of the projected light field, and processing and analyzing measurement data, and the data processing terminal executes the specular elimination method based on metasurface.

[0078] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for removing specular highlights based on metasurfaces, characterized in that, Includes the following steps: Project white and black images onto the object under test to obtain the contour of the imaging saturation region and the maximum projection brightness value. Project the low-brightness stripe pattern under the maximum projection brightness onto the surface of the object under test to establish the pixel coordinate mapping between the projection end and the imaging end. Based on the pixel coordinate mapping relationship between the projection end and the imaging end, the area to be modulated on the projection end image plane is derived from the contour pixels of the saturated area of ​​the camera image plane. Then, based on the position of the highlight area in the object being measured, the angle relationship between the projection end and the imaging end of the highlight area and the polarization state of the corresponding projected light are derived. For the aperture projection pixels in the highlight region of the projection end that do not have a projection-camera mapping relationship, interpolation is performed to fill the gaps by combining the projection polarization state of the neighboring pixels of the aperture projection pixels; a polarization state transition interval is added between the saturated and unsaturated regions of the image plane at the projection end, and the polarization state of the projected light rays of the pixels in the polarization state transition interval is smoothed to determine the optimal projection polarization state distribution of the entire light field. By using the results of high-light detection and polarization state inversion at the projection end, the polarization state of the light emitted from the tunable metasurface at the projection end is controlled, so that it adapts to changes in the measured surface and the measurement angle. After the polarization state of the projected light field is adjusted pixel by pixel, an adaptive projection fringe pattern is generated and projected onto the surface of the object under test. At the imaging end, the transmission direction of the polarizer is adjusted to filter out the specular reflection component. Then, the three-dimensional point cloud data of the object under test is calculated by the fringe imaging pattern after filtering out the specular reflection.

2. The method for removing specular highlights based on metasurfaces according to claim 1, characterized in that, The process of projecting white and black images onto the object under test to obtain the contour of the imaging saturation region and the maximum projection brightness value, projecting the low-brightness stripe pattern under the maximum projection brightness onto the surface of the object under test, and establishing a pixel coordinate mapping between the projection end and the imaging end includes: A white and black image is projected onto the surface of the object under test. The pixel coordinates of the saturated region contour in the camera image are found based on the specular grayscale threshold. Based on the projected white and black images, a grayscale mapping model of projection-imaging is established. The maximum unsaturated imaging grayscale, i.e., the specular grayscale threshold, is used as input. The maximum projection brightness value of the projector is obtained according to the mapping model. Finally, the low-brightness fringe pattern under the maximum projection brightness is projected onto the surface of the object under test. The pixel coordinate mapping between the projection end and the imaging end is established based on the phase information of the fringe pattern or the speckle information of the speckle pattern.

3. The method for removing specular highlights based on metasurfaces according to claim 1, characterized in that, The step of retrieving the angular relationship between the projection end and the imaging end of the highlight region and the corresponding polarization state of the projected light rays based on the position of the highlight region in the object under test includes: Based on the spatial pose relationship between the projection end, the imaging end, and the object, the angle between the projected light and the reflected light in the highlight region of the measured object is calculated. Then, based on the area and divergence angle of the projection light source, the surface contour, reflectivity, roughness, processing texture of the measured object, and the correlation between the projection and imaging spatial pose, combined with the Fresnel reflection model of the dielectric surface and the angle between projection and imaging, the change in polarization state of the light after reflection from the surface is calculated. Furthermore, by maximizing the difference in polarization state between the reflected light from the highlight region and the reflected light from the non-highlight region after reflection from the surface of the measured object, the polarization state of the projected light in the highlight region is calculated in reverse.

4. The method for removing specular highlights based on metasurfaces according to claim 1, characterized in that, The interpolation filling based on the projection polarization states of the neighboring pixels of the hole projection pixel includes: For the hole projection pixels in the highlight area of ​​the projection end that have no projection-camera mapping relationship, polarization state interpolation is performed to fill in the polarization state. That is, based on surface continuity, the polarization state of the hole pixel at the projection end is determined by the polarization mean of the pixels in its eight neighborhoods using an interpolation method to determine the polarization state of the projection light of that pixel.

5. The method for removing specular highlights based on metasurfaces according to claim 1, characterized in that, The step of adding a polarization state transition interval between the saturated and unsaturated regions of the image plane at the projection end, and smoothing the polarization state of the projected light rays of the pixels within the polarization state transition interval to determine the optimal projection polarization state distribution of the entire light field includes: A polarization state transition interval is added between the saturated and unsaturated regions of the projection. The saturated region of the projection end image plane is expanded to generate the transition interval. Within the transition interval, the polarization state of the projected light rays from the non-highlight region smoothly transitions to the polarization state of the highlight region, and finally the optimal projection polarization state distribution of the entire projected light field is determined.

6. The method for removing specular highlights based on metasurfaces according to claim 1, characterized in that, The process of controlling the polarization state of the light emitted from the tunable metasurface at the projection end, based on the results of high-light detection and polarization state inversion at the projection end, to make it adaptively change with variations in the measured surface and the measurement angle, includes: The polarization state of the projected light is modulated by a tunable metasurface device. By utilizing the flexible light field modulation capability of the tunable metasurface, the metaatoms of the tunable metasurface are controlled so that the polarization state of each outgoing light after the incident light passes through the tunable metasurface changes accordingly with the measured surface, projection, and imaging angle. That is, the polarization state of the projected light corresponding to the highlight area is actively adjusted to elliptically polarized light by the tunable metasurface, so that it becomes linearly polarized light after being reflected by the surface of the measured object, and has the greatest difference from the linearly polarized light of other non-highlight components.

7. The method for removing specular highlights based on metasurfaces according to claim 1, characterized in that, The process of adjusting the transmission direction of the polarizer at the imaging end to filter out the specular reflection component, and then calculating the three-dimensional point cloud data of the object under test from the stripe imaging pattern after filtering out the specular reflection, includes: Projected fringes are generated based on the polarization state distribution of the projected light field, and the high-light reflection component is filtered out at the imaging end by adjusting the light transmission direction of the polarizer. Projected fringes are generated by a programmable light source array, and after passing through a tunable metasurface device, they form variable polarization state projected fringes with a preset polarization state distribution of the projected light field. These fringes are projected onto the surface of the object under test, and the high-light reflection component is filtered out at the imaging end by adjusting the light transmission direction of the polarizer. Then, the three-dimensional point cloud data of the object under test is calculated from the fringe imaging diagram after filtering out the high light.

8. A specular removal system based on metasurfaces, characterized in that, include: A module is established to project white and black images onto the object under test, obtain the contour of the imaging saturation region and the maximum projection brightness value, project the low brightness stripe pattern under the maximum projection brightness onto the surface of the object under test, and establish the pixel coordinate mapping between the projection end and the imaging end. The inversion module is used to invert the outline pixels of the saturated area of ​​the camera image plane to the area that needs to be modulated on the image plane of the projection end based on the pixel coordinate mapping relationship between the projection end and the imaging end. Then, based on the position of the highlight area in the object under test, it inverts the angular relationship between the projection end and the imaging end of the highlight area and the polarization state of the corresponding projected light. The determination module is used to count the number of hole projection pixels in the highlight area of ​​the projection end that do not have a projection-camera mapping relationship, and to interpolate and fill the hole projection pixels by combining the projection polarization state of the neighboring pixels; a polarization state transition interval is added between the saturated and unsaturated regions of the image plane at the projection end, and the polarization state of the projected light rays of the pixels in the polarization state transition interval is smoothed to determine the optimal projection polarization state distribution of the entire light field. The control module is used to control the polarization state of the light emitted from the tunable metasurface at the projection end by using the results of high-light detection and polarization state inversion at the projection end, so that it adapts to changes in the measured surface and the measurement angle. The calculation module is used to generate an adaptive projection fringe pattern after the polarization state of the projected light field is adjusted pixel by pixel, and then projected onto the surface of the object under test. At the imaging end, the transmission direction of the polarizer is adjusted to filter out the specular reflection component, and then the three-dimensional point cloud data of the object under test is calculated from the fringe imaging pattern after the specular reflection is filtered out.

9. A specular removal device based on metasurfaces, characterized in that, include: Programmable light sources, tunable metasurface devices, cameras, polarizers, and data processing terminals; Tunable metasurface devices are arranged in front of a programmable array light source for pixel-by-pixel modulation of polarization information of the projected light field. A polarizer is placed in front of the camera to filter out the specular component reflected from the surface of the object being measured; a data processing terminal is used for controlling various devices, calculating the polarization state of the projected light field, and processing and analyzing measurement data. The data processing terminal executes the specular elimination method based on metasurfaces as described in any one of claims 1 to 7.

Images