A high light suppression method and system based on amplitude truncation average

Abstract

The application discloses a high light suppression method and system based on modulation degree truncation average, the method of the application firstly projects a uniform gray scale image to the surface of a measured object, divides reflectivity regions according to the light intensity histogram of a collected image, and predicts an optimal exposure time group of each region; then, based on the time group, a plurality of phase shift grating fringes are adaptively projected; finally, taking the fringe modulation degree as a judgment basis, the effective phase points in the plurality of exposure images are subjected to truncation average processing, a high dynamic range (HDR) grating image is synthesized, and phase demodulation and three-dimensional reconstruction are carried out. The application effectively suppresses the high light overexposure problem of the surface of a high-reflective object such as metal through a software algorithm, overcomes the defects that the multiple exposure method is prone to motion ghosting and relies on hardware devices, and realizes high-precision and high-efficiency three-dimensional topography measurement without increasing the complexity and cost of the system.

Description

Technical Field

[0001] This invention belongs to the fields of optical 3D measurement, machine vision and image processing technology, and specifically relates to a method and system for suppressing highlights based on modulation truncation averaging, which is particularly suitable for high-precision and high-efficiency 3D measurement and industrial inspection of metal parts with high reflectivity surfaces. Background Technology

[0002] With the rapid development of advanced manufacturing and testing technologies, 3D topography measurement technology has been widely applied in aerospace, weaponry, automotive, and shipbuilding industries. Key components of spacecraft, missiles, and other weaponry are mostly made of metal, ensuring sufficient strength, durability, and machinability to withstand extreme conditions and guarantee the normal operation of the entire system. Therefore, to ensure the accuracy of details when measuring the topography of such components, non-contact 3D measurement technology is typically used to obtain point cloud data of the target workpiece. Optical 3D topography measurement technology, as an emerging interdisciplinary field, integrates advanced theories from optics, mechanics, electrical engineering, and computer science with intelligent vision technology, and has become one of the key research areas. In recent years, many scholars have conducted extensive research on optical 3D measurement, with common methods including optical interferometry, time-of-flight methods, structured light projection technology, and stereo vision technology. However, due to the high reflectivity of metal surfaces, incident light undergoes interface reflection at the high-brightness surface during measurement, converging at certain points to produce flares. Limited by the dynamic range of the camera itself, the image at the flare location exhibits saturated pixels, severely affecting image quality and subsequent 3D reconstruction.

[0003] When performing optical 3D measurements on metallic components with highly reflective surfaces, a specific coating is typically sprayed onto the surface of the object to convert its highly reflective parts into diffuse reflective parts. However, this introduces errors from the coating, and cleaning is difficult. Regarding the problem of localized overexposure in optical 3D measurements of metallic components, existing highlight suppression methods include multiple exposure time methods, which require a large number of images and a very stable measurement environment to avoid ghosting between multiple images; adaptive projection intensity methods, which cannot fully adapt the mapping relationship between the camera and the projector and have relatively complex algorithms; and other methods, such as adding polarizing filters, increase system complexity and cost. Therefore, developing a 3D topography measurement technique that balances measurement efficiency and accuracy while being adaptable to different highly reflective objects is a crucial problem that urgently needs to be solved.

[0004] In the paper "Method and System for Measuring Three-Dimensional Topography of Object Surfaces with High Dynamic Range" (CN115839677A) authored by Wang Zhangying, Zhang Zonghua, et al., a projector projects a blue stripe image and a uniform blue image onto the surface of the object being measured. Then, utilizing the different color channel responses of a color camera to the monochromatic stripe projection and color channel separation technology in the color image, the stripe images corresponding to the blue and green channels are separated from the deformed blue stripe image to synthesize a high dynamic range stripe image. This method improves measurement efficiency; however, the selection of the image mask is only related to the camera's color channel response and lacks universality for the high-reflectance of different objects. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for suppressing highlights based on modulation truncation averaging, which can further improve the speed and accuracy of three-dimensional point cloud measurement of metallic parts with high reflectivity surfaces.

[0006] The technical solution for implementing the present invention is as follows: A highlight suppression method based on modulation truncation averaging includes the following steps: S1. Pre-project a uniform grayscale image onto the surface of the object being measured, and adjust the initial exposure time to prevent the overall image from being overexposed; S2. Acquire a grayscale image of the surface of the object being measured, draw a light intensity histogram, and divide the area into multiple regions with similar reflectivity based on the light intensity distribution; S3. Based on the maximum light intensity and ideal light intensity of each region, predict the optimal exposure time for each region; S4. Based on the optimal exposure time group, adaptively project multiple sets of phase-shifting grating stripe images; S5. Set the stripe modulation threshold, judge the modulation of each pixel in multiple sets of exposed images, and generate an effective phase point mask; S6. Perform truncation and averaging on the effective phase points at the same pixel position under multiple exposure times to synthesize a high dynamic range raster image; S7. Perform phase calculation and three-dimensional reconstruction based on the high dynamic range grating image.

[0007] Furthermore, the step of predicting the optimal exposure time includes: According to the formula Calculate the optimal exposure time for each region, where For optimal exposure time, t This is the current exposure time. To achieve ideal light intensity, This represents the current light intensity being collected.

[0008] Furthermore, the formula for calculating the stripe tone intensity is as follows:

[0009] in, The midpoint of the phase-shifted image ( x , y (Regulation information) H a and H b , respectively, are the sine and cosine components of the phase-shifted image, and N is the number of phase-shifting steps.

[0010] Furthermore, the effective phase point mask is generated as follows: If the modulation of a certain pixel Greater than the set threshold Y， If the value is 1, it is marked as a valid phase point; otherwise, it is 0.

[0011] Furthermore, the truncated averaging process includes: For the same pixel location, effective phase points at multiple exposure times are masked and superimposed. If the number of effective exposures is greater than 1, the multiple exposure images at that location are weighted and averaged to generate a composite image.

[0012] Furthermore, the stripe modulation threshold is set to 20.

[0013] A system for implementing the above method includes: High-resolution projectors are used to project uniform grayscale images and phase-shifting grating stripes; High-resolution camera, used to acquire images; A timing controller is used to synchronize the control of the projector and the camera; Computers are used to perform image processing, modulation calculations, mask generation, image synthesis, and 3D reconstruction.

[0014] Furthermore, the system also includes a horizontal movement device and a checkerboard calibration plate for system calibration.

[0015] Beneficial effects: 1. This invention automatically divides the surface of the object under test into regions with different reflectivities by analyzing the light intensity histogram of the pre-projected image, and accurately predicts the optimal exposure time for each region. This method overcomes the blindness and experience-based dependence in exposure time selection in traditional multiple exposure methods, avoids the acquisition of a large number of invalid images, and thus significantly improves the overall efficiency of three-dimensional measurement.

[0016] 2. This invention uses stripe modulation as a basis for judgment, which can effectively distinguish between valid phase points and invalid phase points caused by overexposure, underexposure, or noise. By using the truncation averaging method to synthesize only the data of the valid area, it significantly reduces jumps and errors in phase calculation, and finally obtains a more complete and accurate 3D point cloud model, especially ensuring the data quality of highly reflective areas.

[0017] 3. This invention employs a truncated averaging strategy to synthesize HDR images, effectively offsetting pixel-level misalignment (ghosting) caused by slight movements of the measured object or minor vibrations of the equipment during multiple image acquisitions. This reduces the method's requirements for the stability of the measurement environment, enhances its robustness, and makes it more suitable for non-ideal laboratory environments such as industrial sites.

[0018] 4. The method of this invention achieves high dynamic range measurement solely through optimized software algorithms and processing flows, eliminating the need for expensive hardware such as polarizing filters and high dynamic range cameras. While ensuring high measurement accuracy, it effectively controls system complexity and cost, resulting in greater engineering application value and wider application potential. Attached Figure Description

[0019] Figure 1 This is a highlight suppression method based on modulation truncation averaging; Figure 2 This is a layout diagram of the measurement system; Among them, 1-the object with high reflectivity to be measured; 2-horizontal movement device; 3-chessboard calibration plate; 4-high resolution projector; 5-high resolution camera; 6-time controller; 7-computer; Figure 3 Diagram of a fringe projection optical measurement system; Figure 4 This is a schematic diagram of the light intensity histogram of the object being measured. Figure 5 3D reconstruction of specular suppression for planar metal parts; Figure 6 High dynamic range image synthesis for planar metal parts; Figure 7 HDR metallic step specular suppression for 3D reconstruction. Detailed Implementation

[0020] This invention provides a method and system for suppressing highlights based on modulation truncation averaging. This method can demodulate the modulated projection grating on the surface of a component with high light reflectivity and obtain three-dimensional point cloud data through phase calculation, thereby realizing the shape scanning measurement of the entire surface of the key component.

[0021] The method of this invention is based on the modulation cutoff averaging of highly reflective objects. The measurement process is as follows: Figure 1As shown, firstly, a uniform grayscale image of (254, 254, 254) is pre-projected onto the surface of the object being measured; then, the initial exposure time t0 is adjusted so that there are no overexposed areas in the overall image; next, the camera acquires the image of the object being measured, draws a light intensity histogram, and calculates the optimal exposure time for different regions; then, an adaptive projection grating fringe image is generated based on the optimal exposure time for different regions; finally, a modulation threshold is set, and based on the fringe modulation, truncation averaging is performed to find the same effective phase point regions in multiple sets of images to synthesize an HDR fringe image, followed by subsequent phase calculation and 3D reconstruction.

[0022] The measurement process of grating projection technology is as follows: Figure 2 As shown, the process includes computer-generated grating fringes projected onto the surface of the object being measured by a projector, a camera acquiring the modulated grating fringes image, and sending it to a computer for phase calculation. Combined with the mapping relationship between the unfolded phase and depth obtained from system calibration, a 3D reconstruction of the object's surface is completed. In actual measurement, the influence of ambient light is considered; the actual light intensity acquired by the camera can be used... I ( x , y ) is represented as: (1) in, α The surface reflectance of the object being measured. I p This refers to the projection brightness of the projector. I e For ambient light intensity, αI e The ambient light intensity reflected by the object being measured. The ambient light intensity that directly enters the camera. k For camera sensitivity, t For camera exposure time, the overall situation is as follows: Figure 3 When the projection brightness is high enough or the environment is dark, the ambient light intensity... I e and If all of them can be ignored, then equation (1) can be simplified to: (2) From equation (2), we can see that at a certain point k and α Keeping it constant, the camera captures the actual light intensity. I With camera exposure time t Projection intensity of the projector I p Regarding the relationship between the light intensity captured by the camera at a certain pixel location and the surface reflectivity of the object being measured, the surface reflectivity of a specific pixel cannot be obtained as a specific value due to camera sensitivity limitations. k Exposure time tWith projected light intensity I p With a constant exposure time, the difference in light intensity captured by the camera at different pixel positions can be reflected in the difference in surface reflectivity. Therefore, by dividing the object under test into pixel regions with similar light intensities, we can obtain locations with similar surface reflectivity. Adjusting and optimizing the exposure time for different regions can reduce the randomness in exposure time selection and increase measurement efficiency.

[0023] According to equation (2), the ideal light intensity for collection is... I a and the required optimal exposure time t b The relationship is shown in equation (3): (3) After transforming the relation, we can obtain: (4) Transforming equation (2), we obtain equation (5): (5) By combining equations (4) and (5), the optimal exposure time is obtained. t b Compared with ideal light intensity I a Actual collected light intensity I The relationship between them is shown in equation (6): (6) When the acquired image is 8-bit data, the light intensity encoding range for each pixel is 0-255, where 0 indicates no light signal was captured at that pixel location, and 255 indicates oversaturation at that pixel location. Ideal light intensity... I a With a value of 254, a high signal-to-noise ratio is maintained while avoiding camera saturation. Therefore, the optimal exposure time can be predicted simply by obtaining the actual exposure time and the collected light intensity value.

[0024] After the camera acquires light signals and forms an image, it processes the signal via a chip protocol and outputs a simulated light intensity signal. This signal has been forcibly normalized to a dimensionless constant within the range of 0-255 by the camera's internal hardware and software. The exposure time is adjusted to avoid overall overexposure, and a uniform grayscale image is pre-projected to plot the light intensity histogram of the acquired image.

[0025] A light intensity histogram is plotted on the ideal object to be measured based on the image captured by the camera, and the object is divided into multiple clusters. Each cluster has one peak value and two trough values, such as... Figure 4As shown, pixels within each cluster have similar surface reflectivity, so a single projection of a single cluster is sufficient for measurement. To completely cover all locations within the entire cluster, the second trough location of each cluster, i.e., the maximum light intensity value within the cluster, is selected to calculate the optimal exposure time.

[0026] Assuming the light intensity histogram of the acquired image includes k There are several clusters, and the maximum light intensity of each cluster is [value missing]. , ... Substituting the exposure time and light intensity into equation (6), the optimal exposure time for different regions can be obtained as shown in equation (7): (7) In the phase calculation process of grating projection technology, the modulation formula of a set of grating fringe images needs to be based on... N Calculate the sine image of the phase-shifted grating fringe image and H a With cosine graph and H b As shown in (8): (8) in, F n+ 1 is the first n +1 phase shift step corresponding image. The midpoint of the phase shift image is calculated according to equation (8). x , y (Regulation information) T ( x , y )for: (9) The fused grating fringe image obtained by this method has a high signal-to-noise ratio and can maximize fringe contrast while ensuring that all pixels are not overexposed. However, it requires high stability of the measurement environment when dynamically adjusting the camera exposure time and acquiring multiple images in the same scene. When there is movement of the object under test or camera / projector jitter during continuous shooting, overlapping ghost images may appear during image fusion, affecting the accuracy of phase calculation and 3D topography. To reduce the error caused by jitter of the object under test or equipment, the same portions of the images that meet the requirements are truncated and averaged according to the modulation index in different sets of exposure images, thereby improving the stability of the system.

[0027] Using the fringe modulation degree of the grating as the criterion, a fringe modulation degree truncation threshold is set to process phase points. Pixels filtered out are considered invalid phase points, while phase points exceeding the threshold are considered valid phase points. The fewer invalid phase points, the more accurate the deformable grating solution, which is more conducive to subsequent 3D reconstruction and measurement of the target. A multi-exposure time method is used to perform 3D measurement of highly reflective objects. Assuming a selection... i Images with different exposure times are stitched together, and pixel-by-pixel comparison is performed based on the fringe modulation information. According to the fringe modulation information calculated by equation (9) and combined with the set threshold Y, the effective phase points in each image that are not overexposed are selected to calculate the phase point mask. As in equation (10): (10) Calculate pixel by pixel according to equation (11) i The sum of effective phase point masks for a group of images If the sum is greater than 1, it means that the pixel meets the phase accuracy requirement under multiple exposure times. The image mask is used to calculate the truncated average of multiple sets of points at the same pixel position, and the result is shown in Equation (12).

[0028] (11) (12) In the formula: M j ( x , y ), j =1, 2, 3… i It represents i The image mask was acquired at different exposure times. During the structured light 3D measurement, the fringe modulation threshold was set to 20 to better remove phase outliers and ensure the accuracy of phase measurement. The acquired original grating image and the mask image were fused into a set of HDR grating fringe images, as shown in Equation (13): (13) The obtained HDR grating stripe image is then subjected to subsequent phase calculation and 3D reconstruction.

[0029] A measurement system was constructed to measure planar metal parts with discontinuous surfaces. A uniform grayscale image was pre-projected onto the surface of the metal part, and the initial exposure time was adjusted to ensure that the entire surface was not overexposed, ignoring the influence of ambient light intensity. Grayscale images and light intensity histograms were acquired. Optimal exposure time groups were calculated based on the light intensity histograms for adaptive projection measurement. Deformed grating images were acquired by a camera, and effective pixels in each image group whose modulation met the measurement conditions were selected and truncated at the same location to generate an image mask. An HDR stripe image of the metal part was synthesized based on the mask image, and subsequent phase calculation and 3D reconstruction were performed based on the HDR image. Figure 5 and Figure 6 As shown.

[0030] To further verify the measurement effect of the proposed method on objects with high surface reflectivity, five customized HDR metal steps with different surface reflectivities were used. The surface roughness of step surfaces A, B, C, D, and E ranged from Ra 6.3 to Ra 0.4. The same procedure was followed: pre-projected images were used, and grayscale images and intensity histograms were collected to calculate the optimal exposure time group for adaptive projection measurement. Effective pixels in each image group whose modulation met the measurement conditions were selected, and their positions were truncated and averaged to generate an image mask. Finally, the specular suppression measurement of the HDR metal steps was completed. Figure 7 As shown.

[0031] Example based on Figure 2 The specific implementation process of the method of the present invention for the system shown is as follows: (1) Place the checkerboard calibration plate 3 in front of the high-resolution camera 5 to perform camera calibration. (2) Place the checkerboard calibration plate 3 on the horizontal moving device 2 and perform system calibration with the high-resolution projector 4 and the high-resolution camera 5; (3) The projector and camera are synchronously controlled by the computer 7 connected to the timing controller 6, and a uniform grayscale image is pre-projected onto the high reflectivity object 1 to be measured and its histogram is observed. (4) Based on the different regions of the object surface reflectivity collected by the high-resolution camera 5, multiple optimal exposure time groups are adaptively predicted. (5) Set the stripe modulation threshold, and truncate each group of raster images pixel by pixel to select qualified areas, thereby increasing image stability. (6) The images are stitched together to form a high dynamic range image, and phase calculation and three-dimensional reconstruction are performed to realize rapid and high-precision measurement of high light reflectivity components.

[0032] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A highlight suppression method based on modulation cutoff averaging, characterized in that, Includes the following steps: S1. Pre-project a uniform grayscale image onto the surface of the object being measured, and adjust the initial exposure time to prevent the overall image from being overexposed; S2. Acquire a grayscale image of the surface of the object being measured, draw a light intensity histogram, and divide the area into multiple regions with similar reflectivity based on the light intensity distribution; S3. Based on the maximum light intensity and ideal light intensity of each region, predict the optimal exposure time for each region; S4. Based on the optimal exposure time group, adaptively project multiple sets of phase-shifting grating stripe images; S5. Set the stripe modulation threshold, judge the modulation of each pixel in multiple sets of exposed images, and generate an effective phase point mask; S6. Perform truncation and averaging on the effective phase points at the same pixel position under multiple exposure times to synthesize a high dynamic range raster image; S7. Perform phase calculation and three-dimensional reconstruction based on the high dynamic range grating image.

2. The method according to claim 1, characterized in that, The steps for predicting the optimal exposure time include: According to the formula Calculate the optimal exposure time for each region, where For optimal exposure time, t This is the current exposure time. To achieve ideal light intensity, This represents the current light intensity being collected.

3. The method according to claim 1, characterized in that, The formula for calculating the stripe tone intensity is: in, The midpoint of the phase-shifted image ( x , y (Regulation information) H a and H b , respectively, are the sine and cosine components of the phase-shifted image, and N is the number of phase-shifting steps.

4. The method according to claim 3, characterized in that, The effective phase point mask is generated as follows: If the modulation of a certain pixel Greater than the set threshold Y， If the value is 1, it is marked as a valid phase point; otherwise, it is 0.

5. The method according to any one of claims 1-4, characterized in that, The truncated averaging process includes: For the same pixel location, effective phase points at multiple exposure times are masked and superimposed. If the number of effective exposures is greater than 1, the multiple exposure images at that location are weighted and averaged to generate a composite image.

6. The method according to claim 5, characterized in that, The stripe modulation threshold is set to 20.

7. A system for implementing the method of claim 1, characterized in that, include: High-resolution projectors are used to project uniform grayscale images and phase-shifting grating stripes; High-resolution camera, used to acquire images; A timing controller is used to synchronize the control of the projector and the camera; Computers are used to perform image processing, modulation calculations, mask generation, image synthesis, and 3D reconstruction.

8. The system according to claim 7, characterized in that, It also includes a horizontal movement device and a checkerboard calibration plate for system calibration.

Images