An automatic multi-exposure selection method based on fringe projection

Abstract

The application discloses an automatic multi-exposure selection method based on fringe projection. Firstly, a projector is used to project a phase-shift fringe pattern to a measured scene, and a camera synchronously collects a group of fringe images with a preset exposure value. A pseudo-white image is obtained by using a highest-frequency fringe image. An overexposure area with a brightness greater than a preset threshold value in the pseudo-white image is found, and a seed filling algorithm is used to divide the overexposure area into a plurality of sub-areas. The pixel number and average brightness of the largest overexposure sub-area are used to iteratively calculate an optimal initial exposure time, so that the initial exposure is as large as possible and the measured scene has no obvious overexposure area under the initial exposure. An automatic multi-exposure algorithm based on a fringe image modulation degree is used to calculate a multi-exposure number and automatically iterate subsequent exposure times, high-quality absolute phases under the exposure times are fused to obtain a multi-exposure fusion absolute phase. The calibration parameters of a binocular camera are used to perform stereo rectification on the absolute phases of left and right cameras. The absolute phases of the left and right cameras are phase-matched to obtain a two-dimensional disparity map, and high-speed and high-precision three-dimensional topography measurement is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical measurement, and in particular relates to an automatic multi-exposure selection method based on fringe projection. Background Art

[0002] In fringe projection profilometry, multi-exposure fusion is a very effective method for achieving high dynamic range measurements in scenes with complex surface reflectivity. In multi-exposure fusion, a camera captures fringe images at multiple exposure times, allowing nearly all areas of the scene with varying reflectivity to be optimally measured at a single exposure time. Fusion of these measurements enables high dynamic range measurement of the entire scene. However, multi-exposure fusion often requires manual experience to determine exposure combinations for different measurement scenarios, which is both time-consuming and labor-intensive, and difficult to achieve optimal results. Summary of the Invention

[0003] The purpose of the present invention is to propose an automatic multi-exposure selection method based on fringe projection.

[0004] The technical solution for achieving the purpose of the present invention is: an automatic multi-exposure selection method based on fringe projection, comprising the following steps:

[0005] Step 1: Use a projector to project a phase-shifted fringe pattern onto the scene being measured. The camera simultaneously captures a set of fringe patterns at a preset exposure value, and uses the highest-frequency fringe pattern to generate a pseudo-white image.

[0006] Step 2: Find the overexposed area in the pseudo-white image whose brightness is greater than the preset threshold, and use the seed filling algorithm to partition the overexposed area into several sub-areas;

[0007] Step 3: Based on the number of pixels and average brightness of the largest overexposed sub-region, the exposure time is iterated and calculated to obtain the optimal initial exposure time, so that the initial exposure is as large as possible and there is no obvious overexposed area in the measured scene under this exposure;

[0008] Step 4: Calculate the number of multi-exposures using an automatic multi-exposure algorithm based on the fringe pattern modulation depth and automatically iterate the subsequent exposure times. Fuse the high-quality absolute phases at each exposure time to obtain the multi-exposure fused absolute phase.

[0009] Step 5: Use the calibration parameters of the binocular camera to perform stereo correction on the absolute phases of the left and right cameras, and perform phase matching on the absolute phases of the left and right cameras to obtain a two-dimensional disparity map, thereby achieving high-speed and high-precision three-dimensional shape measurement.

[0010] Preferably, step 2 finds the overexposed area in the pseudo-white image whose brightness is greater than a preset threshold, and uses a seed filling algorithm to partition the overexposed area into several sub-areas:

[0011] ①Pseudo-white that only includes overexposed areas Figure 2 Valueization, select one or more seed points with non-zero values ​​as starting points and mark them as visited;

[0012] ② Starting from the current seed point, search for pixels in its upper left, upper, upper right, left, right, lower left, lower, and lower right neighborhoods in sequence. If the pixel has not been visited and its value is not zero, mark it as visited and add it to the current connected domain.

[0013] ③ Update the label values ​​of all pixels in the current connected domain to the same label value as the seed point;

[0014] ④ Repeat ② and ③ until all pixels have been visited.

[0015] Preferably, in step 3, the exposure time is iterated and calculated based on the number of pixels and the average brightness of the largest overexposed sub-region to obtain the optimal initial exposure time, so that the initial exposure is as large as possible and there is no obvious overexposed area in the measured scene under this exposure. The specific method is:

[0016] Find the sub-region with the largest number of pixels and determine whether the number of pixels in this region is less than the set threshold P1; if it exceeds the threshold, it is considered that there is an overexposed area in the image under the current exposure t0, and the overexposed area is assigned to the variable satu_mask in the form of a binary mask image, and t0 is halved and returned to step 1; if it does not exceed the threshold, determine whether the variable satu_mask has been assigned a value. If satu_mask has not been assigned a value, it means that the highlight area of ​​the measured scene has not been recorded, then t0 is doubled and returned to step 1; if satu_mask has been assigned a value, the average brightness Avg of the first P1 points in the highlight area satu_mask according to the pseudo-white image under the current exposure time t0 I Calculate the optimal initial exposure time t1.

[0017] Preferably, the optimal initial exposure time t1 is:

[0018]

[0019] in, Indicates the overexposure threshold of image light intensity.

[0020] Preferably, the subsequent exposure time is calculated as follows:

[0021]

[0022] in, The exposure time is t n When the modulation index is less than and closest to the modulation index threshold I t”; n=1,2,...,M, where M is a positive integer representing the total number of exposures; Indicates the overexposure threshold of image light intensity.

[0023] Preferably, the calculation formula for the total number of exposures is:

[0024]

[0025] Where, I” min Represents the light intensity threshold of the shadow area under the initial exposure. Points with a value less than this value are considered to be in the shadow area. Indicates that when the exposure time is t1, the modulation index is less than and closest to the modulation index threshold I t "The light intensity of the pixel, It represents the overexposure threshold of the image light intensity, and floor() represents the rounding down function.

[0026] Preferably, the absolute phase Φ after fusion is specifically:

[0027]

[0028] Where, Φ i is the absolute phase obtained from the i-th exposure, A i The absolute phase of the i-th exposure is greater than the threshold I t The mask corresponding to the area ”.

[0029] Compared with the existing technology, the present invention has the following significant advantages: the present invention automatically determines the optimal initial exposure according to the measured scene, and combines it with the automatic multi-exposure algorithm based on the fringe pattern modulation to achieve fully automatic multi-exposure fusion for any scene.

[0030] The present invention will be described in further detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Schematic diagram of the process of an automatic multi-exposure selection method based on fringe projection.

[0032] Figure 2 Result diagram of the automatic multi-exposure selection method based on fringe projection of the present invention. DETAILED DESCRIPTION

[0033] An automatic multi-exposure selection method based on fringe projection comprises the following steps:

[0034] Step 1: Use a projector to project a phase-shifted fringe pattern onto the scene being measured. The camera simultaneously captures a set of fringe patterns at a preset exposure value. The highest-frequency fringe pattern is used to generate a pseudo-white image. The intensity of each pixel in this pseudo-white image is the maximum intensity value of the corresponding pixel in the multiple phase-shifted fringe patterns. This step is to obtain information about the scene being measured.

[0035] Step 2: Find the overexposed areas in the pseudo-white image whose brightness is greater than the preset threshold, and use the seed filling algorithm to partition the overexposed areas into several sub-areas:

[0036] ①Pseudo-white that only includes overexposed areas Figure 2 Valueization, select one or more seed points with non-zero values ​​as starting points and mark them as visited;

[0037] ② Then, starting from the current seed point, search for pixels in its upper left, upper, upper right, left, right, lower left, lower, and lower right neighborhoods in sequence. If the pixel has not been visited and its value is not zero, mark it as visited and add it to the current connected domain.

[0038] ③ Update the label values ​​of all pixels in the current connected domain to the same label value as the seed point;

[0039] ④ Repeat ② and ③ until all pixels have been visited.

[0040] Each subregion obtained after partitioning by the seed filling algorithm is an eight-connected region, that is, starting from each pixel in the region, you can reach any pixel in the region by moving in eight directions without going out of the region.

[0041] Step 3: Based on the number of pixels and average brightness of the largest overexposed sub-area, the exposure time is iterated and calculated to obtain the optimal initial exposure time, so that the initial exposure is as large as possible and there is no obvious overexposed area in the measured scene under this exposure. The specific process is as follows:

[0042] Find the sub-region with the largest number of pixels and determine whether the number of pixels in this region is less than the threshold value P1, which is generally set to one ten-thousandth of the total number of pixels in the image. If it exceeds the threshold, it is considered that there is an overexposed area in the image at the current exposure value t0. The overexposed area is assigned to the variable satu_mask in the form of a binary mask (the significance of this variable is to record the highlight area of ​​the scene), and t0 is halved. Return to step 1 and re-collect the fringe image with the changed t0 as the exposure value to obtain a pseudo-white image, and execute the subsequent steps in sequence. If it does not exceed the threshold, determine whether the variable satu_mask has been assigned. If satu_mask has not been assigned, it means that the highlight area of ​​the measured scene has not been recorded. Then, double t0 and return to step 1. Re-collect the fringe image with the changed t0 as the exposure value to obtain a pseudo-white image, and execute the subsequent steps in sequence. If satu_mask has been assigned, calculate the average brightness Avg of the first P1 points in the highlight area satu_mask based on the pseudo-white image at the current exposure time t0. I Calculate the optimal initial exposure time t1. Since the brightness increases linearly with exposure time, t1 can be calculated as follows:

[0043]

[0044] in, Indicates the overexposure threshold of the image light intensity, which is generally set to 245 for grayscale images.

[0045] Step 4: Calculate the number of multi-exposures using an automatic multi-exposure algorithm based on the fringe pattern modulation depth and automatically iterate the subsequent exposure times. Fuse the high-quality absolute phases at each exposure time to obtain the multi-exposure fused absolute phase:

[0046] The method used in step 4 to calculate the number of multi-exposures and automatically iterate the subsequent exposure time adopts the automatic multi-exposure algorithm based on the fringe pattern modulation proposed by Rao et al. from Southeast University. In the fringe projection system, the phase error of the absolute phase follows a Gaussian distribution and the standard deviation satisfies the following formula:

[0047]

[0048] Where N is the number of phase shift steps, σ represents the standard deviation of the random noise distribution, and I' represents the modulation depth of the fringe pattern. According to the research of Zuo et al., when σ Φ When the phase error ΔΦ is less than 0.025, there is a 95.45% probability that it is less than 0.05, and the phase quality is acceptable at this time. According to the research of Rao et al., the standard deviation of the random noise of the camera varies with the brightness and has a maximum value. Therefore, when the phase shift step number is determined, there is a modulation index threshold I t "When I" at a certain point on the fringe diagram is greater than I t", the phase quality of the solved absolute phase at this point is acceptable.

[0049] In Rao's method, the camera needs to first collect a set of fringe images without saturation light intensity at exposure time t1. Find the fringe images with a modulation index greater than I t The phase quality of the region G1 under the exposure time t1 is satisfactory. In this region, the modulation index is closest to I t "The pixel P1 and calculate the maximum light intensity of the pixel Since the light intensity increases linearly with exposure time, the second exposure time t2 can be calculated as follows:

[0050]

[0051] in, is the threshold representing the saturation light intensity. At exposure time t2, the light intensity of P1 is close to saturation, and the pixels in the G1 area are almost overexposed, while the brightness and modulation of the pixels outside the G1 area increase the most and reasonably with the increase of exposure time. In the second set of fringe patterns, continue to find the pixel with the modulation closest to I t " and calculate the maximum light intensity of the pixel Calculate the third exposure time according to the following formula:

[0052]

[0053] This cycle is repeated to obtain a set of optimal exposure time combinations for the tested scene. In the first exposure fringe pattern, it is considered that the fringe modulation is less than the threshold value I” min The pixels are located in the shadow area, and the exposure time calculation process is terminated using the threshold, that is, the total number of multi-exposures M can be obtained by the following formula:

[0054]

[0055] in, That is, for the convenience of calculation, it is assumed that the maximum light intensity at each exposure time remains unchanged.

[0056] Based on the above, the iterative formula for subsequent exposure time is:

[0057]

[0058] in, The exposure time is t n When the modulation index is less than and closest to the modulation index threshold I t ”; n=1,2,...,M, where M is a positive integer representing the total number of exposures.

[0059] The calculation formula for the total number of exposures M is:

[0060]

[0061] The above formula can be written as:

[0062]

[0063] Among them, I” min Represents the light intensity threshold of the shadow area under the initial exposure. Points with a value less than this value are considered to be in the shadow area. That is, for the convenience of calculation, it is assumed that No change; floor() represents the floor function.

[0064] After calculating the absolute phase corresponding to M exposure times, each absolute phase Φ n Only keep the corresponding exposure time adjustment system greater than the threshold I t "The area. The mask corresponding to this area is A n , and A n+1 Does not include A n The final fused absolute phase Φ can be calculated as follows:

[0065]

[0066] Step 5: Use the calibration parameters of the binocular camera to perform stereo correction on the absolute phases of the left and right cameras, and perform phase matching on the absolute phases of the left and right cameras to obtain a two-dimensional disparity map, thereby achieving high-speed and high-precision three-dimensional shape measurement.

[0067] Figure 2 In the figure, (a1)-(a5) represent the fringe patterns at different exposure times; (b1)-(b5) represent the absolute phases calculated at different exposure times; (c1)-(c5) represent the areas taken at different exposure times during phase fusion; (d) and (e) represent the fusion phases of the left and right cameras, respectively; (f) represents the two-dimensional disparity map obtained based on the left and right fusion phases.

Claims

1. An automatic multi-exposure selection method based on fringe projection, characterized in that: The steps include: Step 1: Use a projector to project a phase-shifted fringe pattern onto the scene being measured. The camera simultaneously captures a set of fringe patterns at a preset exposure value, and uses the highest-frequency fringe pattern to generate a pseudo-white image. Step 2: Find the overexposed area in the pseudo-white image whose brightness is greater than the preset threshold, and use the seed filling algorithm to partition the overexposed area into several sub-areas; Step 3: Based on the number of pixels and average brightness of the largest overexposed sub-region, the exposure time is iterated and calculated to obtain the optimal initial exposure time, so that the initial exposure is as large as possible and there is no obvious overexposed area in the measured scene under this exposure; Step 4: Calculate the number of multi-exposures using an automatic multi-exposure algorithm based on the fringe pattern modulation depth and automatically iterate the subsequent exposure times. Fuse the high-quality absolute phases at each exposure time to obtain the multi-exposure fused absolute phase. Step 5: Use the calibration parameters of the binocular camera to perform stereo correction on the absolute phases of the left and right cameras, and perform phase matching on the absolute phases of the left and right cameras to obtain a two-dimensional disparity map, thereby achieving high-speed and high-precision three-dimensional shape measurement.

2. The automatic multi-exposure selection method based on fringe projection according to claim 1, characterized in that: Step 2: Find the overexposed area in the pseudo-white image whose brightness is greater than the preset threshold, and use the seed filling algorithm to partition the overexposed area into several sub-areas: ① Binarize the pseudo-white image containing only the overexposed area, select one or more seed points with non-zero values ​​as the starting points, and mark them as visited; ② Starting from the current seed point, search for pixels in its upper left, upper, upper right, left, right, lower left, lower, and lower right neighborhoods in sequence. If the pixel has not been visited and its value is not zero, mark it as visited and add it to the current connected domain. ③ Update the label values ​​of all pixels in the current connected domain to the same label value as the seed point; ④ Repeat ② and ③ until all pixels have been visited.

3. The automatic multi-exposure selection method based on fringe projection according to claim 1, characterized in that: Step 3: The exposure time is iterated and calculated based on the number of pixels and average brightness of the largest overexposed sub-region to obtain the optimal initial exposure time. The specific method is as follows: Find the sub-region with the largest number of pixels and determine whether the number of pixels in this region is less than the set threshold P1; if it exceeds the threshold, it is considered that there is an overexposed area in the image under the current exposure t0, and the overexposed area is assigned to the variable satu_mask in the form of a binary mask image, and t0 is halved and returned to step 1; if it does not exceed the threshold, determine whether the variable satu_mask has been assigned a value. If satu_mask has not been assigned a value, it means that the highlight area of ​​the measured scene has not been recorded, then t0 is doubled and returned to step 1; if satu_mask has been assigned a value, the average brightness Avg of the first P1 points in the highlight area satu_mask according to the pseudo-white image under the current exposure time t0 I Calculate the optimal initial exposure time t1.

4. The automatic multi-exposure selection method based on fringe projection according to claim 3, characterized in that: The optimal initial exposure time t1 is specifically: in, Indicates the overexposure threshold of image light intensity.

5. The automatic multi-exposure selection method based on fringe projection according to claim 1, characterized in that: The subsequent exposure time is calculated as: in, The exposure time is t n When the modulation index is less than and closest to the modulation index threshold I t ”; n=1,2,...,M, where M is a positive integer representing the total number of exposures; Indicates the overexposure threshold of image light intensity.

6. The automatic multi-exposure selection method based on fringe projection according to claim 5, characterized in that: The total number of exposures is calculated as: Where, I” min Represents the light intensity threshold of the shadow area under the initial exposure. Points with a value less than this value are considered to be in the shadow area. Indicates that when the exposure time is t1, the modulation index is less than and closest to the modulation index threshold I t "The light intensity of the pixel, It represents the overexposure threshold of the image light intensity, and floor() represents the rounding down function.

7. The automatic multi-exposure selection method based on fringe projection according to claim 1, characterized in that: The absolute phase Φ after fusion is specifically: Where, Φ i is the absolute phase obtained from the i-th exposure, A i The absolute phase of the i-th exposure is greater than the threshold I t The mask corresponding to the area ”.

Images