Automatic multi-exposure selection method based on fringe projection
Through the automatic multi-exposure selection method based on stripe projection, the overexposed area is partitioned using the pseudo-white map and seed filling algorithm, and combined with the automatic multi-exposure algorithm of the stripe pattern modulation system, the problem of artificial combination exposure in the existing technology is solved, and the correct High dynamic range measurement and high-speed and high-precision three-dimensional morphological measurement for complex surface reflectivity scenarios.
Patent Information
- Application Number
- CN202411112457.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-14
AI Technical Summary
The existing multi-exposure fusion technology requires artificially giving exposure combinations in different measurement scenarios based on experience, which is time-consuming and labor-intensive and difficult to obtain optimal results, making it difficult to achieve high dynamic range measurements for complex surface reflectivity scenarios.
Using an automatic multi-exposure selection method based on stripe projection, the phase shifted stripe pattern is projected to the measured scene through the projector, and the overexposure area is partitioned using a pseudo-white map and seed filling algorithm, iteratively calculates the optimal initial exposure time, and combines the stripe pattern tone. The system's automatic multi-exposure algorithm automatically determines the number of multiple exposures and subsequent exposure times, achieving complete automatic multi-exposure fusion of the scene.
High dynamic range measurement of complex surface reflectivity scenes is realized, the optimal initial exposure time and multiple exposure times are automatically determined, the measurement accuracy and efficiency are improved, and high-speed and high-precision three-dimensional morphological measurement is achieved.
Smart Images

Figure CN119022819A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of optical measurement, and in particular relates to an automatic multi-exposure selection method based on fringe projection. Background Art
[0002] In fringe projection profilometry, multi-exposure fusion technology is a very effective method to achieve high dynamic range measurement of scenes with complex surface reflectivity. In multi-exposure fusion technology, the camera collects fringe images at multiple different exposure times, so that areas with different reflectivity in the scene can almost achieve optimal measurement at a certain exposure time. The measurement results of different areas are combined to achieve high dynamic range measurement of the entire scene. However, multi-exposure fusion technology often requires artificial exposure combinations for different measurement scenes based on experience, which is time-consuming and labor-intensive and difficult to obtain optimal results. Summary of the invention
[0003] The purpose of the present invention is to propose an automatic multi-exposure selection method based on fringe projection.
[0004] The technical solution to achieve the purpose of the present invention is: an automatic multi-exposure selection method based on fringe projection, comprising the following steps:
[0005] Step 1: Use a projector to project a phase-shifted fringe pattern onto the scene being tested, and the camera synchronously collects a set of fringe patterns at a preset exposure value, and uses the highest frequency fringe pattern to obtain a pseudo-white image;
[0006] Step 2: Find the overexposed area in the pseudo-white image whose brightness is greater than a preset threshold, and use the seed filling algorithm to partition the overexposed area into several sub-areas;
[0007] Step 3: According to the number of pixels and average brightness of the largest overexposed sub-area, the exposure time is iterated and calculated to obtain the optimal initial exposure time, so that the initial exposure is as large as possible and there is no obvious overexposed area in the tested scene under this exposure;
[0008] Step 4: Calculate the number of multi-exposures and automatically iterate the subsequent exposure time through the automatic multi-exposure algorithm based on the fringe pattern modulation, and fuse the high-quality absolute phases at each exposure time to obtain the absolute phase of multi-exposure fusion;
[0009] Step 5: Use the calibration parameters of the binocular camera to perform stereo correction on the absolute phases of the left and right cameras, and perform phase matching on the absolute phases of the left and right cameras to obtain a two-dimensional disparity map to achieve high-speed and high-precision three-dimensional shape measurement.
[0010] Preferably, step 2 finds an overexposed area in the pseudo white image whose brightness is greater than a preset threshold, and uses a seed filling algorithm to partition the overexposed area into a plurality of sub-areas:
[0011] ①Pseudo-white that only contains the overexposed area Figure 2 Valueization, select one or more seed points with non-zero values as starting points and mark them as visited;
[0012] ② Starting from the current seed point, search for pixels in the eight neighborhoods of upper left, upper, upper right, left, right, lower left, lower, and lower right in turn. If the pixel has not been visited and its value is not zero, mark it as visited and add it to the current connected domain;
[0013] ③ Update the label values of all pixels in the current connected domain to the same label value as the seed point;
[0014] ④ Repeat ② and ③ until all pixels have been visited.
[0015] Preferably, in step 3, the exposure time is iterated and calculated according to the number of pixels and the average brightness of the overexposed sub-region with the largest area to obtain the optimal initial exposure time, so that the initial exposure is as large as possible and there is no obvious overexposed area in the measured scene under this exposure. The specific method is:
[0016] Find the sub-region with the largest number of pixels and determine whether the number of pixels in this region is less than the set threshold value P1; if it exceeds the threshold, it is considered that there is an overexposed area in the image under the current exposure t0, and the overexposed area is assigned to the variable satu_mask in the form of a binary mask image, and t0 is halved and returned to step 1; if it does not exceed the threshold, determine whether the variable satu_mask has been assigned a value. If satu_mask has not been assigned a value, it means that the highlight area of the measured scene has not been recorded, then t0 is doubled and returned to step 1; if satu_mask has been assigned a value, the average brightness Avg of the first P1 points in the highlight area satu_mask according to the pseudo-white image under the current exposure time t0 I Calculate the optimal initial exposure time t1.
[0017] Preferably, the optimal initial exposure time t1 is specifically:
[0018]
[0019] in, Indicates the overexposure threshold of image light intensity.
[0020] Preferably, the subsequent exposure time is calculated as follows:
[0021]
[0022] in, The exposure time is t n When the modulation index is less than and closest to the modulation index threshold I t"; n = 1, 2, ..., M, where M is a positive integer representing the total number of exposures; Indicates the overexposure threshold of image light intensity.
[0023] Preferably, the calculation formula for the total number of exposures is:
[0024]
[0025] In the formula, I” min Represents the light intensity threshold of the shadow area under the initial exposure. Points less than this value are considered to be in the shadow area. Indicates that when the exposure time is t1, the modulation is less than and closest to the modulation threshold I t "The light intensity of the pixel, It represents the overexposure threshold of the image light intensity, and floor() represents the rounding down function.
[0026] Preferably, the fused absolute phase Φ is specifically:
[0027]
[0028] In the formula, Φ i is the absolute phase obtained from the i-th exposure, A i The absolute phase of the i-th exposure is greater than the threshold I t The mask corresponding to the area ”.
[0029] Compared with the prior art, the present invention has the following significant advantages: the present invention automatically determines the optimal initial exposure according to the measured scene, and combines it with the automatic multi-exposure algorithm based on the fringe pattern modulation to achieve fully automatic multi-exposure fusion for any scene.
[0030] The present invention is described in further detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 A flowchart of an automatic multi-exposure selection method based on fringe projection is shown.
[0032] Figure 2 Result diagram of the automatic multi-exposure selection method based on fringe projection of the present invention. DETAILED DESCRIPTION
[0033] An automatic multi-exposure selection method based on fringe projection comprises the following steps:
[0034] Step 1: Use a projector to project a phase-shifted fringe pattern onto the scene being tested. The camera simultaneously collects a set of fringe patterns at a preset exposure value. A pseudo-white image is obtained using the highest-frequency fringe pattern. The light intensity of each pixel of the pseudo-white image is the maximum light intensity value of the corresponding pixel in the multiple phase-shifted fringe patterns. This step is to obtain information about the scene being tested;
[0035] Step 2: Find the overexposed area in the pseudo-white image whose brightness is greater than the preset threshold, and use the seed filling algorithm to partition the overexposed area into several sub-areas:
[0036] ①Pseudo-white that only contains the overexposed area Figure 2 Valueization, select one or more seed points with non-zero values as starting points and mark them as visited;
[0037] ②Then starting from the current seed point, search for pixels in its upper left, upper, upper right, left, right, lower left, lower, and lower right neighborhoods in turn. If the pixel has not been visited and its value is not zero, mark it as visited and add it to the current connected domain;
[0038] ③ Update the label values of all pixels in the current connected domain to the same label value as the seed point;
[0039] ④ Repeat ② and ③ until all pixels have been visited.
[0040] Each sub-region obtained after partitioning by the seed filling algorithm is an eight-connected region, that is, starting from each pixel in the region, you can reach any pixel in the region by moving in eight directions without going out of the region.
[0041] Step 3: According to the number of pixels and average brightness of the largest overexposed sub-area, the exposure time is iterated and calculated to obtain the optimal initial exposure time, so that the initial exposure is as large as possible and there is no obvious overexposed area in the tested scene under this exposure. The specific process is as follows:
[0042] Find the sub-region with the largest number of pixels and determine whether the number of pixels in this region is less than the threshold value P1, which is generally set to one ten-thousandth of the total number of pixels in the image. If it exceeds the threshold, it is considered that there is an overexposed area in the image at the current exposure value t0, and the overexposed area is assigned to the variable satu_mask in the form of a binary mask (the significance of this variable is to record the highlight area of the scene), and t0 is halved, and return to step 1, and the fringe image is re-collected with the changed t0 as the exposure value to obtain a pseudo-white image, and the subsequent steps are performed in sequence. If it does not exceed the threshold, determine whether the variable satu_mask has been assigned. If satu_mask has not been assigned, it means that the highlight area of the measured scene has not been recorded, then t0 is doubled, and return to step 1, and the fringe image is re-collected with the changed t0 as the exposure value to obtain a pseudo-white image, and the subsequent steps are performed in sequence. If satu_mask has been assigned, the average brightness Avg of the first P1 points in the highlight area satu_mask according to the pseudo-white image at the current exposure time t0 I Calculate the optimal initial exposure time t1. Since the brightness increases linearly with the exposure time, t1 can be calculated as follows:
[0043]
[0044] in, Indicates the overexposure threshold of the image light intensity, which is generally set to 245 for grayscale images.
[0045] Step 4: Calculate the number of multi-exposures and automatically iterate the subsequent exposure times through the automatic multi-exposure algorithm based on the fringe pattern modulation, and fuse the high-quality absolute phases at each exposure time to obtain the absolute phase of multi-exposure fusion:
[0046] The method of calculating the number of multi-exposures and automatically iterating the subsequent exposure time in step 4 adopts the automatic multi-exposure algorithm based on the fringe pattern modulation proposed by Rao et al. of Southeast University. In the fringe projection system, the phase error of the absolute phase satisfies the Gaussian distribution and the standard deviation satisfies the following formula:
[0047]
[0048] Where N is the number of phase shift steps, σ represents the standard deviation of the random noise distribution, and I” represents the modulation degree of the fringe pattern. According to the research of Zuo et al., when σ Φ When it is less than 0.025, the phase error ΔΦ has a probability of 95.45% less than 0.05, and the phase quality is acceptable at this time. According to the research of Rao et al., the standard deviation of the random noise of the camera varies with the brightness and has a maximum value. Therefore, when the number of phase shift steps is determined, there is a modulation threshold value I t "When I" at a certain point on the fringe diagram is greater than I t”, the phase quality of the solved absolute phase at this point is acceptable.
[0049] In Rao's method, the camera needs to first collect a set of fringe images without saturation intensity at exposure time t1. Find the fringe images with modulation greater than I t The phase quality of the region G1 under the exposure time t1 is satisfactory. In this region, the modulation degree is close to I t "The pixel P1 and calculate the maximum light intensity of the pixel Since the light intensity increases linearly with exposure time, the second exposure time t2 can be calculated as follows:
[0050]
[0051] in, is the threshold representing the saturation light intensity. At exposure time t2, the light intensity of P1 is close to saturation, and the pixels in the G1 area are almost overexposed, while the brightness and modulation of the pixels outside the G1 area increase the most and reasonably with the increase of exposure time. In the second set of fringe patterns, continue to find the pixel with the modulation closest to I t " and calculate the maximum light intensity of the pixel Calculate the third exposure time according to the following formula:
[0052]
[0053] This cycle is repeated to obtain a set of optimal exposure time combinations for the tested scene. In the first exposed fringe pattern, it is considered that the fringe modulation is less than the threshold value I” min The pixels are located in the shadow area, and the threshold is used to terminate the exposure time calculation process, that is, the total number of multi-exposures M can be obtained by the following formula:
[0054]
[0055] in, That is, for the convenience of calculation, it is assumed that the maximum light intensity at each exposure time remains unchanged.
[0056] Determined from the above, the iterative formula for subsequent exposure time is:
[0057]
[0058] in, The exposure time is t n When the modulation index is less than and closest to the modulation index threshold I t "; n = 1, 2, ..., M, where M is a positive integer representing the total number of exposures.
[0059] The total number of exposures M is calculated as follows:
[0060]
[0061] The above formula can be written as:
[0062]
[0063] Among them, I” min Represents the light intensity threshold of the shadow area under the initial exposure. Points less than this value are considered to be in the shadow area. That is, for the convenience of calculation, it is assumed that No change; floor() represents the floor function.
[0064] After calculating the absolute phase corresponding to M exposure times, each absolute phase Φ n Only keep the corresponding exposure time adjustment system greater than the threshold I t "The mask corresponding to this area is A n , and A n+1 Does not include A n The final fused absolute phase Φ can be calculated by the following formula:
[0065]
[0066] Step 5: Use the calibration parameters of the binocular camera to perform stereo correction on the absolute phases of the left and right cameras, and perform phase matching on the absolute phases of the left and right cameras to obtain a two-dimensional disparity map to achieve high-speed and high-precision three-dimensional shape measurement.
[0067] Figure 2 In the figure, (a1)-(a5) represent the fringe patterns at different exposure times; (b1)-(b5) represent the absolute phases calculated at different exposure times; (c1)-(c5) represent the areas taken at different exposure times during phase fusion; (d)(e) represent the fusion phases of the left and right cameras, respectively; (f) represents the two-dimensional disparity map obtained based on the left and right fusion phases.
Claims
1. An automatic multi-exposure selection method based on fringe projection, characterized in that: The steps include: Step 1: Use a projector to project a phase-shifted fringe pattern onto the scene being tested, and the camera synchronously collects a set of fringe patterns at a preset exposure value, and uses the highest frequency fringe pattern to obtain a pseudo-white image; Step 2: Find the overexposed area in the pseudo-white image whose brightness is greater than a preset threshold, and use the seed filling algorithm to partition the overexposed area into several sub-areas; Step 3: According to the number of pixels and average brightness of the largest overexposed sub-area, the exposure time is iterated and calculated to obtain the optimal initial exposure time, so that the initial exposure is as large as possible and there is no obvious overexposed area in the tested scene under this exposure; Step 4: Calculate the number of multi-exposures and automatically iterate the subsequent exposure time through the automatic multi-exposure algorithm based on the fringe pattern modulation, and fuse the high-quality absolute phases at each exposure time to obtain the absolute phase of multi-exposure fusion; Step 5: Use the calibration parameters of the binocular camera to perform stereo correction on the absolute phases of the left and right cameras, and perform phase matching on the absolute phases of the left and right cameras to obtain a two-dimensional disparity map to achieve high-speed and high-precision three-dimensional shape measurement.
2. The automatic multi-exposure selection method based on fringe projection according to claim 1, characterized in that: Step 2: Find the overexposed area in the pseudo-white image whose brightness is greater than the preset threshold, and use the seed filling algorithm to partition the overexposed area into several sub-areas: ① Binarize the pseudo-white image containing only the overexposed area, select one or more seed points with non-zero values as the starting points, and mark them as visited; ② Starting from the current seed point, search for pixels in the eight neighborhoods of upper left, upper, upper right, left, right, lower left, lower, and lower right in turn. If the pixel has not been visited and its value is not zero, mark it as visited and add it to the current connected domain; ③ Update the label values of all pixels in the current connected domain to the same label value as the seed point; ④ Repeat ② and ③ until all pixels have been visited.
3. The automatic multi-exposure selection method based on fringe projection according to claim 1, characterized in that: Step 3 Iterate and calculate the exposure time according to the number of pixels and average brightness of the largest overexposed sub-region to obtain the optimal initial exposure time, so that the initial exposure is as large as possible and there is no obvious overexposed area in the measured scene under this exposure. The specific method is: Find the sub-region with the largest number of pixels and determine whether the number of pixels in this region is less than the set threshold value P1; if it exceeds the threshold, it is considered that there is an overexposed area in the image under the current exposure t0, and the overexposed area is assigned to the variable satu_mask in the form of a binary mask image, and t0 is halved and returned to step 1; if it does not exceed the threshold, determine whether the variable satu_mask has been assigned a value. If satu_mask has not been assigned a value, it means that the highlight area of the measured scene has not been recorded, then t0 is doubled and returned to step 1; if satu_mask has been assigned a value, the average brightness Avg of the first P1 points in the highlight area satu_mask according to the pseudo-white image under the current exposure time t0 I Calculate the optimal initial exposure time t1.
4. The automatic multi-exposure selection method based on fringe projection according to claim 3, characterized in that: The optimal initial exposure time t1 is specifically: in, Indicates the overexposure threshold of image light intensity.
5. The automatic multi-exposure selection method based on fringe projection according to claim 1, characterized in that: The subsequent exposure time is calculated as: in, The exposure time is t n When the modulation index is less than and closest to the modulation index threshold I t "; n = 1, 2, ..., M, where M is a positive integer representing the total number of exposures; Indicates the overexposure threshold of image light intensity.
6. The automatic multi-exposure selection method based on fringe projection according to claim 5, characterized in that: The total number of exposures is calculated as: In the formula, I” min Represents the light intensity threshold of the shadow area under the initial exposure. Points less than this value are considered to be in the shadow area. Indicates that when the exposure time is t1, the modulation is less than and closest to the modulation threshold I t "The light intensity of the pixel, It represents the overexposure threshold of the image light intensity, and floor() represents the rounding down function.
7. The automatic multi-exposure selection method based on fringe projection according to claim 1, characterized in that: The absolute phase Φ after fusion is specifically: In the formula, Φ i is the absolute phase obtained from the i-th exposure, A i The absolute phase of the i-th exposure is greater than the threshold I t The mask corresponding to the area ”.
Citation Information
Patent Citations
Light-emitting surface microscopic three-dimensional measurement method based on multi-frequency phase shift scheme
CN111473745A
High reflective surface three-dimensional reconstruction algorithm based on structured light
CN114820812A
Self-adaptive multiple exposure method for three-dimensional topography measurement
CN116614713A
Self-adaptive fringe method for detecting saturation point by using high-frequency signal
CN116824069A
Structured light three-dimensional reconstruction method and device based on adaptive stripes
CN117078860A
Cited By
Polarization dislocation code fused phase shift coding method and system
CN120043468A
