A speckle matching method and a three-dimensional measurement method, apparatus and system based thereon.

The speckle matching method based on dynamic lateral stretching and ZNCC matching cost calculation solves the problems of complex calculation and low accuracy in speckle matching, and realizes efficient and high-precision three-dimensional measurement.

CN117516407BActive Publication Date: 2026-11-17XI AN ZHISENSOR TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210886755.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-07-26
Publication Date
2026-11-17
Estimated Expiration
2042-07-26

AI Technical Summary

Technical Problem

Existing speckle matching methods are computationally complex and have low matching accuracy, especially when the surface morphology of the object being measured is complex, resulting in large errors.

Method used

By processing the relative phase maps of cameras A and B, calculating the phase width of each pixel, and performing speckle matching through dynamic horizontal stretching of the matching window, combined with ZNCC matching cost calculation, the matching accuracy and efficiency are improved.

Benefits of technology

It achieves high-precision speckle matching under complex surface morphology conditions, simplifies the calculation process, and improves matching speed and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117516407B_ABST
    Figure CN117516407B_ABST
Patent Text Reader

Abstract

This invention relates to a speckle matching method and a three-dimensional measurement method, apparatus, and system based thereon, overcoming the problems of computational complexity and low matching accuracy in existing speckle matching methods. First, for each measured pixel in camera A, multiple matching pixels corresponding to each pixel to be matched in camera B are calculated, and the scale value corresponding to each pixel to be matched is calculated. Then, the initial matching window of each measured pixel in camera A is successively stretched laterally by a scale factor to obtain stretched matching windows corresponding to each measured pixel and each pixel to be matched. Next, in the speckle pattern of camera B, the pixel window corresponding to each pixel to be matched is extracted as the matching window. The ZNCC matching cost is calculated using the multiple stretched matching windows of each measured pixel in camera A and the matching windows of the corresponding pixels to be matched in camera B to complete the speckle matching. This algorithm is simple and can achieve high-precision measurement even for objects with complex surface morphology.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of optical three-dimensional measurement, specifically relating to a speckle matching method and a three-dimensional measurement method, device and system based thereon. Background Technology

[0002] Structured light 3D measurement technology is an optical non-contact 3D measurement method with the advantages of high speed, high accuracy, and no contact with the object being measured. It has been widely used in fields such as face recognition, online inspection, package sorting, intelligent manufacturing, and medical imaging.

[0003] Structured light systems typically consist of a projector and a camera. During operation, the projector projects a pre-programmed structured light pattern onto the surface of an object, and the camera captures and decodes the corresponding photograph to obtain an absolute phase map. Subsequently, spatial constraint equations are established to solve for the three-dimensional shape of the object being measured.

[0004] Structured light coding is the core of structured light 3D measurement methods. The number of patterns and decoding logic in structured light coding affect the system's measurement time and accuracy. Conventional structured light coding methods include Gray code, sinusoidal phase-shift code, and speckle. Gray code offers high decoding accuracy and is unambiguous, but requires a large number of patterns and has a slow measurement speed. Sinusoidal phase-shift code requires fewer patterns and has a fast measurement speed, but suffers from periodic ambiguity. The speckle method requires only one pattern for global decoding, but has low decoding accuracy and a complex matching process. Due to its high decoding accuracy and speed, sinusoidal phase-shift code is often used in fast measurement systems. Conventional sinusoidal phase-shift code uses a three-step or four-step phase-shift method for image encoding. After decoding, a relative phase map with periodic ambiguity is obtained. Further methods are used to eliminate the periodic ambiguity to obtain the absolute phase map.

[0005] Commonly used methods for eliminating periodic ambiguity include Gray code, multi-frequency heterodyne, and speckle matching. Among these, speckle matching requires the fewest images, needing only one to complete global matching and eliminate periodic ambiguity. However, speckle matching requires complex computational steps such as matching cost calculation and cost aggregation, and the matching accuracy depends on the surface morphology of the object being measured. When the surface morphology of the object being measured is too complex and the angle changes significantly, the matching error worsens. Summary of the Invention

[0006] The purpose of this invention is to provide a speckle matching method and system, as well as a three-dimensional measurement method, device and system based on the speckle matching method, to overcome the problems of computational complexity and low matching accuracy of existing speckle matching methods.

[0007] The technical solution of the present invention is to provide a speckle matching method, which is characterized by including the following steps:

[0008] Step 1: Process the relative phase maps of camera A and camera B to obtain the phase width of each pixel in camera A and camera B;

[0009] Step 2: For each measured pixel in camera A, calculate the scale = w2 / w1 corresponding to the multiple matching pixels in camera B. Here, w2 is the phase width of each matching pixel in camera B, w1 is the phase width of the measured pixel in camera A, and the scale value indicates that the speckle pattern of camera A at this pixel needs to be stretched horizontally by a scale factor to match the speckle pattern of the matching pixel in camera B.

[0010] Step 3: For each measured pixel in camera A, extract a pixel window with a pixel radius of R from the speckle pattern of camera A as the initial matching window for that point. Based on the different scale values ​​of each measured pixel in camera A and the multiple matching pixels corresponding to that point in camera B, successively stretch the initial matching window of each measured pixel by a horizontal scale factor to obtain the stretched matching window corresponding to each measured pixel and each matching pixel.

[0011] Step 4: In the speckle pattern of camera B, take the multiple unmatched pixels corresponding to each measured pixel in camera A as the center, and R as the pixel radius, and extract the pixel window corresponding to each unmatched pixel as the matching window.

[0012] Step 5: Using multiple stretch matching windows of each measured pixel in camera A (each stretch matching window corresponds one-to-one with the pixel to be matched), calculate the zncc matching cost with the matching window of the corresponding pixel to be matched in camera B. Take the phase order K corresponding to the maximum zncc as the phase order K corresponding to each measured pixel in camera A to complete speckle matching.

[0013] The present invention also provides a three-dimensional measurement method based on the above-mentioned speckle matching method, which is characterized by including the following steps:

[0014] Step 1: Decode the stripe pattern modulated by the object being measured, captured simultaneously by camera A and camera B, to obtain the relative phase maps corresponding to camera A and camera B respectively; Based on the relative phase maps corresponding to camera A and camera B, determine the multiple matching pixels in camera B for each measured pixel in camera A.

[0015] Step 2: Perform speckle matching calculations on each measured pixel in camera A and the corresponding multiple matching pixels in camera B to obtain the phase order corresponding to the matching pixel with the highest matching degree, and then obtain the absolute phase map. Determine the spatial coordinates of the measured point based on the absolute phase map.

[0016] Step 2.1: Process the relative phase map of camera A and camera B to obtain the phase width of each pixel in camera A and camera B;

[0017] Step 2.2: For each measured pixel in camera A, calculate the corresponding multiple matching pixels in camera B. For each matching pixel, calculate scale = w2 / w1, where w2 is the phase width of each matching pixel in camera B, w1 is the phase width of the measured pixel in camera A, and the scale value indicates that the speckle pattern of camera A at this pixel needs to be stretched horizontally by a scale factor to match the speckle pattern of the matching pixel in camera B.

[0018] Step 2.3: For each measured pixel in camera A, extract a pixel window with a pixel radius of R from the speckle pattern of camera A as the initial matching window for that point. Based on the different scale values ​​of each measured pixel in camera A and the multiple matching pixels corresponding to that point in camera B, successively stretch the initial matching window of each measured pixel by a horizontal scale factor to obtain the stretched matching window corresponding to each measured pixel and each matching pixel.

[0019] Step 2.4: In the speckle pattern of camera B, take the multiple unmatched pixels corresponding to each measured pixel in camera A as the center, and R as the pixel radius, and extract the pixel window corresponding to each unmatched pixel as the matching window.

[0020] Step 2.5: Using multiple stretch matching windows of each measured pixel in camera A (each stretch matching window corresponds one-to-one with the pixel to be matched), calculate the ZNCC matching cost with the matching window of the corresponding pixel to be matched in camera B. Find the K corresponding to the maximum ZNCC, which is the order K corresponding to each measured pixel in camera A. Then obtain the absolute phase map and determine the spatial coordinates of the measured point based on the absolute phase map.

[0021] Furthermore, step 1 specifically includes:

[0022] Step 1.1: Decode the stripe pattern modulated by the object being measured and captured simultaneously by camera A and camera B to obtain the relative phase diagrams corresponding to camera A and camera B respectively.

[0023] Step 1.2: Based on the relative phase map of camera A, determine the pixel point to be measured in camera A;

[0024] Step 1.3: Based on the calibration parameters of camera A and stripe projector, calculate the spatial coordinates of the corresponding points of each measured pixel in camera A under each phase order of the stripe projector.

[0025] Step 1.4: Using the calibration parameters between the two cameras, for each measured pixel, project its corresponding point at each phase order of the stripe projector onto camera B to obtain the pixel position of these corresponding points in camera B, that is, obtain multiple unmatched pixel points of the measured pixel in camera A in camera B.

[0026] Furthermore, in step 1.2, all points falling within the measurement range of camera A are taken as the measured pixels.

[0027] Furthermore, the stripe pattern mentioned in step 1 is a phase-coded pattern projected onto the object under test by a stripe projector.

[0028] Furthermore, the aforementioned phase encoding pattern is a four-step phase shift code.

[0029] To improve the speed of speckle matching, this invention also provides another speckle matching method, which is characterized by including the following steps:

[0030] Step 1: Determine the points for speckle matching calculation by sampling at intervals;

[0031] Step 1.1: Treat each continuously distributed phase region of the same film in the relative phase diagram of camera A as a connected component, and mark the connected components in the relative phase diagram of camera A.

[0032] Step 1.2: Statistically count the pixel positions of each connected component, sample the pixels at intervals to obtain a set of downsampled points, and use the set of downsampled points corresponding to each connected component as the pixel points in that connected component that need to be calculated for speckle matching, and define them as the measured pixel points.

[0033] Step 2: Perform speckle matching on the measured pixels in each connected region in turn to obtain the order K corresponding to each measured pixel in each connected region.

[0034] Step 2.1: Process the relative phase map of camera A and camera B to obtain the phase width of each pixel in camera A and camera B;

[0035] Step 2.2: For the multiple matching pixels in camera B corresponding to the measured pixels in each connected region of camera A, calculate the scale = w2 / w1 for each matching pixel, where w2 is the phase width of each matching pixel in camera B corresponding to the measured pixels in each connected region of camera A, w1 is the phase width of the measured pixels in each connected region of camera A, and the scale value indicates that the speckle pattern of camera A at this pixel needs to be stretched horizontally by a scale factor to match the speckle pattern of the matching pixel in camera B.

[0036] Step 2.3: For each measured pixel in each connected region of camera A, extract a pixel window with a pixel radius of R from the speckle pattern of camera A as the initial matching window for that point. Based on the different scale values ​​of each measured pixel in each connected region of camera A and the multiple matching pixels corresponding to that point in camera B, successively stretch the initial matching window of each measured pixel by a horizontal scale factor to obtain the stretched matching window corresponding to each measured pixel and each matching pixel.

[0037] Step 2.4: In the speckle pattern of camera B, take the multiple unmatched pixels corresponding to each measured pixel in each connected region of camera A as the center, and R as the pixel radius, and extract the pixel window corresponding to each unmatched pixel as the matching window.

[0038] Step 2.5: Using multiple stretched matching windows of each measured pixel in each connected component of camera A (each stretched matching window corresponds one-to-one with the pixel to be matched), perform ZNCC matching cost calculation with the matching window of the corresponding pixel to be matched in camera B to obtain the order K corresponding to each measured pixel in each connected component.

[0039] Step 3: Set the phase order K with the most occurrences in each connected component as the phase order of that connected component, and set the order of all other pixels in that connected component to K to complete speckle matching.

[0040] The present invention also provides a three-dimensional measurement method for the above-mentioned speckle matching method, which is characterized by including the following steps:

[0041] Step 1: Decode the stripe pattern modulated by the object being measured and captured simultaneously by camera A and camera B to obtain the relative phase diagrams corresponding to camera A and camera B respectively.

[0042] Step 2: Treat each continuously distributed phase region of the same film in the relative phase diagram of camera A as a connected component, and mark the connected components in the relative phase diagram of camera A.

[0043] Step 3: Statistically count the pixel positions of each connected component, sample the pixels at intervals to obtain a set of downsampled points, and use the set of downsampled points corresponding to each connected component as the set of pixels in that connected component that need to be calculated for speckle matching, and define it as the pixel to be measured.

[0044] Step 4: Perform speckle matching on the measured pixels in each connected region in turn to obtain the phase order K corresponding to each measured pixel in each connected region.

[0045] Step 4.1: Process the relative phase map of camera A and camera B to obtain the phase width of each pixel in camera A and camera B;

[0046] Step 4.2: For the multiple matching pixels in camera B corresponding to the measured pixels in each connected region of camera A, calculate the scale = w2 / w1 for each matching pixel, where w2 is the phase width of each matching pixel in camera B corresponding to the measured pixels in each connected region of camera A, w1 is the phase width of the measured pixels in each connected region of camera A, and the scale value indicates that the speckle pattern of camera A at this pixel needs to be stretched horizontally by a scale factor to match the speckle pattern of the matching pixel in camera B.

[0047] Step 4.3: For each measured pixel in each connected region of camera A, extract a pixel window with a pixel radius of R from the speckle pattern of camera A as the initial matching window for that point. Based on the different scale values ​​of each measured pixel in each connected region of camera A and the multiple matching pixels corresponding to that point in camera B, successively stretch the initial matching window of each measured pixel by a horizontal scale factor to obtain the stretched matching window corresponding to each measured pixel and each matching pixel.

[0048] Step 4.4: In the speckle pattern of camera B, take the multiple unmatched pixels corresponding to each measured pixel in each connected region of camera A as the center, and R as the pixel radius, and extract the pixel window corresponding to each unmatched pixel as the matching window.

[0049] Step 4.5: Using multiple stretched matching windows of each measured pixel in each connected component in camera A (each stretched matching window corresponds one-to-one with the pixel to be matched), perform ZNCC matching cost calculation with the matching window of the corresponding pixel to be matched in camera B to obtain the order K corresponding to each measured pixel in each connected component.

[0050] Step 5: Set the phase order K with the most occurrences in each connected component as the phase order of that connected component, and set the phase order of all other pixels in that connected component to K. Based on the phase order K corresponding to all pixels in camera A, obtain the absolute phase map, and determine the spatial coordinates of the measured point based on the absolute phase map.

[0051] Further, the multiple matching pixels in camera B corresponding to the measured pixels in each connected component of camera A in step 4.2 are determined through the following steps:

[0052] Step a: Based on the calibration parameters of camera A and stripe projector, calculate the spatial coordinates of the corresponding points of the measured pixels in each connected region of camera A at each phase order of the stripe projector.

[0053] Step b: Using the calibration parameters between the two cameras, for each measured pixel, project its corresponding point at each phase order of the stripe projector onto camera B to obtain the pixel position of these corresponding points in camera B, that is, obtain multiple unmatched pixel points of the measured pixel in camera A in camera B.

[0054] Furthermore, the stripe pattern mentioned in step 1 is a phase-coded pattern projected onto the object under test by a stripe projector.

[0055] Furthermore, the phase encoding pattern is a four-step phase shift code.

[0056] The present invention also provides a three-dimensional measuring device, which is characterized in that it includes a stripe projector, a speckle projector, a camera A, a camera B, and a computer;

[0057] The stripe projector is used to project stripe patterns onto the object being measured;

[0058] The speckle projector is used to project a speckle pattern onto the object being measured;

[0059] Cameras A and B are used to capture the modulated stripe and speckle patterns of the object under test.

[0060] The computer stores a computer program, which, when executed by a processor, implements the steps of the aforementioned rapid three-dimensional measurement method based on phase shifting and speckle projection.

[0061] The present invention also provides a three-dimensional measurement system, including a memory and a processor, wherein the memory stores a computer program, and the special feature is that when the computer program is executed by the processor, it implements the steps of the above-mentioned three-dimensional measurement method.

[0062] The present invention also provides a speckle matching system, including a memory and a processor, wherein the memory stores a computer program, and the special feature is that when the computer program is executed by the processor, it implements the steps of the above-described speckle matching method.

[0063] The beneficial effects of this invention are:

[0064] 1. The three-dimensional measurement method of the present invention is based on the speckle matching method of dynamic lateral stretching. This method obtains the lateral stretching coefficient corresponding to each pixel point based on other encoding methods (such as sinusoidal phase shift code). The accuracy of speckle matching is improved by dynamically stretching the image laterally. The algorithm is simple. In addition, it can also achieve high-precision measurement for objects with complex surface morphology.

[0065] 2. The three-dimensional measurement method of the present invention can be based on the accelerated speckle matching method to complete the matching with high matching efficiency. This method determines the sub-regions that need to be matched based on other encoding methods (such as sinusoidal phase shift codes), then performs matching calculations by interval sampling, and obtains the final sub-region phase order K through region voting, thereby improving the efficiency of speckle matching. Attached Figure Description

[0066] Figure 1 This is a schematic diagram of the three-dimensional measuring device in Example 1;

[0067] The attached diagram is labeled as follows: 1-Camera A, 2-Camera B, 3-Striped projector, 4-Pitch projector;

[0068] Figure 2 This refers to the four-step phase shift code pattern in Example 1;

[0069] Figure 3 The encoding pattern of camera A in Example 1;

[0070] Figure 4 This is the relative phase diagram of camera B in Example 1;

[0071] Figure 5 This is a speckle pattern of camera A in Example 1. Detailed Implementation

[0072] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0073] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0074] Example 1

[0075] The three-dimensional measuring device proposed in this embodiment is as follows: Figure 1 As shown, the system consists of camera A1, camera B2, stripe projector 3, speckle projector 4, and a computer (not shown). Cameras A1 and B2 have a resolution of 1024*1280. Stripe projector 3 projects stripe patterns with a resolution of 1024*1280. Speckle projector 4 projects fixed speckle patterns with a field of view of 50°*50°. Camera A1 and stripe projector 3 are calibrated, and the calibration parameters are denoted as para1. Camera A1 and camera B2 are calibrated, and the conversion parameters between them are denoted as para2.

[0076] This embodiment uses a four-step phase shift code as the phase encoding scheme. Figure 2 This displays the first pattern of a four-step phase-shift code, with a phase period of 16 pixels and a total encoded phase count of 1024 pixels. Figure 2 By shifting the code 4 pixels to the right sequentially, the other three coded patterns of the four-step phase shift code can be obtained. The image after projecting the coded patterns onto the surface of the object being measured using a stripe projector 3 is shown below. Figure 3 As shown.

[0077] Using a stripe projector 3, the four-step phase shift code is sequentially projected onto the surface of the object under test. By taking photographs using cameras A and B, phase deconstruction can be performed to obtain the relative phase maps S_Phase_A and S_Phase_B corresponding to the two cameras. The relative phase map of camera A is shown below. Figure 4 As shown, the phase resolution formula is:

[0078] Phase=(arctan((I_4-I_2) / (I_3-I_1))+π)*8 / π

[0079] Wherein: I_1, I_2, I_3 and I_4 represent the 1st to 4th phase-shifted images, respectively.

[0080] Then, the speckle pattern is projected onto the surface of the object being measured using speckle projector 4, and corresponding photos are taken using cameras A1 and B2. The speckle photo taken by camera A1 is shown below. Figure 5 As shown.

[0081] Before performing speckle matching, it is necessary to determine the measured pixels in camera A1 and the multiple matching pixels in camera B2:

[0082] Based on the relative phase map of camera A1, the measured pixels in camera A1 are determined; all points falling within the measurement range of camera A1 are taken as measured pixels. According to the calibration parameters of camera A1 and stripe projector 3, the spatial coordinates of the corresponding points of each measured pixel in camera A1 at each phase order of stripe projector 3 are calculated; using the calibration parameters between the two cameras, for each measured pixel, its corresponding points at each phase order of stripe projector 3 are projected onto camera B2 to obtain the pixel positions of these corresponding points in camera B2, that is, to obtain multiple matching pixels of the measured pixels in camera A1 in camera B2.

[0083] Next, speckle matching will be performed:

[0084] Speckle matching calculations are performed on each measured pixel in camera A1 and multiple matching pixels in camera B2 to obtain the phase order corresponding to the matching pixel with the highest matching degree, and then an absolute phase map is obtained. The spatial coordinates of the measured point are determined based on the absolute phase map.

[0085] First, the relative phase maps of camera A1 and camera B2 are processed to obtain the phase width of each pixel in camera A1 and camera B2.

[0086] Secondly, for each measured pixel in camera A1, the corresponding multiple matching pixels in camera B2 are calculated. The scale = w2 / w1 is calculated for each matching pixel, where w2 is the phase width of each matching pixel in camera B2, w1 is the phase width of the measured pixel in camera A1, and the scale value indicates that the speckle pattern of camera A1 at this pixel needs to be stretched horizontally by a scale factor to match the speckle pattern of the matching pixel in camera B2.

[0087] Secondly, for each measured pixel in camera A1, a pixel window with a pixel radius of R is extracted from the speckle pattern of camera A1 with that point as the center, and used as the initial matching window for that point; based on the different scale values ​​of each measured pixel in camera A1 and the multiple matching pixels corresponding to that point in camera B2, the initial matching window of each measured pixel is successively stretched horizontally by a scale factor to obtain the stretched matching window corresponding to each measured pixel and each matching pixel;

[0088] Finally, in the speckle pattern of camera B2, taking the multiple unmatched pixels corresponding to each measured pixel in camera A1 as the center and R as the pixel radius, the pixel window corresponding to each unmatched pixel is extracted as the matching window; using the multiple stretched matching windows of each measured pixel in camera A1 (each stretched matching window corresponds one-to-one with the unmatched pixel), the zncc matching cost is calculated with the matching window of the corresponding unmatched pixel in camera B2, and the phase order K corresponding to the maximum zncc is found, which is the phase order K corresponding to each measured pixel in camera A1, thereby obtaining the absolute phase map, and determining the spatial coordinates of the measured point based on the absolute phase map.

[0089] Example 2

[0090] The three-dimensional measurement device proposed in this embodiment is the same as that in Embodiment 1, such as... Figure 1 As shown, the system consists of camera A1, camera B2, stripe projector 3, speckle projector 4, and a computer (not shown). Cameras A1 and B2 have a resolution of 1024*1280. Stripe projector 3 projects stripe patterns with a resolution of 1024*1280. Speckle projector 4 projects fixed speckle patterns with a field of view of 50°*50°. Camera A1 and stripe projector 3 are calibrated, and the calibration parameters are denoted as para1. Camera A1 and camera B2 are calibrated, and the conversion parameters between them are denoted as para2.

[0091] This embodiment uses a four-step phase shift code as the phase encoding scheme. Figure 2 This displays the first pattern of a four-step phase-shift code, with a phase period of 16 pixels and a total encoded phase count of 1024 pixels. Figure 2 By shifting the code 4 pixels to the right sequentially, the other three coded patterns of the four-step phase shift code can be obtained. The image after projecting the coded patterns onto the surface of the object being measured using a stripe projector 3 is shown below. Figure 3 As shown.

[0092] Using a stripe projector 3, the four-step phase shift code is sequentially projected onto the surface of the object under test. Photos are then taken using cameras A1 and B2, allowing for phase deconstruction and the acquisition of the relative phase maps S_Phase_A and S_Phase_B corresponding to the two cameras. The relative phase map of camera A1 is shown below. Figure 4 As shown, the phase resolution formula is:

[0093] Phase=(arctan((I_4-I_2) / (I_3-I_1))+π)*8 / π

[0094] Wherein: I_1, I_2, I_3 and I_4 represent the 1st to 4th phase-shifted images, respectively.

[0095] The relative phase map consists of 64 sub-phase regions, with a phase distribution of S_Phase = 1 to 16 within each sub-phase region. The phase order of each sub-phase region is K = 0 to 63, and the absolute phase A_Phase_A of each sub-phase region can be expressed as A_Phase_A = S_Phase_A + 16 * K. It should be noted that this embodiment uses a four-step phase shift method, and the 16-pixel period is only used to illustrate the implementation steps. In practical applications, other phase shift steps and periods can still be used with the method proposed in this invention.

[0096] Then, the speckle pattern is projected onto the surface of the object being measured using speckle projector 4, and corresponding photos are taken using cameras A1 and B2. The speckle photo taken by camera A1 is shown below. Figure 5 As shown.

[0097] Next, speckle matching is performed. First, the points that need to be calculated for speckle matching are determined through interval sampling:

[0098] 1) Perform image processing on the relative phase map of camera A1, treating each continuously distributed phase region of the same film as a connected component, and label the connected components on the relative phase map of camera A1.

[0099] 2) Statistically analyze the pixels in each connected component, and sample the pixels at intervals to obtain a downsampled point set P_sub. The interval number is set to 10 (in other embodiments, this interval number can be set according to the actual desired acceleration factor; for example, if the acceleration is one time, the interval number is set to 1; if the acceleration is N times, the interval number is set to N). P_sub represents the set of pixels in the sub-th connected component that need to be speckle matched, which is the set of pixels to be measured.

[0100] 3) Perform speckle matching on the pixels in P_sub corresponding to each connected component in turn, and set the phase order K with the largest number of pixels as the order of the connected component, and set the order of other pixels in the connected component to K.

[0101] The above steps can be used to quickly complete speckle matching. The specific process for speckle matching in step 3) is as follows:

[0102] Step 3.1: For the measured pixel P_sub in camera A1, substitute K = 0 to 63 into A_Phase_A = S_Phase_A + 16*K, and calculate the spatial coordinates Point_(0 to 63) corresponding to K = 0 to 63 using the calibration parameter para1. Project the points falling within the measurement range of camera A1 onto the image of camera B2 to obtain the corresponding pixel coordinates Point_pixel_(1 to n), which are the multiple unmatched pixels in camera B2 for each measured pixel in camera A1; where n represents the number of points falling within the measurement range of camera A1.

[0103] Step 3.2: Process the relative phase map of camera A1 and camera B2 to obtain the phase width of each pixel in camera A1 and camera B2;

[0104] Step 3.3: For the multiple unmatched pixels in camera B2 corresponding to the measured pixels in each connected region of camera A1, calculate the scale (1~n) = w2 / w1 for each unmatched pixel, where w2 is the phase width of each unmatched pixel in camera B2 corresponding to the measured pixels in each connected region of camera A1, w1 is the phase width of the measured pixels in each connected region of camera A1, and the scale value indicates that the speckle pattern of camera A1 at this pixel needs to be stretched horizontally by a scale factor to match the speckle pattern of the unmatched pixel in camera B2;

[0105] Step 3.4: For each measured pixel in each connected region of camera A1, extract a pixel window with a pixel radius of 15 from the speckle pattern of camera A1, centered on that point, as the initial matching window for that point; based on the different scale values ​​of each measured pixel in each connected region of camera A1 and the multiple unmatched pixels corresponding to that point in camera B2, successively stretch the initial matching window of each measured pixel by a horizontal scale factor to obtain the stretched matching window corresponding to each measured pixel and each unmatched pixel;

[0106] Step 3.5: In the speckle pattern of camera B2, take the multiple unmatched pixels corresponding to each measured pixel in each connected region of camera A1 as the center, and with a pixel radius of 15, extract the pixel window corresponding to each unmatched pixel as the matching window.

[0107] Step 3.6: Using multiple stretched matching windows of each measured pixel in each connected region of camera A1 (each stretched matching window corresponds one-to-one with the pixel to be matched), and the matching window size of the corresponding pixel to be matched in camera B2 is 15*15, calculate the ZNCC matching cost to obtain the phase order K corresponding to each measured pixel in each connected region.

[0108] Step 4: Set the phase order K with the most occurrences in each connected component as the phase order of that connected component, and set the phase order of all other pixels in that connected component to K.

[0109] After obtaining the phase order K corresponding to each pixel, the absolute phase map can be obtained using a formula. Then, the three-dimensional coordinates corresponding to each pixel can be calculated using the calibration parameters of the structured light system. The calculation process from the absolute phase map and calibration parameters to the three-dimensional coordinates is knowledge that those in the industry should understand and be familiar with, and will not be described in detail here. It should be noted that in the above embodiments, the selected number of projector pixels, four-step phase shift code, and other specific parameters such as speckle matching window size and interval sampling size are only for better describing the implementation of the present invention. In practical applications, selecting cameras, projectors, encoding schemes and parameters of other resolutions, or adjusting the relative positions of the camera, projector, and speckle projector 4, will not affect the effect of the dynamic lateral stretching method and connected component voting method proposed in this invention.

Claims

1. A three-dimensional measurement method based on speckle matching, characterized in that: Step 1: Determine the multiple matching pixels in camera B for each pixel being measured in camera A; specifically: Step 1.1: Decode the stripe pattern modulated by the object under test, which is simultaneously captured by camera A and camera B, to obtain the relative phase diagrams corresponding to camera A and camera B respectively; the stripe pattern is a phase-coded pattern projected onto the object under test by a stripe projector. Step 1.2: Based on the relative phase map of camera A, determine the pixel point to be measured in camera A; Step 1.3: Based on the calibration parameters of camera A and stripe projector, calculate the spatial coordinates of the corresponding points of each measured pixel in camera A under each phase order of the stripe projector. Step 1.4: Using the calibration parameters between the two cameras, for each measured pixel, project its corresponding point at each phase order of the stripe projector onto camera B to obtain the pixel position of these corresponding points in camera B, that is, obtain multiple unmatched pixel points of the measured pixel in camera A in camera B. Step 2: Perform speckle matching calculations on each measured pixel in camera A and the corresponding multiple matching pixels in camera B; Step 2.1: Process the relative phase map of camera A and camera B to obtain the phase width of each pixel in camera A and camera B; Step 2.2: For each measured pixel in camera A, calculate the corresponding multiple matching pixels in camera B for each matching pixel, where w2 is the phase width of the measured pixel in camera A and w1 is the phase width of the measured pixel in camera A. The scale value indicates that the speckle pattern of camera A at this pixel needs to be stretched horizontally by a scale factor to match the speckle pattern of the matching pixel in camera B. Step 2.3: For each measured pixel in camera A, extract a pixel window with a pixel radius of R from the speckle pattern of camera A as the initial matching window for that point. Based on the different scale values ​​of each measured pixel in camera A and the multiple matching pixels corresponding to that point in camera B, successively stretch the initial matching window of each measured pixel by a horizontal scale factor to obtain the stretched matching window corresponding to each measured pixel and each matching pixel. Step 2.4: In the speckle pattern of camera B, take the multiple unmatched pixels corresponding to each measured pixel in camera A as the center, and R as the pixel radius, and extract the pixel window corresponding to each unmatched pixel as the matching window. Step 2.5: Using multiple stretched matching windows of each measured pixel in camera A, calculate the ZNCC matching cost with the matching window of the corresponding pixel to be matched in camera B. Take the phase order K corresponding to the maximum ZNCC as the phase order K of each measured pixel in camera A, and then obtain the absolute phase map. Determine the spatial coordinates of the measured point based on the absolute phase map.

2. The method of three-dimensional measurement based on speckle matching method according to claim 1, characterized in that: In step 1.2, all points falling within the measurement range of camera A are taken as the measured pixels.

3. The method of three-dimensional measurement based on speckle matching method according to claim 2, characterized in that: The phase encoding pattern is a four-step phase shift code.

4. A three-dimensional measurement method based on speckle matching, characterized in that: Step 1: Decode the stripe pattern modulated by the object under test, which is simultaneously captured by camera A and camera B, to obtain the relative phase diagrams corresponding to camera A and camera B respectively; the stripe pattern is a phase-coded pattern projected onto the object under test by a stripe projector. Step 2: Treat each continuously distributed phase region of the same film in the relative phase diagram of camera A as a connected component, and mark the connected components in the relative phase diagram of camera A. Step 3: Statistically count the pixel positions of each connected component, sample the pixels at intervals to obtain a set of downsampled points; use the set of downsampled points corresponding to each connected component as the pixels in that connected component that need to be used for speckle matching calculation, and define them as the pixels to be measured. Step 4: Perform speckle matching on the measured pixels in each connected region in turn to obtain the phase order K corresponding to each measured pixel in each connected region. Step 4.1: Process the relative phase map of camera A and camera B to obtain the phase width of each pixel in camera A and camera B; Step 4.2: For the measured pixel points in each connected region of camera A, calculate the scale = w2 / w1 corresponding to the multiple matching pixels in camera B for each matching pixel point. Here, w2 is the phase width of the measured pixel point in each connected region of camera A in camera B, w1 is the phase width of the measured pixel point in each connected region of camera A, and the scale value indicates that the speckle pattern of camera A at that pixel point needs to be horizontally stretched by a scale factor to match the speckle pattern of the matching pixel point in camera B. The following steps determine the multiple matching pixels corresponding to the measured pixel points in each connected region of camera A in camera B: Step a: Based on the calibration parameters of camera A and stripe projector, calculate the spatial coordinates of the corresponding points of the measured pixels in each connected region of camera A at each phase order of the stripe projector. Step b: Using the calibration parameters between the two cameras, for each measured pixel, project its corresponding point at each phase order of the stripe projector onto camera B to obtain the pixel position of these corresponding points in camera B, that is, obtain multiple unmatched pixels of the measured pixel in camera A in camera B; Step 4.3: For each measured pixel in each connected region of camera A, extract a pixel window with a pixel radius of R in the speckle pattern of camera A as the initial matching window of that point; Based on the different scale values ​​of each measured pixel in each connected region of camera A and the multiple unmatched pixels corresponding to that point in camera B, successively stretch the initial matching window of each measured pixel by a horizontal scale factor to obtain the stretched matching window corresponding to each measured pixel and each unmatched pixel; Step 4.4: In the speckle pattern of camera B, take the multiple unmatched pixels corresponding to each measured pixel in each connected region of camera A as the center, and R as the pixel radius, and extract the pixel window corresponding to each unmatched pixel as the matching window. Step 4.5: Using multiple stretched matching windows of each measured pixel in each connected region in camera A, perform ZNCC matching cost calculation with the matching window of the corresponding pixel to be matched in camera B to obtain the phase order K corresponding to each measured pixel in each connected region. Step 5: Set the phase order K with the most occurrences in each connected component as the phase order of that connected component, and set the phase order of all other pixels in that connected component to K. Based on the phase order K corresponding to all pixels in camera A, obtain the absolute phase map, and determine the spatial coordinates of the measured point based on the absolute phase map.

5. The speckle matching method based three-dimensional measurement method according to claim 4, characterized in that: The phase encoding pattern is a four-step phase shift code.

6. A three-dimensional measuring device, characterized by: Includes a stripe projector, a speckle projector, camera A, camera B, and a computer; The stripe projector is used to project stripe patterns onto the object being measured; The speckle projector is used to project a speckle pattern onto the object being measured; Cameras A and B are used to capture the modulated stripe and speckle patterns of the object under test. The computer stores a computer program, which, when executed by a processor, implements the steps of the three-dimensional measurement method based on the speckle matching method as described in any one of claims 1 to 5.

7. A three-dimensional measurement system comprising a memory and a processor, the memory having stored therein a computer program, characterized in that: When the computer program is executed by the processor, it implements the steps of the three-dimensional measurement method based on the speckle matching method as described in any one of claims 1 to 5.

8. A speckle matching system comprising a memory and a processor, said memory having stored therein a computer program, characterized in that: When the computer program is executed by the processor, it implements the steps of the three-dimensional measurement method based on the speckle matching method as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • Digital speckle full-field deformation measurement method based on adaptive window matching

    CN109916322A