Frequency-shift structured light measurement method and device

Abstract

The application provides a frequency shift structured light measurement method and device, the method comprises the following steps: projecting a time-delayed transform frequency shift coded image to a measured object by using a projector in a structured light measurement system, and capturing the projected image by using a camera; performing time-delayed transform decoding on the captured projected image to generate an initial matching body; performing threshold processing on the initial matching body to generate a binary matching body; performing anisotropic connectivity detection on the binary matching body, selecting a plurality of regions with the largest connectivity, and obtaining a connected binary matching body; calculating a sub-pixel projector coordinate corresponding to each point with a value of 1 in the connected binary matching body, thereby generating a coordinate triplet containing the sub-pixel projector coordinate corresponding to the point; calculating a point cloud three-dimensional coordinate corresponding to the coordinate triplet by using a triangulation method, thereby completing three-dimensional measurement of the measured object. The application has high solving efficiency and high precision, and can solve the problem that a non-Lambertian body is difficult to measure in a structured light system.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional measurement, and specifically relates to a frequency-shifting structured light measurement method and device. Background Technology

[0002] Transparency and mutual reflection present two major and common challenges to existing structured light measurement methods. First, when measuring such objects, a single camera pixel receives superimposed signals from multiple 3D points, traversing different optical paths involving transmission and reflection; this signal coupling violates the fundamental assumption of direct, single-path light transmission upon which traditional decoding methods (such as phase-shifting methods) rely. Second, the intensity of reflected light from non-Lambertian surfaces is inherently highly dependent on the viewing angle, resulting in significant differences in observed intensity. Therefore, the measurement system must be robust to reflected light intensity, making it exceptionally difficult to simultaneously and accurately measure both dark diffuse reflection regions and bright specular highlights in a single acquisition.

[0003] In theory, frequency-shift structured light measurement can decouple superimposed signals from different 3D points that converge to a single camera pixel. However, the widespread application of this method is limited by several factors: First, the frequency-shift method discards the argument information of the complex sequence output by the discrete Fourier transform, resulting in a loss of coding resolution and accuracy; second, some artifacts in the output point cloud may be connected to the target surface, while existing filtering techniques are only designed for transparency or mutual reflection separately, lacking a framework that can uniformly handle both; third, current sub-pixel thinning algorithms have systematic errors, resulting in insufficient accuracy to meet the needs of practical applications. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a frequency-shifting structured light measurement method and apparatus. This invention features high computational efficiency and high accuracy, and can solve the problem of difficult measurement of non-Lambertian bodies in structured light systems.

[0005] A first aspect of this invention provides a method for measuring frequency-shifted structured light, comprising:

[0006] The projector in the structured light measurement system projects several time-delayed topology transform frequency shift encoded images onto the object under test, and then the camera captures the projected images in sequence.

[0007] The captured projection image is decoded by time-delay transformation to generate the corresponding initial matching body;

[0008] The initial matching body is thresholded to generate a corresponding binary matching body. In the binary matching body, each 0 value represents that the corresponding point is not on the object being tested, and each 1 value represents that the corresponding point is on the object being tested.

[0009] Anisotropic connectivity detection is performed on the binary matching volume, and several regions with the highest connectivity are selected to obtain a connected binary matching volume.

[0010] Using a binary subpixel thinning interpolator, the subpixel projector coordinates corresponding to each point with a value of 1 in the connected binary matching volume are calculated, thereby generating a coordinate triplet containing the subpixel projector coordinates for that point.

[0011] The three-dimensional coordinates of the point cloud corresponding to the coordinate triplet are calculated using triangulation, thereby completing the three-dimensional measurement of the object under test.

[0012] In one specific embodiment of the present invention, it further includes:

[0013] Before projecting several time-delayed transform frequency-shift coded images onto the object under test using the projector in the structured light measurement system, the system is calibrated to obtain the camera intrinsic parameter matrix. Projector intrinsic parameter matrix and extrinsic parameter matrix :

[0014] ,

[0015]

[0016] in, The horizontal focal length of the camera. The vertical focal length of the camera. The position of the camera's horizontal optical center. The longitudinal optical center position of the camera. This is the horizontal focal length of the projector. This refers to the projector's vertical focal length. The horizontal optical center position of the projector. This refers to the longitudinal optical center position of the projector; This represents the value at the corresponding position in the camera-to-projector rotation matrix. ; Let be the translation vector from the camera to the projector.

[0017] In a specific embodiment of the present invention, the step of performing time-extended transform decoding on the captured projection image includes:

[0018] (1) For any pixel in the captured image Extract the light intensity of that point in the captured image, and denote it as a sequence. ;in, These are the x and y coordinates of the camera pixels, respectively; d is the image number; and D is the number of coded images in the projection.

[0019] (2) The sequence obtained in step (1) The extension is a sequence of length 2D-1. The extended portion is supplemented with 0, that is:

[0020]

[0021] (3) The sequence obtained in step (2) Perform a Discrete Fourier Transform or a Fast Fourier Transform to obtain a transform sequence of length 2D-1. ;

[0022] (4) Take each pixel In sequence The first D elements in the corresponding array form a array of size D. The three-dimensional data is denoted as the initial matching body. ,in These represent the number of pixels horizontally and vertically on the camera, respectively.

[0023] In a specific embodiment of the present invention, the thresholding process for the initial matching body includes:

[0024] (1) Initialize a matching body with the initial body Equal-size empty binary three-dimensional data ;

[0025] (2) All of the above satisfy and The point is Set the value at the corresponding position in the middle to 1;

[0026] in, For the initial matching body In the middle The value of the position;

[0027] Then China satisfies The point is The value at the corresponding position is set to 0, where The threshold for the matching body;

[0028] After processing, the final binary matching body is obtained. .

[0029] In a specific embodiment of the present invention, obtaining a connected binary matching volume includes:

[0030] (1) Binary matching volume A cube is formed by the neighborhood of any point in the cube. Six adjacent directions corresponding to the face center of the cube and twelve adjacent directions corresponding to the edges of the cube are selected, for a total of 18 directions.

[0031] These 18 directions are divided into 9 direction types: , , , , , , , , ;in, They represent along The direction reached by moving 1 unit in either the positive or negative direction of the axis. The direction type obtained by any two combinations represents the direction reached by moving 1 unit along the positive or negative directions of the corresponding two axes;

[0032] Will , , , , As a valid direction; if a point lies in a valid direction of another point, then the two points are connected; otherwise, the two points are not connected.

[0033] (2) Binary matching volume The set of points is composed of all points with a value of 1. Initialize the connected component number i = 0;

[0034] in, Binary matching body In the middle The value of the position;

[0035] (3) Initialize an empty set of points ; Select any point in X and add Then delete the point in X;

[0036] (4) Select from X that matches the current value. Connect any point in the middle and add it to the middle. Get the updated current Then continue selecting from X and updating the current... Connect any point in the middle and add it to the middle. until no new points are added to X. In this process, each time a point is selected from X, that point is deleted from X.

[0037] (5) Judgment:

[0038] If X is not an empty set, let the connected component number i = i + 1, and then return to step (3); otherwise, proceed to step (6).

[0039] (6) Initialize a binary matching body Equal-sized empty binary three-dimensional data is denoted as ;from arrive Select the k sets with the largest number of points from all sets, and then sort all the points contained in these k sets. Set the value at the corresponding position in the middle to 1 to get the updated value. As the final connected binary matching body;

[0040] In this context, each 0 value represents that the corresponding point is not on the object being measured, and each 1 value represents that the corresponding point is on the object being measured.

[0041] In a specific embodiment of the present invention, generating the coordinate triplet containing the sub-pixel projector coordinates corresponding to the point includes:

[0042] (1) Define the binary subpixel interpolator ,in, Let r be the output subpixel projector coordinates, and r be the shape parameter, satisfying:

[0043]

[0044] (2) Let the pixel range of the projector be Within this range, the sub-pixel coordinates of the projector Perform equidistant sampling. The maximum horizontal pixel count of the projector;

[0045] (3) Based on the results of step (2), for each Values ​​are used to construct a virtual capture light intensity sequence:

[0046]

[0047] Where L is the unit code length;

[0048] (4) Based on the results of step (3), the sequence The extension is a sequence of length 2D-1. The extended portion is supplemented with 0, that is:

[0049]

[0050] (5) Based on the result of step (4), perform a discrete Fourier transform or a fast Fourier transform on the sequence to obtain a transform sequence of length 2D-1. ;

[0051] (6) Based on the result of step (5), take The first D elements form a sequence of length D. ;

[0052] (7) Based on the result of step (6), the output makes Take the d-coordinate of the maximum value and calculate the corresponding r-value:

[0053]

[0054] Thus generating each Corresponding triples ;

[0055] (8) Based on the results of step (7), utilize all triples Constructing a binary subpixel interpolator ;

[0056] (9) Based on the results of step (8), perform a connected binary matching volume. Each point with a value of 1 Calculate its corresponding r, and then calculate based on the binary pixel interpolator. Thus, the triplet is obtained. .

[0057] In a specific embodiment of the present invention, the step of calculating the three-dimensional coordinates of the point cloud corresponding to the coordinate triplet using triangulation includes:

[0058] (1) The triplet Normalize each coordinate separately:

[0059]

[0060]

[0061]

[0062] (2) Based on the results of step (1), calculate the three-dimensional coordinates of the point cloud. ;

[0063]

[0064]

[0065] .

[0066] A second aspect of the present invention provides a frequency-shifting structured light measurement device, comprising:

[0067] The image projection and acquisition module is used to project several time-delayed topology transform frequency shift encoded images onto the object under test using the projector in the structured light measurement system, and then use a camera to capture the projected images in sequence.

[0068] The time-extended transformation decoding module is used to perform time-extended transformation decoding on the captured projection image to generate the corresponding initial matching body;

[0069] The thresholding module is used to perform thresholding processing on the initial matching body to generate a corresponding binary matching body. In the binary matching body, each 0 value represents that the corresponding point is not on the object being tested, and each 1 value represents that the corresponding point is on the object being tested.

[0070] An anisotropic connectivity detection module is used to perform anisotropic connectivity detection on the binary matching volume, select several regions with the highest connectivity, and obtain a connected binary matching volume.

[0071] The subpixel projector coordinate generation module is used to calculate the subpixel projector coordinates corresponding to each point with a value of 1 in the connected binary matching volume using a binary subpixel thinning interpolator, thereby generating a coordinate triplet containing the subpixel projector coordinates corresponding to that point.

[0072] The point cloud 3D coordinate generation module is used to calculate the point cloud 3D coordinates corresponding to the coordinate triplet using triangulation, thereby completing the 3D measurement of the object under test.

[0073] A third aspect of the present invention provides an electronic device comprising:

[0074] At least one processor; and a memory communicatively connected to said at least one processor;

[0075] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to perform the frequency-shift structured light measurement method described above.

[0076] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to perform the frequency-shift structured light measurement method described above.

[0077] Features and beneficial effects of the present invention:

[0078] This invention, based on a structured light measurement system, achieves high-resolution encoding of projector coordinates using a time-extended transformation-based encoding method. After the camera captures an image, it is decoded using a corresponding time-extended decoding method, mapping the image space to a matching volume space to establish a binary matching volume. Anisotropic connectivity detection within the binary matching volume is then used to obtain object surface points. For each surface point, a binary subpixel thinning interpolator is used to achieve efficient subpixel thinning. Finally, a 3D point cloud is obtained through the mapping relationship between the matching volume and 3D space.

[0079] This invention, based on mature monocular structured light technology, proposes a frequency-shifting structured light encoding method and decoding scheme to address the measurement of objects with internal multiple reflections, especially those with specular reflections or transparent / semi-transparent materials. This invention features high computational efficiency and high accuracy, and can solve the problem of difficult measurement of non-Lambertian bodies in structured light systems. Attached Figure Description

[0080] Figure 1 This is an overall flowchart of a frequency-shift structured light measurement method according to an embodiment of the present invention. Detailed Implementation

[0081] This invention proposes a method and apparatus for measuring frequency-shifted structured light. The invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0082] A first aspect of this invention provides a method for measuring frequency-shifted structured light, comprising:

[0083] The projector in the structured light measurement system projects several time-delayed topology transform frequency shift encoded images onto the object under test, and then the camera captures the projected images in sequence.

[0084] The captured projection image is decoded by time-delay transformation to generate the corresponding initial matching body;

[0085] The initial matching body is thresholded to generate a corresponding binary matching body. In the binary matching body, each 0 value represents that the corresponding point is not on the object being tested, and each 1 value represents that the corresponding point is on the object being tested.

[0086] Anisotropic connectivity detection is performed on the binary matching volume, and several regions with the highest connectivity are selected to obtain a connected binary matching volume.

[0087] Using a binary subpixel thinning interpolator, the subpixel projector coordinates corresponding to each point with a value of 1 in the connected binary matching volume are calculated, thereby generating a coordinate triplet containing the subpixel projector coordinates for that point.

[0088] The three-dimensional coordinates of the point cloud corresponding to the coordinate triplet are calculated using triangulation, thereby completing the three-dimensional measurement of the object under test.

[0089] In a specific embodiment of the present invention, the overall process of the frequency-shift structured light measurement method is as follows: Figure 1 As shown, it includes the following steps:

[0090] (1) Use the projector in the structured light measurement system to project several time-delayed transform frequency shift encoded images onto the object being measured in sequence.

[0091] This embodiment has no special requirements for the object being tested; in one specific embodiment of the present invention, the object being tested is a transparent glass bottle.

[0092] Among them, note:

[0093] ,

[0094] in, coordinates The projection value of the pixel in the d-th coded image. Here, λ represents the x-coordinate and y-coordinate of the projector pixels, d represents the image number, A represents the image amplitude, B represents the image offset, L represents the unit encoding length, and D represents the number of encoded images projected. In a specific embodiment of the present invention, A is 127, B is 127, L is 5, and D is 216.

[0095] In one embodiment of the present invention, a structured light measurement system is constructed using a single projector and single camera system. The camera resolution is 2560×2048, and the projector resolution is 1920×1080. The system is calibrated before use to obtain the camera intrinsic parameter matrix. Projector intrinsic parameter matrix and extrinsic parameter matrix :

[0096] ,

[0097]

[0098] in, The horizontal focal length of the camera. The vertical focal length of the camera. The position of the camera's horizontal optical center. The longitudinal optical center position of the camera. This is the horizontal focal length of the projector. This refers to the projector's vertical focal length. The horizontal optical center position of the projector. This refers to the longitudinal optical center position of the projector; This represents the value at the corresponding position in the camera-to-projector rotation matrix. ; Let be the translation vector from the camera to the projector.

[0099] (2) Use the camera in the structured light measurement system to capture each image projected in step (1) sequentially, denoted as . .in As coordinates The light intensity corresponding to the pixel in the d-th captured image. These are the x and y coordinates of the camera pixels, respectively.

[0100] (3) Perform time-delay transformation decoding on the image captured in step (2), and the decoded data forms a data of size [size missing]. The three-dimensional data is denoted as the initial matching body. .in, These represent the horizontal and vertical pixel counts of the camera, respectively. In one embodiment of the invention, W is 2560 and H is 2048.

[0101] The specific steps are as follows:

[0102] (3-1) For any pixel in the captured image Extract the light intensity of this point in D captured images, and denote it as the sequence. .

[0103] (3-2) The sequence obtained in step (3-1) The extension is a sequence of length 2D-1. The extended portion is supplemented with 0, that is:

[0104]

[0105] (3-3) The sequence obtained in step (3-2) Perform a Discrete Fourier Transform or a Fast Fourier Transform to obtain a transform sequence of length 2D-1. .

[0106] (3-4) Take each pixel point In sequence The first D elements in the corresponding array form a array of size D. The three-dimensional data is denoted as the initial matching body. .

[0107] (4) The initial matching body obtained in step (3) Thresholding is performed to generate the corresponding binary matching volume, denoted as . .

[0108] in, Size is , In this context, each 0 value represents that the corresponding point is not on the object being measured, and each 1 value represents that the corresponding point is on the object being measured.

[0109] The specific steps are as follows:

[0110] (4-1) Initialize a matching body with the initial body Equal-size empty binary three-dimensional data .

[0111] (4-2) satisfy all and The point is The value at the corresponding position is set to 1.

[0112] in For the initial matching body In the middle The value of the position.

[0113] Then China satisfies The point is The corresponding value at the specified position is set to 0, where The threshold value is 20 in this embodiment.

[0114] After processing, the final binary matching body is obtained. .

[0115] (5) The binary matching volume obtained in step (4) Perform anisotropic connectivity detection, select several regions with the highest connectivity, and obtain a size of 3D data as a connected binary matching volume .

[0116] In this context, each 0 value represents that the corresponding point is not on the object being measured, and each 1 value represents that the corresponding point is on the object being measured.

[0117] The specific steps are as follows:

[0118] (5-1) Binary matching volume A cube is formed by the neighborhood of any point in the cube. A total of 18 directions are selected, which are the 6 adjacent directions corresponding to the face center of the cube and the 12 adjacent directions corresponding to the edges of the cube.

[0119] In this embodiment, the 18 directions are divided into 9 direction types: , , , , , , , , .in, They represent along The direction reached by moving 1 unit in either the positive or negative direction of the axis. The direction type obtained by any two combinations represents the direction reached by moving 1 unit along the positive or negative directions of the corresponding two axes.

[0120] Furthermore, in this embodiment, for the nine directional types, , , , , These five types of directions are defined as valid directions. If a point lies on a valid direction of another point, then the two points are defined as connected; otherwise, the two points are not connected.

[0121] (5-2) Binary matching volume The set of points is composed of all points with a value of 1. Initialize the connected component number i=0.

[0122] in, Binary matching body In the middle The value of the position.

[0123] (5-3) Initialize an empty set of points Select any point in X and add it. Then delete the point in X.

[0124] (5-4) Select from X that matches the current... Connect any point in the middle and add it to the middle. Get the updated current Then continue selecting from X and updating the current... Connect any point in the middle and add it to the middle. until no new points are added to X. In this process, each time a point is selected from X, that point is deleted from X.

[0125] (5-5) Judgment:

[0126] If X is not an empty set, let the connected component number i = i + 1, and then return to step (5-3); otherwise, proceed to step (5-6).

[0127] (5-6) Initialize a binary matching body Equal-sized empty binary three-dimensional data is denoted as .from arrive Select all points contained in the top k sets with the largest number of points from all sets, and then... Set the value at the corresponding position in the middle to 1 to get the updated value. As the final connected binary matching body.

[0128] In one embodiment of the present invention, k is 4.

[0129] (6) Based on the results of step (5), for Each point in the array with a value of 1 The subpixel projector coordinates corresponding to this point are calculated using a binary subpixel thinning interpolator. This generates the corresponding triplet. The specific steps are as follows:

[0130] (6-1) Define a binary subpixel interpolator ,in, Let r be the output subpixel projector coordinates, and r be the shape parameter, satisfying:

[0131]

[0132] (6-2) Let the pixel range of the projector be... Within this range, the sub-pixel coordinates of the projector Perform equidistant sampling. This represents the projector's maximum horizontal pixel count.

[0133] In one embodiment of the present invention, the spacing is 0.05 projector pixels.

[0134] (6-3) Based on the results of step (6-2), for each Values ​​are used to construct a virtual capture light intensity sequence:

[0135]

[0136] Where L is the unit code length.

[0137] (6-4) Based on the results of step (6-3), the sequence The extension is a sequence of length 2D-1. The extended portion is supplemented with 0, that is:

[0138]

[0139] (6-5) Based on the results of step (6-4), perform a Discrete Fourier Transform or Fast Fourier Transform on the sequence to obtain a transformed sequence of length 2D-1. .

[0140] (6-6) Based on the results of step (6-5), take The first D elements form a sequence of length D. .

[0141] (6-7) Based on the result of step (6-6), the output makes Take the d-coordinate of the maximum value and calculate the corresponding r-value:

[0142]

[0143] Thus generating each Corresponding triples .

[0144] (6-8) Based on the results of step (6-7), utilize all triples Constructing a binary subpixel interpolator .

[0145] (6-9) Based on the results of step (6-8), perform a connected binary matching volume. Each point with a value of 1 Calculate its corresponding r, and then calculate based on the binary pixel interpolator. Thus, the triplet is obtained. (7) Based on the results of step (6), calculate each triplet using triangulation. Corresponding point cloud 3D coordinates The specific steps are as follows:

[0146] (7-1) Combine the triples Normalize each coordinate separately:

[0147]

[0148]

[0149]

[0150] (7-2) Based on the results of step (7-1), calculate the three-dimensional coordinates of the point cloud. .

[0151]

[0152]

[0153]

[0154] Then, using the three-dimensional coordinates of the point cloud The object being measured can then be reconstructed, and the measurement is complete.

[0155] To achieve the above embodiments, a second aspect of the present invention provides a frequency-shifting structured light measurement device, comprising:

[0156] The image projection and acquisition module is used to project several time-delayed topology transform frequency shift encoded images onto the object under test using the projector in the structured light measurement system, and then use a camera to capture the projected images in sequence.

[0157] The time-extended transformation decoding module is used to perform time-extended transformation decoding on the captured projection image to generate the corresponding initial matching body;

[0158] The thresholding module is used to perform thresholding processing on the initial matching body to generate a corresponding binary matching body. In the binary matching body, each 0 value represents that the corresponding point is not on the object being tested, and each 1 value represents that the corresponding point is on the object being tested.

[0159] An anisotropic connectivity detection module is used to perform anisotropic connectivity detection on the binary matching volume, select several regions with the highest connectivity, and obtain a connected binary matching volume.

[0160] The subpixel projector coordinate generation module is used to calculate the subpixel projector coordinates corresponding to each point with a value of 1 in the connected binary matching volume using a binary subpixel thinning interpolator, thereby generating a coordinate triplet containing the subpixel projector coordinates corresponding to that point.

[0161] The point cloud 3D coordinate generation module is used to calculate the point cloud 3D coordinates corresponding to the coordinate triplet using triangulation, thereby completing the 3D measurement of the object under test.

[0162] In one specific embodiment of the present invention, it further includes:

[0163] Before projecting several time-delayed transform frequency-shift coded images onto the object under test using the projector in the structured light measurement system, the system is calibrated to obtain the camera intrinsic parameter matrix. Projector intrinsic parameter matrix and extrinsic parameter matrix :

[0164] ,

[0165]

[0166] in, The horizontal focal length of the camera. The vertical focal length of the camera. The position of the camera's horizontal optical center. The longitudinal optical center position of the camera. This is the horizontal focal length of the projector. This refers to the projector's vertical focal length. The horizontal optical center position of the projector. This refers to the longitudinal optical center position of the projector; This represents the value at the corresponding position in the camera-to-projector rotation matrix. ; Let be the translation vector from the camera to the projector.

[0167] In a specific embodiment of the present invention, the step of performing time-extended transform decoding on the captured projection image includes:

[0168] (1) For any pixel in the captured image Extract the light intensity of that point in the captured image, and denote it as a sequence. ;in, These are the x and y coordinates of the camera pixels, respectively; d is the image number; and D is the number of coded images in the projection.

[0169] (2) The sequence obtained in step (1) The extension is a sequence of length 2D-1. The extended portion is supplemented with 0, that is:

[0170]

[0171] (3) The sequence obtained in step (2) Perform a Discrete Fourier Transform or a Fast Fourier Transform to obtain a transform sequence of length 2D-1. ;

[0172] (4) Take each pixel In sequence The first D elements in the corresponding array form a array of size D. The three-dimensional data is denoted as the initial matching body. ,in These represent the number of pixels horizontally and vertically on the camera, respectively.

[0173] In a specific embodiment of the present invention, the thresholding process for the initial matching body includes:

[0174] (1) Initialize a matching body with the initial body Equal-size empty binary three-dimensional data ;

[0175] (2) All of the above satisfy and The point is Set the value at the corresponding position in the middle to 1;

[0176] in, For the initial matching body In the middle The value of the position;

[0177] Then China satisfies The point is The value at the corresponding position is set to 0, where The threshold for the matching body;

[0178] After processing, the final binary matching body is obtained. .

[0179] In a specific embodiment of the present invention, obtaining a connected binary matching volume includes:

[0180] (1) Binary matching volume A cube is formed by the neighborhood of any point in the cube. Six adjacent directions corresponding to the face center of the cube and twelve adjacent directions corresponding to the edges of the cube are selected, for a total of 18 directions.

[0181] These 18 directions are divided into 9 direction types: , , , , , , , , ;in, They represent along The direction reached by moving 1 unit in either the positive or negative direction of the axis. The direction type obtained by any two combinations represents the direction reached by moving 1 unit along the positive or negative directions of the corresponding two axes;

[0182] Will , , , , As a valid direction; if a point lies in a valid direction of another point, then the two points are connected; otherwise, the two points are not connected.

[0183] (2) Binary matching volume The set of points is composed of all points with a value of 1. Initialize the connected component number i = 0;

[0184] in, Binary matching body In the middle The value of the position;

[0185] (3) Initialize an empty set of points ; Select any point in X and add Then delete the point in X;

[0186] (4) Select from X that matches the current value. Connect any point in the middle and add it to the middle. Get the updated current Then continue selecting from X and updating the current... Connect any point in the middle and add it to the middle. until no new points are added to X. In this process, each time a point is selected from X, that point is deleted from X.

[0187] (5) Judgment:

[0188] If X is not an empty set, let the connected component number i = i + 1, and then return to step (3); otherwise, proceed to step (6).

[0189] (6) Initialize a binary matching body Equal-sized empty binary three-dimensional data is denoted as ;from arrive Select the k sets with the largest number of points from all sets, and then sort all the points contained in these k sets. Set the value at the corresponding position in the middle to 1 to get the updated value. As the final connected binary matching body;

[0190] In this context, each 0 value represents that the corresponding point is not on the object being measured, and each 1 value represents that the corresponding point is on the object being measured.

[0191] In a specific embodiment of the present invention, generating the coordinate triplet containing the sub-pixel projector coordinates corresponding to the point includes:

[0192] (1) Define the binary subpixel interpolator ,in, Let r be the output subpixel projector coordinates, and r be the shape parameter, satisfying:

[0193]

[0194] (2) Let the pixel range of the projector be Within this range, the sub-pixel coordinates of the projector Perform equidistant sampling. The maximum horizontal pixel count of the projector;

[0195] (3) Based on the results of step (2), for each Values ​​are used to construct a virtual capture light intensity sequence:

[0196]

[0197] Where L is the unit code length;

[0198] (4) Based on the results of step (3), the sequence The extension is a sequence of length 2D-1. The extended portion is supplemented with 0, that is:

[0199]

[0200] (5) Based on the result of step (4), perform a discrete Fourier transform or a fast Fourier transform on the sequence to obtain a transform sequence of length 2D-1. ;

[0201] (6) Based on the result of step (5), take The first D elements form a sequence of length D. ;

[0202] (7) Based on the result of step (6), the output makes Take the d-coordinate of the maximum value and calculate the corresponding r-value:

[0203]

[0204] Thus generating each Corresponding triples ;

[0205] (8) Based on the results of step (7), utilize all triples Constructing a binary subpixel interpolator ;

[0206] (9) Based on the results of step (8), perform a connected binary matching volume. Each point with a value of 1 Calculate its corresponding r, and then calculate based on the binary pixel interpolator. Thus, the triplet is obtained. .

[0207] In a specific embodiment of the present invention, the step of calculating the three-dimensional coordinates of the point cloud corresponding to the coordinate triplet using triangulation includes:

[0208] (1) The triplet Normalize each coordinate separately:

[0209]

[0210]

[0211]

[0212] (2) Based on the results of step (1), calculate the three-dimensional coordinates of the point cloud. ;

[0213]

[0214]

[0215] .

[0216] This enables the measurement of objects with internal multiple reflections, especially those with specular reflections or transparent / semi-transparent materials. It offers high calculation efficiency and accuracy, solving the problem of difficult measurement of non-Lambertian objects in structured light systems.

[0217] To implement the above embodiments, a third aspect of the present invention provides an electronic device, comprising:

[0218] At least one processor; and a memory communicatively connected to said at least one processor;

[0219] The memory stores instructions that can be executed by the at least one processor, and the instructions are configured to perform the frequency-shift structured light measurement method described above.

[0220] To implement the above embodiments, a fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing the computer to execute the above-described frequency-shift structured light measurement method.

[0221] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0222] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform a frequency-shift structured light measurement method according to the above embodiments.

[0223] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0224] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0225] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0226] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0227] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0228] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0229] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments.

[0230] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0231] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for measuring frequency-shifted structured light, characterized in that, include: The projector in the structured light measurement system projects several time-delayed topology transform frequency shift encoded images onto the object under test, and then the camera captures the projected images in sequence. The captured projection image is decoded by time-delay transformation to generate the corresponding initial matching body; The initial matching body is thresholded to generate a corresponding binary matching body. In the binary matching body, each 0 value represents that the corresponding point is not on the object being tested, and each 1 value represents that the corresponding point is on the object being tested. Anisotropic connectivity detection is performed on the binary matching volume, and several regions with the highest connectivity are selected to obtain a connected binary matching volume. Using a binary subpixel thinning interpolator, the subpixel projector coordinates corresponding to each point with a value of 1 in the connected binary matching volume are calculated, thereby generating a coordinate triplet containing the subpixel projector coordinates for that point. The three-dimensional coordinates of the point cloud corresponding to the coordinate triplet are calculated using triangulation, thereby completing the three-dimensional measurement of the object under test.

2. The method according to claim 1, characterized in that, Also includes: Before projecting several time-delayed transform frequency-shift coded images onto the object under test using the projector in the structured light measurement system, the system is calibrated to obtain the camera intrinsic parameter matrix. Projector intrinsic parameter matrix and extrinsic parameter matrix : ， in, The horizontal focal length of the camera. The vertical focal length of the camera. The position of the camera's horizontal optical center. The longitudinal optical center position of the camera. This is the horizontal focal length of the projector. This refers to the projector's vertical focal length. The horizontal optical center position of the projector. This refers to the longitudinal optical center position of the projector; This represents the value at the corresponding position in the camera-to-projector rotation matrix. ; Let be the translation vector from the camera to the projector.

3. The method according to claim 2, characterized in that, The step of performing time-extended transform decoding on the captured projection image includes: (1) For any pixel in the captured image Extract the light intensity of that point in the captured image, and denote it as a sequence. ;in, These are the x and y coordinates of the camera pixels, respectively; d is the image number; and D is the number of coded images in the projection. (2) The sequence obtained in step (1) The extension is a sequence of length 2D-1. The extended portion is supplemented with 0, that is: (3) The sequence obtained in step (2) Perform a Discrete Fourier Transform or a Fast Fourier Transform to obtain a transform sequence of length 2D-1. ; (4) Take each pixel In sequence The first D elements in the corresponding array form a array of size D. The three-dimensional data is denoted as the initial matching body. ,in These represent the number of pixels horizontally and vertically on the camera, respectively.

4. The method according to claim 3, characterized in that, The thresholding process for the initial matched body includes: (1) Initialize a matching body with the initial body Equal-size empty binary three-dimensional data ; (2) All of the above satisfy and The point is Set the value at the corresponding position in the middle to 1; in, For the initial matching body In the middle The value of the position; Then China satisfies The point is The value at the corresponding position is set to 0, where The threshold for the matching body; After processing, the final binary matching body is obtained. .

5. The method according to claim 4, characterized in that, The process of obtaining a connected binary matching volume includes: (1) Binary matching volume A cube is formed by the neighborhood of any point in the cube. Six adjacent directions corresponding to the face center of the cube and twelve adjacent directions corresponding to the edges of the cube are selected, for a total of 18 directions. These 18 directions are divided into 9 direction types: , , , , , , , , ;in, They represent along The direction reached by moving 1 unit in either the positive or negative direction of the axis. The direction type obtained by any two combinations represents the direction reached by moving 1 unit along the positive or negative directions of the corresponding two axes; Will , , , , As a valid direction; if a point lies in a valid direction of another point, then the two points are connected; otherwise, the two points are not connected. (2) Binary matching volume The set of points is composed of all points with a value of 1. Initialize the connected component number i = 0; in, Binary matching body In the middle The value of the position; (3) Initialize an empty set of points ; Select any point in X and add Then delete the point in X; (4) Select from X that matches the current value. Connect any point in the middle and add it to the middle. Get the updated current Then continue selecting from X and updating the current... Connect any point in the middle and add it to the middle. until no new points are added to X. In this process, each time a point is selected from X, that point is deleted from X. (5) Judgment: If X is not an empty set, let the connected component number i = i + 1, and then return to step (3); otherwise, proceed to step (6). (6) Initialize a binary matching body Equal-sized empty binary three-dimensional data is denoted as ;from arrive Select the k sets with the largest number of points from all sets, and then sort all the points contained in these k sets. Set the value at the corresponding position in the middle to 1 to get the updated value. As the final connected binary matching body; In each case, each 0 value represents that the corresponding point is not on the measured object, and each 1 value represents that the corresponding point is on the measured object.

6. The method of claim 5, wherein, The process of generating the coordinate triplet corresponding to the point, which includes the sub-pixel projector coordinates, includes: (1) Defining a binary sub-pixel interpolator wherein, is the output sub-pixel projector coordinate, r is a shape parameter, satisfying: (2) Let the projector pixel range be Within this range, sample the projector subpixel coordinates at equal intervals, with projector maximum horizontal pixel, ; (3) Based on the result of step (2), for each value, construct a virtual captured light intensity sequence: Where L is the unit code length; (4) Based on the result of step (3), the sequence is extended to a sequence of length 2D-1 where the extension part is padded with 0, i.e.: (5) Based on the result of step (4), a discrete Fourier transform or fast Fourier transform is performed on the sequence to obtain a transformed sequence of length 2D-1 ; (6) Based on the result of step (5), take the first D elements of to form a sequence of length D ; (7) Based on the result of step (6), outputting such that Taking the maximum value of the d coordinate, the corresponding r value is calculated: Thus generating each corresponding triple ; (8) Based on the result of step (7), utilize all triplets Constructing a binary sub-pixel interpolator ; (9) Based on the result of step (8), the connected binary matchers each of which has a value of 1 its corresponding r is calculated, and then the ternary pixel interpolator is calculated , so as to obtain the ternary group .

7. The method of claim 6, wherein, The calculation of the point cloud 3D coordinates corresponding to the coordinate triplet using triangulation includes: (1) The triplet Normalize each coordinate separately: (2) Based on the result of step (1), calculate the three-dimensional coordinates of the point cloud ; 。 8. A frequency-shift structured light measurement device, characterized in that, include: The image projection and acquisition module is used to project several time-delayed topology transform frequency shift encoded images onto the object under test using the projector in the structured light measurement system, and then use a camera to capture the projected images in sequence. The time-extended transformation decoding module is used to perform time-extended transformation decoding on the captured projection image to generate the corresponding initial matching body; The thresholding module is used to perform thresholding processing on the initial matching body to generate a corresponding binary matching body. In the binary matching body, each 0 value represents that the corresponding point is not on the object being tested, and each 1 value represents that the corresponding point is on the object being tested. An anisotropic connectivity detection module is used to perform anisotropic connectivity detection on the binary matching volume, select several regions with the highest connectivity, and obtain a connected binary matching volume. The subpixel projector coordinate generation module is used to calculate the subpixel projector coordinates corresponding to each point with a value of 1 in the connected binary matching volume using a binary subpixel thinning interpolator, thereby generating a coordinate triplet containing the subpixel projector coordinates corresponding to that point. The point cloud 3D coordinate generation module is used to calculate the point cloud 3D coordinates corresponding to the coordinate triplet using triangulation, thereby completing the 3D measurement of the object under test.

9. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the method according to any one of claims 1-7.

Images