Single-frame circular stripe structured light three-dimensional reconstruction method and device and electronic equipment

Abstract

The invention provides a single-frame circular stripe structured light three-dimensional reconstruction method and device and electronic equipment, and relates to the technical field of three-dimensional imaging, and the method comprises the steps: obtaining a preset standard circular stripe image which comprises a circular stripe pattern; the circle center of the circular stripe pattern coincides with the pole of the projection plane of the projection equipment; projecting the standard circular stripe image to a to-be-reconstructed object through projection equipment, and modulating the standard circular stripe image through the to-be-reconstructed object; acquiring a deformed stripe image of the to-be-reconstructed object through image acquisition equipment; extracting phase information of the deformed fringe image to obtain truncation phase information; unwrapping the truncated phase information to obtain target continuous phase information; according to the method, the three-dimensional point cloud data is determined based on the target continuous phase information, and the three-dimensional model of the object to be reconstructed is reconstructed based on the three-dimensional point cloud data, so that the real-time performance of single-frame projection can be ensured, and the reconstruction precision can be improved.

Description

Technical Field

[0001] This application relates to the field of three-dimensional imaging technology, and more specifically, to a method, apparatus, and electronic device for three-dimensional reconstruction of a single-frame circular stripe structured light. Background Technology

[0002] Structured light 3D imaging is a method that projects a standard fringe image onto the surface of an object to be reconstructed using a projection device, acquires a phase map composed of each pixel by an image acquisition device, and then performs 3D reconstruction to obtain a 3D model of the object to be reconstructed.

[0003] Currently, in existing technologies, unidirectional straight stripes are typically used for projection. However, this method requires restrictions on the positional relationship between the projection device and the image acquisition device. In practical applications, this can easily lead to a significant loss of accuracy because the measurement environment cannot meet these restrictions. Furthermore, even when the positional relationship strictly meets the restrictions, the theoretical maximum accuracy cannot be achieved. Summary of the Invention

[0004] In order to at least overcome the above-mentioned deficiencies in the prior art, the purpose of this application is to provide a single-frame circular stripe structured light three-dimensional reconstruction method, apparatus and electronic device.

[0005] In a first aspect, embodiments of this application provide a single-frame circular stripe structured light 3D reconstruction method, applied to a structured light 3D reconstruction system, the structured light 3D reconstruction system including a projection device and an image acquisition device;

[0006] The single-frame circular stripe structured light 3D reconstruction method includes:

[0007] A pre-set standard circular stripe image is obtained, the standard circular stripe image including a circular stripe pattern; the center of the circular stripe pattern coincides with the pole of the projection plane of the projection device;

[0008] The standard circular stripe image is projected onto the object to be reconstructed using the projection device, and the standard circular stripe image is modulated by the object to be reconstructed.

[0009] The image acquisition device acquires a deformed stripe image of the object to be reconstructed, the deformed stripe image including a deformed stripe pattern;

[0010] Extract the phase information of the deformed stripe image to obtain truncated phase information;

[0011] The truncated phase information is unwrapped to obtain the continuous phase information of the target.

[0012] Determine three-dimensional point cloud data based on the target continuous phase information, and reconstruct a three-dimensional model of the object to be reconstructed based on the three-dimensional point cloud data.

[0013] In a possible implementation, the step of extracting phase information of the deformed fringe image to obtain truncated phase information comprises:

[0014] Perform qualitative analysis based on spectral energy distribution and quantitative analysis based on spatial frequency distribution on the deformed fringe image respectively to obtain filter parameters;

[0015] Design a filter based on the filter parameters, and extract phase information of the deformed fringe image through the filter to obtain truncated phase information.

[0016] In a possible implementation, the step of performing qualitative analysis based on spectral energy distribution on the deformed fringe image comprises:

[0017] Determine a first expression of the deformed fringe image based on the deformed fringe image;

[0018] Perform Fourier transform on the first expression of the deformed fringe image to obtain a first Fourier transform result;

[0019] Perform qualitative analysis on the deformed fringe image based on the first Fourier transform result to obtain spectral energy distribution information of the deformed fringe image.

[0020] In a possible implementation, the step of performing quantitative analysis based on spatial frequency distribution on the deformed fringe image comprises:

[0021] Determine a second expression of the deformed fringe image based on the deformed fringe image;

[0022] Perform Fourier transform on the second expression of the deformed fringe image to obtain a second Fourier transform result;

[0023] Calculate spatial frequency of the deformed fringe pattern according to the second Fourier transform result;

[0024] Calculate the number of periods of the deformed fringe image according to the spatial frequency of the deformed fringe pattern.

[0025] In a possible implementation, the step of unwrapping the truncated phase information to obtain target continuous phase information comprises:

[0026] Perform phase unwrapping processing on the truncated phase information to obtain initial continuous phase information;

[0027] The initial continuous phase information is subjected to a value range transformation process to obtain target continuous phase information.

[0028] In a possible implementation, the step of determining three-dimensional point cloud data based on the target continuous phase information comprises:

[0029] determining radial distance information of the standard circular fringe image according to the target continuous phase information; the radial distance information is used to represent a distance between each pixel point of the standard circular fringe image and a center of the circular fringe pattern;

[0030] calculating polar radius information of the deformed fringe image; the polar radius information is used to represent a distance between each pixel point of the deformed fringe image and a phase pole point of a projection plane;

[0031] determining three-dimensional point cloud data according to the radial distance information and the polar radius information.

[0032] In a possible implementation, the three-dimensional point cloud data (X w , Y w , Z w ) is calculated in the following manner:

[0033]

[0034] wherein, represents a coordinate of an optical pole; r p represents the radial distance information; represents polar radius information corresponding to the phase pole point of the projection plane.

[0035] In a second aspect, the embodiments of the present application further provide a single-frame circular fringe structured light three-dimensional reconstruction device, applied to a structured light three-dimensional reconstruction system, wherein the structured light three-dimensional reconstruction system comprises a projection device and an image acquisition device.

[0036] The single-frame circular fringe structured light three-dimensional reconstruction device comprises:

[0037] a receiving module, configured to acquire a pre-set standard circular fringe image, wherein the standard circular fringe image comprises a circular fringe pattern; a center of the circular fringe pattern coincides with a pole point of a projection plane of the projection device;

[0038] a projection module, configured to project the standard circular fringe image onto an object to be reconstructed by the projection device, and modulate the standard circular fringe image by the object to be reconstructed;

[0039] an acquisition module, configured to acquire a deformed fringe image of the object to be reconstructed by the image acquisition device, wherein the deformed fringe image comprises a deformed fringe pattern.

[0040] The extraction module is used to extract the phase information of the deformed stripe image to obtain truncated phase information;

[0041] The unwrapping module is used to unwrap the truncated phase information to obtain the target continuous phase information;

[0042] The reconstruction module is used to determine three-dimensional point cloud data based on the target continuous phase information, and to reconstruct the three-dimensional model of the object to be reconstructed based on the three-dimensional point cloud data.

[0043] Thirdly, embodiments of this application also provide an electronic device, including:

[0044] Memory, used to store one or more programs;

[0045] The processor, when the one or more programs are executed by the processor, implements the single-frame circular stripe structured light three-dimensional reconstruction method provided in the first aspect above.

[0046] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the single-frame circular stripe structured light three-dimensional reconstruction method provided in the first aspect.

[0047] Based on any of the above aspects, the single-frame circular stripe structured light three-dimensional reconstruction method, apparatus and electronic device provided in the embodiments of this application can significantly improve the adaptability of single-frame stripe projection to the environment by using standard circular stripe images for three-dimensional reconstruction, and improve reconstruction accuracy while ensuring the real-time performance of single-frame projection. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings required in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of the structured light 3D reconstruction system provided in this embodiment;

[0050] Figure 2 This is a flowchart illustrating the single-frame circular stripe structured light 3D reconstruction method provided in this embodiment;

[0051] Figure 3 This is a schematic diagram of the epipolar geometry of the structured light 3D reconstruction system provided in this embodiment;

[0052] Figure 4aA schematic diagram of the epipolar geometry of straight stripes provided for the present embodiment;

[0053] Figure 4b A schematic diagram of the epipolar geometry of circular stripes provided for the present embodiment;

[0054] Figure 5 A schematic diagram of the projection plane provided for the present embodiment;

[0055] Figure 6 A schematic diagram of the sub-step of step S400 provided for the present embodiment;

[0056] Figure 7 One of the schematic diagrams of the sub-step of step S410 provided for the present embodiment;

[0057] Figure 8 The second of the schematic diagrams of the sub-step of step S410 provided for the present embodiment;

[0058] Figure 9 A correspondence diagram of the spatial frequency distribution and the spectrum of the deformed stripe image provided for the present embodiment;

[0059] Figure 10 A schematic diagram of the sub-step of step S500 provided for the present embodiment;

[0060] Figure 11 A schematic diagram of the sub-step of step S600 provided for the present embodiment;

[0061] Figure 12 A schematic structural block diagram of an electronic device provided for the present embodiment;

[0062] Figure 13 A functional module schematic diagram of a single-frame circular stripe structured light three-dimensional reconstruction device provided for the present embodiment.

[0063] Icon: 100 - projection device; 200 - image acquisition device; 300 - object to be reconstructed; 400 - computer; 110 - standard circular stripe image; 210 - deformed stripe image; 700 - electronic device; 710 - processor; 720 - computer readable storage medium; 730 - single-frame circular stripe structured light three-dimensional reconstruction device; 731 - receiving module; 732 - projection module; 733 - acquisition module; 734 - extraction module; 735 - unwrapping module; 736 - reconstruction module. DETAILED DESCRIPTION

[0064] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0065] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present application.

[0066] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0067] In the description of the present application, it should be noted that the terms "upper", "lower" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship commonly placed when the product of the application is used, only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, therefore, cannot be understood as a limitation on the present application. In addition, the terms "first", "second" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0068] It should be noted that, in the case of no conflict, different features in the embodiments of the present application can be combined with each other.

[0069] The inventor has found that at present, Fringe Projection Profilometry (FPP) is usually used to perform structured light three-dimensional reconstruction on an object to be reconstructed. The traditional structured light three-dimensional reconstruction method usually needs to project multiple frames of different fringe images, and then accurate phase information is obtained through phase unwrapping algorithms between multiple frames of images. However, this multi-frame projection mode has the problem of slow measurement speed, which is difficult to meet the real-time measurement demand.

[0070] To meet real-time measurement requirements, single-frame structured light technology can be used, which achieves 3D reconstruction by projecting a single fringe image to reduce measurement time. Current single-frame structured light technology typically uses unidirectional straight stripes for projection. Unidirectional straight stripes refer to a stripe pattern with stripes that have a single direction and are distributed in a straight line. However, this method requires constraints on the positional relationship between the projection device and the image acquisition device. Since current measurement environments often fail to meet these constraints, especially during single-frame measurements where the measurement environment is complex and unstable, it can easily lead to significant accuracy loss.

[0071] This embodiment provides a solution to the above problems. The specific implementation of this application will be described in detail below with reference to the accompanying drawings.

[0072] Please refer to Figure 1 , Figure 1 This embodiment provides a structured light 3D reconstruction system, which may include a computer 400, a projection device 100, and an image acquisition device 200. The computer 400 and the projection device 100 can communicate with each other via a wired or wireless network. The computer 400 can generate a standard circular fringe image 110 and send it to the projection device 100, which can then project the image 110 onto the object 300 to be reconstructed. The computer 400 and the image acquisition device 200 can also communicate with each other via a wired or wireless network. The image acquisition device 200 can acquire a deformed fringe image 210 of the object 300 to be reconstructed and send it to the computer 400, enabling the computer 400 to perform 3D reconstruction using the deformed fringe image 210.

[0073] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating the single-frame circular fringe structured light 3D reconstruction method provided in this embodiment. The single-frame circular fringe structured light 3D reconstruction method can be applied to, for example... Figure 1 The structured light 3D reconstruction system shown can include the following steps in its single-frame circular stripe structured light 3D reconstruction method.

[0074] Step S100: Obtain a pre-set standard circular stripe image 110, wherein the standard circular stripe image 110 includes a circular stripe pattern; the center of the circular stripe pattern coincides with the pole of the projection plane of the projection device 100.

[0075] Please refer to Figure 3 , plane π C This can represent the imaging plane of the image acquisition device 200, plane π P This can represent the imaging plane of the projection device 100, O pAs the optical center (optical core) of the image acquisition device 200, the optical center O of the image acquisition device 200 is... c The coordinate system with the origin as the first coordinate system corresponding to the image acquisition device 200 is O. c As the optical center of the projection device 100, the optical center O of the projection device 100 is... p The coordinate system with the origin as the second coordinate system corresponding to the projection device 100. The optical center O of the image acquisition device 200 c With the optical center O of the projection device 100 p The connecting lines between them can be respectively connected to the imaging plane π of the image acquisition device 200. C and the imaging plane π of the projection device 100 P Intersect at point e c and point e p Point e c Let point e be the pole of the imaging plane of the image acquisition device 200. p Let M be the pole of the projection plane. Point M can be any point in three-dimensional space, and point M can be the optical center O of the image acquisition device 200. c With the optical center O of the projection device 100 p Form a plane that can intersect with plane π C and plane π P They intersect at line l respectively c and line l p straight line l c and line l p These are polar lines. As the position of point M changes, polar lines will be generated at different positions, and all polar lines intersect at the pole.

[0076] Please refer to Figure 4a On the projection plane, the angle between the phase change direction of the fringes and the epipolar line is δ. Phase sensitivity can be used to characterize the sensitivity of the phase change on the projector plane when the height of an object's surface changes slightly. The higher the phase sensitivity, the higher the accuracy of the calculated point cloud data. The phase sensitivity S can be calculated as follows:

[0077]

[0078] Among them, z q It can represent the height of a spatial point corresponding to point q, that is, the height of point M, for different z. q The corresponding polar lines are different. (x) p y p ) can represent the coordinates of a point on the projection plane, and f can represent the spatial frequency.

[0079] When the angle δ between the phase change direction of the fringe and the polar line is 0°, the value of the phase sensitivity is maximum, and at this time, the phase change direction of the fringe is parallel to the polar line. When the standard circular fringe image 110 is a straight fringe pattern, since the phase change direction of the straight fringe is fixed, δ cannot be ensured to be always 0°. Please refer to Figure 4b When the standard circular fringe image 110 is a circular fringe pattern, the center of the circular fringe pattern can coincide with the polar point of the projection plane of the projection device 100, and at this time, the phase change direction of the fringe can always be parallel to the polar line.

[0080] Therefore, in the present embodiment, please refer to Figure 5 The parameters of the structured light three-dimensional reconstruction system can be obtained by calibration first, and then a circular fringe pattern whose center coincides with the polar point of the projection plane of the projection device 100 is generated as the standard circular fringe image 110. The expression of the standard circular fringe image 110 can be:

[0081]

[0082] where A p may represent the average intensity of the standard circular fringe image 110, B p may represent the modulation degree of the standard circular fringe image 110, f can represent the spatial frequency, r p may represent the distance from any point on the projection plane to the center of the circle , may represent the minimum distance from the projection plane to the center of the circle , and the corresponding point on the projection plane can be may represent the maximum distance from the projection plane to the center of the circle , and the corresponding point on the projection plane can be

[0083] In some examples, in an 8-bit image, the values of the average intensity and the modulation degree of the standard circular fringe image 110 can both be 127.5. The value of the spatial frequency can be an integer. The coordinates of the center of the circle can be obtained through the calibration process.

[0084] By adjusting the minimum distance from the projection plane to the center of the circle , and the maximum distance from the projection plane to the center of the circle , the encoding range of the phase on the projection plane can be fixed to [0, 2π]. Please refer again to Figure 5 Generally, the polar point of the projection plane is located outside the projection plane, and when the position of the polar point is very far, the circular fringe will be infinitely close to the straight fringe, and at this time, the straight fringe is a quasi-optimal fringe.

[0085] In step S200, the standard circular fringe image 110 is projected onto the object 300 to be reconstructed by the projection device 100, and the standard circular fringe image 110 is modulated by the object 300 to be reconstructed.

[0086] In this embodiment, the standard circular fringe image 110 obtained in step S100 can be projected onto the object 300 to be reconstructed by the projection device 100, and the standard circular fringe image 110 is modulated by the surface profile / terrain of the object 300 to be reconstructed. Exemplarily, the projection device 100 can be a projector.

[0087] In step S300, the deformed fringe image 210 of the object 300 to be reconstructed is captured by the image acquisition device 200, and the deformed fringe image 210 includes a deformed fringe pattern.

[0088] In this embodiment, the deformed fringe image 210 after being modulated by the object 300 to be reconstructed can be captured by the image acquisition device 200, and the deformed fringe image 210 can include a deformed fringe pattern. Exemplarily, the image acquisition device 200 can be a camera.

[0089] In step S400, the phase information of the deformed fringe image 210 is extracted to obtain truncated phase information.

[0090] Single-frame projection can only obtain a single deformed fringe image 210, and the amount of information is less, so that phase extraction can only be performed in the transform domain. In this embodiment, the phase information of the deformed fringe image 210 can be extracted in the frequency domain based on Fourier Transform Profilometry (FTP), so as to obtain truncated phase information.

[0091] In step S500, the truncated phase information is unwrapped to obtain target continuous phase information.

[0092] In this embodiment, the truncated phase information obtained in step S400 can be unwrapped to obtain target continuous phase information.

[0093] In step S600, three-dimensional point cloud data is determined based on the target continuous phase information, and a three-dimensional model of the object 300 to be reconstructed is reconstructed based on the three-dimensional point cloud data.

[0094] In the embodiment, the three-dimensional point cloud data can be calculated according to the target continuous phase information obtained in step S500, and the three-dimensional reconstruction is performed on the object to be reconstructed 300 according to the three-dimensional point cloud data, so as to obtain the three-dimensional model corresponding to the object to be reconstructed 300. The three-dimensional point cloud data can refer to the three-dimensional coordinate information of any pixel point in the deformed fringe image 210 to the center of the deformed fringe pattern.

[0095] It can be seen that, based on the above design, the single-frame circular fringe structured light three-dimensional reconstruction method provided by the embodiment of the application can significantly improve the adaptability of single-frame fringe projection to the environment by using the standard circular fringe image 110 for three-dimensional reconstruction, and can improve the reconstruction accuracy while ensuring the real-time of single-frame projection.

[0096] In a possible implementation, referring to Figure 6 , step S400 can include the following sub-steps.

[0097] In step S410, qualitative analysis based on spectral energy distribution and quantitative analysis based on spatial frequency distribution are performed on the deformed fringe image 210 respectively, to obtain filter parameters.

[0098] In the embodiment, since the spectral form of the circular fringe is relatively complex, in order to accurately extract the phase information of the deformed fringe image 210, a filter that can fit its behavior needs to be designed, therefore, qualitative analysis and quantitative analysis of the fringe spectral characteristics are needed to obtain the filter parameters.

[0099] In step S420, the filter is designed based on the filter parameters, and the phase information of the deformed fringe image 210 is extracted through the filter, to obtain the truncated phase information.

[0100] In the embodiment, the filter can be designed according to the filter parameters obtained in step S410, and the phase information of the deformed fringe image 210 is extracted through the filter, to obtain the truncated phase information.

[0101] In the above design, by performing qualitative analysis on the deformed fringe image 210, direct solution of the non-integrable double spectral integral formula is avoided, and the filter design efficiency is improved, in addition, by performing quantitative analysis on the deformed fringe image 210, the filter parameters can be obtained, and high-precision adaptive filtering is realized.

[0102] In a possible implementation, referring to Figure 7 , in the qualitative analysis, step S410 can include the following sub-steps.

[0103] In step S411, the first expression of the deformed fringe image 210 is determined based on the deformed fringe image 210.

[0104] In the embodiment, due to the limited imaging range of the projection device 100 and the image acquisition device 200, the deformed fringe is usually not a complete circle, but an arc, and the polar radius and the phase angle are fixed in a range under the limitation of the system geometry. Therefore, the deformed fringe pattern captured by the image acquisition device 200 is similar to the circular fringe pattern of the standard circular fringe image 110, except that the parameters of the deformed fringe pattern are different from those of the circular fringe pattern. If the center of the deformed fringe pattern is located at the polar point on the image acquisition device plane , the first expression of the deformed fringe image 210 can be:

[0105]

[0106] wherein A can represent the average intensity of the deformed fringe image 210, B can represent the modulation degree of the deformed fringe image 210, f can represent the spatial frequency, r c can represent the distance from any point on the image acquisition device plane to the center of the circle, and k can be a normalization coefficient. c c can represent the minimum distance from the point on the image acquisition device plane to the center of the circle, can represent the maximum distance from the point on the image acquisition device plane to the center of the circle.

[0107] In step S412, Fourier transform is performed on the first expression of the deformed fringe image 210 to obtain a first Fourier transform result.

[0108] In the embodiment, the first expression in step S411 can be simplified first and expressed in polar coordinates as:

[0109]

[0110] wherein A can represent the average intensity of the deformed fringe image 210, B can represent the modulation degree of the deformed fringe image 210, f can represent the spatial frequency, r c can represent the distance from any point on the image acquisition device plane to the origin of the polar coordinates, and θ can represent the included angle between any point on the image acquisition device plane and the polar coordinate axis, W r (r c ) can represent the window function applied in the radial direction of the fringe due to the truncation effect caused by the limited imaging field of view, W θ (θ) can represent the window function applied in the angular direction of the fringe due to the truncation effect caused by the limited imaging field of view, can represent the minimum distance from the point on the image acquisition device plane to the center of the circle, ​​​The maximum distance from the center of the circle to the point on the image acquisition device plane can be represented.

[0111] According to the polar coordinate expression described above, the fringe on the image acquisition device plane can be parameterized into a radial part and an angular part. Let (u, v) be the Cartesian coordinates of the spectrum plane, and (p, y) be the polar coordinates of the frequency domain. The Fourier transform can be directly performed on the polar coordinate expression to obtain a first Fourier transform result:

[0112]

[0113] The first Fourier transform result can be an expression of the fringe spectrum. The radial intensity distribution of the fringe can be represented as The angular intensity distribution of the fringe can be represented as Since the origin of the fringe polar coordinate expression and the origin of the Cartesian coordinate do not coincide, a pure phase term will appear when performing Fourier transform This term has no effect on the energy distribution of the spectrum, so it can be ignored in qualitative analysis.

[0114] In step S413, qualitative analysis is performed on the deformed fringe image 210 based on the first Fourier transform result to obtain the spectral energy distribution information of the deformed fringe image.

[0115] In this embodiment, qualitative analysis can be performed on the deformed fringe image 210 according to the first Fourier transform result obtained in step S412. The first Fourier transform result is a double integral expression, and the inner integral can represent the angular distribution of the spectral energy, which is highly similar to the zero-order Bessel function, which can be:

[0116]

[0117] The difference between the zero-order Bessel function and the inner integral is that the integrand of the inner integral has a modulation term of the window function W θ , which causes energy diffusion. The overall shape of the inner integral can be similar to the modulated Bessel function, denoted as Angular truncation causes the spectral energy to spread in the frequency domain angle y, but the main lobe is concentrated in the direction of q.

[0118] The outer integral of the first Fourier transform result represents the radial distribution. By substituting the result of the inner integral into the first Fourier transform result, the following expression can be obtained:

[0119]

[0120] wherein When p = f rThe resulting spectrum is a broad main lobe with a shape determined by the window function W r and a radial spread of energy due to the truncation effect, resulting in a radial band around the frequency.

[0121] From the above analysis, the spectrum of the deformed fringe pattern in the two-dimensional frequency domain presents an arc-shaped ring band structure, and on this basis, the filter parameters can be further refined through quantitative analysis of the spatial frequency distribution.

[0122] In a possible implementation, referring to Figure 8 , step S410 can further include the following sub-steps.

[0123] Step S414, determining a second expression of the deformed fringe image 210 based on the deformed fringe image 210.

[0124] In this embodiment, the phase of the deformed fringe image 210 carries three-dimensional information due to the modulation of the surface height of the object to be reconstructed 300, and therefore, the second expression can be:

[0125] I c = a(x, y) + b(x, y)cos[Φ(x, y)]

[0126] wherein a(x, y) can represent the average intensity of the deformed fringe image 210, b(x, y) can represent the modulation degree of the deformed fringe image 210, and Φ(x, y) represents the total phase of the deformed fringe image, which can include the initial carrier phase and the modulation phase.

[0127] Step S415, performing Fourier transform on the second expression of the deformed fringe image 210 to obtain a second Fourier transform result.

[0128] In this embodiment, the second expression of the deformed fringe image 210 can be simplified according to Euler's formula to obtain the following expression:

[0129] I c (x, y) = a(x, y) + c(x, y) + c * (x, y)

[0130] wherein

[0131] Performing Fourier transform on the above expression can obtain the second Fourier transform result:

[0132]

[0133] According to the second Fourier transform result, it can be known that the phase information is stored in the fundamental frequency component The phase information in the fundamental component can be extracted by designing a filter to obtain the truncated phase information.

[0134] In step S416, the spatial frequency of the deformed fringe pattern is calculated according to the second Fourier transform result.

[0135] In the present embodiment, the partial derivatives of the phase part of the second Fourier transform result in two coordinate axis directions are calculated to obtain the spatial frequencies of the fringe in x c and y c directions:

[0136]

[0137] where f can represent the spatial frequency of the fringe, (x c , y c ) can represent the coordinates of any point on the plane of the image acquisition device, r c can represent the distance from the origin of coordinates of any point on the plane of the image acquisition device, θ can represent the angle between any point on the plane of the image acquisition device and the coordinate axis, and k can be a normalization coefficient,

[0138] The above formula shows that the spatial frequency in the radial direction can be decomposed along the coordinate axis to obtain different components, which can form a circular equation in polar coordinates, which shows that the frequency spectrum presents a circular arc in an ideal case.

[0139] In step S417, the period number of the deformed fringe image 210 is calculated according to the spatial frequency of the deformed fringe pattern.

[0140] For a digital image, if the sampling rates in the x direction and the y direction are different, the fringe will appear stretched and distorted in a certain direction, so that it no longer presents a standard circular arc shape, which can also cause its frequency spectrum to finally present an elliptical shape rather than a circular shape. If the deformed fringe image 210 is displayed on a digital image of MxN, the period numbers in two coordinate axis directions after the discrete Fourier transform can be calculated by the following method:

[0141]

[0142] where (u, v) can represent the frequency domain coordinates, T x can represent the period number of the deformed fringe image 210 in the x coordinate axis direction, T y can represent the period number of the deformed fringe image 210 in the y coordinate axis direction, M and N can represent the resolution, f can represent the spatial frequency of the fringe, θ can represent the angle between any point on the plane of the image acquisition device and the coordinate axis, and k can be a normalization coefficient,

[0143] As can be seen from the above formula, the deformed fringe image 210 is in the shape of an ellipse in the frequency spectrum. Please refer to Figure 9 , Figure 9 The example shows the correspondence between the spatial frequency distribution of the deformed fringe image 210 and the frequency spectrum in the present embodiment. The fundamental frequency component in the frequency spectrum is in the shape of an ellipse and is limited within a certain angular range. In addition, the representation of the fringe in the spatial domain corresponds to the features in the frequency domain one by one.

[0144] Therefore, the filter for the circular fringe can be designed according to the following method: the frequency spectrum is in the shape of an elliptical ring, and the ratio of the major axis to the minor axis is MxN. The lengths of the two major axes of the main peak in the frequency spectrum can be designed according to the above formula. That is, the filter can be designed to have an elliptical ring shape with a certain width, and the ratio of the inner and outer major axes needs to be kept as MxN. In addition, the angular range of the frequency spectrum ellipse is consistent with the angular range of the fringe pattern, so the angular range of the filter needs to completely cover the range. In actual design, the range of the starting and ending points of the frequency spectrum ellipse can be calculated according to the above formula to determine the angular coverage area of the filter.

[0145] The main geometric shape of the frequency spectrum is determined by the angular truncation of the circular fringe on the finite image plane, which limits the angular span of the ellipse in the frequency spectrum and enables the use of the circular fringe in Fourier transform profilometry. Please refer to Figure 9 , the fundamental frequency component and its conjugate component appear in the symmetric regions of the same ellipse. When the angular range is small, these components can be well separated. However, if the angular range increases and causes the frequency spectrum to overlap, the fundamental frequency component cannot be effectively separated, resulting in failure of phase reconstruction. Therefore, analysis is only performed within its effective range.

[0146] In one possible implementation, please refer to Figure 10 , step S500 can include the following sub-steps.

[0147] Step S510, performing phase unwrapping processing on the truncated phase information to obtain initial continuous phase information.

[0148] In the present embodiment, since the image acquisition device 200 only acquires one deformed fringe image 210, the truncated phase information can only be phase unwrapping processed by the spatial phase unwrapping method to obtain the initial continuous phase information, and the phase range of the initial continuous phase information is [-∞, +∞].

[0149] Step S520, performing value domain transformation processing on the initial continuous phase information to obtain target continuous phase information.

[0150] In the embodiment, according to the expression of the fringe, the phase range coded in the projection plane is [0, 2π]. In order to establish the corresponding relationship between the deformed fringe pattern and the circular fringe pattern, the value range transformation processing can be performed on the initial continuous phase information, so as to map it to the range of [0, 2π], thereby obtaining the target continuous phase information.

[0151] Specifically, the value range transformation processing can be performed on the initial continuous phase information in the following manner:

[0152]

[0153] Wherein, φ can represent the initial continuous phase information, f can represent the carrier frequency, φtarget can represent the target continuous phase information, i.e., the mapped phase, Δφ0 can represent the phase offset determined by the known reference point φ0.

[0154] Since the initial continuous phase information φ is obtained by spatial phase unwrapping, the value range thereof depends on the starting point of the spatial unwrapping. After the normalization processing using the carrier frequency f, there is only a constant offset between the initial continuous phase information φ and the real phase. By introducing the correction term Δφ0, the offset can be eliminated, so that the mapped phase is consistent with the value range of the real phase.

[0155] For the circular fringe pattern, the selection of the reference point φ0 is related to the position of the center of the circle. At the same time, there is a zero phase point on the image acquisition device plane, which corresponds to the zero phase point on the projection plane, i.e. Figure 3 the point The center of the circle and this zero phase point can determine the size of the phase offset Δφ0, and the coordinate transformation relationship between the point on the image acquisition device plane and the point on the projection plane can be represented by a homography matrix:

[0156]

[0157] Wherein, Ox can represent the center of the circle on the projection plane, Ox' can represent the center of the circle on the image acquisition device plane, Ox0 can represent the zero phase point on the projection plane, Ox0' can represent the zero phase point on the image acquisition device plane, H can represent the homography matrix, which can be used to describe the mapping relationship between the points on the projection plane and the image acquisition device plane, and can be obtained through the calibration process.

[0158] In the actual acquisition of the deformed fringe image 210, the zero phase point cannot be completely acquired within the limited field of view, and thus the point closest to the center in the radial direction can be selected as the reference point. The phase of the point can be calculated according to the radial distance of the point from the center.

[0159] In a possible implementation, referring to Figure 11 , the step S600 can include the following sub-steps.

[0160] In step S610, radial distance information of the standard circular fringe image 110 is determined according to the target continuous phase information; the radial distance information is used to represent the distance between each pixel point of the standard circular fringe image 110 and the center of the circular fringe pattern.

[0161] According to the encoding rule of the phase, the encoded phase corresponds to the radial distance of the point on the projection plane to the center of the fringe pattern, and thus in this embodiment, after the target continuous phase information is obtained, the radial distance information of the standard circular fringe image 110 can be calculated according to the target continuous phase information.

[0162] Specifically, the radial distance information r of the standard circular fringe image 110 can be calculated according to the following formula: p The radial distance information r of the standard circular fringe image 110 can be calculated according to the following formula:

[0163]

[0164] wherein, the target continuous phase information can be represented as φ, the minimum distance from the projection plane to the center can be represented as r min, and the maximum distance from the projection plane to the center can be represented as r max.

[0165] In step S620, polar distance information of the deformed fringe image 210 is calculated; the polar distance information is used to represent the distance between each pixel point in the deformed fringe image 210 and the phase pole point of the projection plane.

[0166] In this embodiment, the polar distance information of the deformed fringe image 210 and the coordinates of the optical pole point can be calculated according to the extended epipolar geometry theory.

[0167] In step S630, three-dimensional point cloud data is determined according to the radial distance information and the polar distance information.

[0168] In this embodiment, the three-dimensional point cloud data can be calculated according to the radial distance information obtained in step S610, the polar distance information obtained in step S620, and the coordinates of the optical pole point.

[0169] Specifically, the three-dimensional point cloud data (X w , Yw , Z w ) can be calculated by the following way:

[0170]

[0171] wherein, can represent the coordinates of the optical pole; r p can represent the radial distance information; can represent the polar radius information corresponding to the phase pole of the projection plane.

[0172] In order to verify the effectiveness of the present application, experiments are carried out on the present application.

[0173] In the present application, a camera with a resolution of 480x640 and a projector with a resolution of 600x800 can be used for experiments. In the calibration process, a calibration board composed of two planes with an included angle of 150° can be used for calibration, each plane of the calibration board includes 56 uniformly arranged hollow circular points, and the calibration of the system can be completed through the known coordinates of these circular points. After calibration, the object 300 to be reconstructed can be reconstructed by the single-frame circular fringe structured light three-dimensional reconstruction method provided in the present application, and the phase recovery accuracy, three-dimensional reconstruction accuracy and flatness are evaluated. For example, a standard plane can be reconstructed by the single-frame circular fringe structured light three-dimensional reconstruction method provided in the present application, and the phase recovery accuracy can be evaluated by the range (PV) and the root mean square error (RMSE), the range (PV) of the phase recovery accuracy of the present application can be 0.005, and the root mean square error (RMSE) of the phase recovery accuracy can be 3.399x10 -4 If straight stripes are used to reconstruct the standard plane, the range (PV) of the phase recovery accuracy can be 0.0055, and the root mean square error (RMSE) of the phase recovery accuracy can be 5.113x10 -4 Compared with the single-frame circular fringe structured light three-dimensional reconstruction method based on straight stripes, the single-frame circular fringe structured light three-dimensional reconstruction method provided in the present application has higher phase sensitivity and can effectively improve the reconstruction accuracy.

[0174] Based on the same inventive concept, the present embodiment also provides an electronic device 700, which can be a server 100 as shown in Figure 1 , please refer to Figure 12 , Figure 12 The example block schematic diagram of the electronic device 700. The electronic device 700 includes a single-frame circular fringe structured light three-dimensional reconstruction apparatus 730, a computer readable storage medium 720 and a processor 710.

[0175] The computer readable storage medium 720 and the processor 710 are electrically connected with each other directly or indirectly to realize the transmission or interaction of data. For example, the elements can be electrically connected with each other through one or more communication buses or signal lines. The single-frame circular stripe structured light three-dimensional reconstruction device 730 includes a plurality of software function modules which can be stored in the computer readable storage medium 720 in the form of software or firmware or solidified in the operating system (OS) of the single-frame circular stripe structured light three-dimensional reconstruction device 730. The processor 710 is configured to execute the executable modules stored in the computer readable storage medium 720, such as the software function modules included in the single-frame circular stripe structured light three-dimensional reconstruction device 730 and computer programs.

[0176] The computer readable storage medium 720 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) and the like. The computer readable storage medium 720 is configured to store programs, and the processor 710 executes the programs after receiving an execution instruction.

[0177] The processor 710 can be an integrated circuit chip having a signal processing capability. The processor 710 described above can be a general purpose processor 710, including a central processing unit (CPU), a network processor (NP), or the like; and can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The processor 710 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. The general purpose processor 710 can be a microprocessor or the processor 710 can also be any conventional processor.

[0178] Please refer to Figure 13The embodiment of the present application further provides a single-frame circular fringe structured light three-dimensional reconstruction device 730. The single-frame circular fringe structured light three-dimensional reconstruction device 730 comprises a plurality of functional modules which can be stored in the computer readable storage medium 720 in the form of software. Functionally, the single-frame circular fringe structured light three-dimensional reconstruction device 730 can comprise a receiving module 731, a projection module 732, an acquisition module 733, an extraction module 734, an unwrapping module 735 and a reconstruction module 736. Wherein:

[0179] The receiving module 731 can be used for acquiring a preset standard circular fringe image 110, wherein the standard circular fringe image 110 comprises a circular fringe pattern, and the center of the circular fringe pattern coincides with the pole point of the projection plane of the projection device 100.

[0180] In the embodiment, the receiving module 731 can be used for performing the step S100 shown in the figure, and the specific description of the receiving module 731 can refer to the description of the step S100. Figure 2

[0181] The projection module 732 can be used for projecting the standard circular fringe image 110 onto the object 300 to be reconstructed by the projection device 100, and modulating the standard circular fringe image 110 by the object 300 to be reconstructed.

[0182] In the embodiment, the projection module 732 can be used for performing the step S200 shown in the figure, and the specific description of the projection module 732 can refer to the description of the step S200. Figure 2

[0183] The acquisition module 733 can be used for acquiring the deformed fringe image 210 of the object 300 to be reconstructed by the image acquisition device 200, and the deformed fringe image 210 comprises a deformed fringe pattern.

[0184] In the embodiment, the acquisition module 733 can be used for performing the step S300 shown in the figure, and the specific description of the acquisition module 733 can refer to the description of the step S300. Figure 2

[0185] The extraction module 734 can be used for extracting the phase information of the deformed fringe image 210 to obtain the truncated phase information.

[0186] In the embodiment, the extraction module 734 can be used for performing the step S400 shown in the figure, and the specific description of the extraction module 734 can refer to the description of the step S400. Figure 2

[0187] ​​​​The unwrapping module 735 can be configured to unwrap the truncated phase information to obtain target continuous phase information.

[0188] In the embodiment, the unwrapping module 735 can be configured to perform Figure 2 The specific description of the unwrapping module 735 can refer to the description of the step S500.

[0189] The reconstruction module 736 can be configured to determine three-dimensional point cloud data based on the target continuous phase information, and reconstruct a three-dimensional model of the object 300 to be reconstructed based on the three-dimensional point cloud data.

[0190] In the embodiment, the reconstruction module 736 can be configured to perform Figure 2 The specific description of the reconstruction module 736 can refer to the description of the step S600.

[0191] In summary, the embodiments of the present application provide a single-frame circular stripe structured light three-dimensional reconstruction method, device and electronic equipment. The three-dimensional reconstruction is performed by using a standard circular stripe image, which can significantly improve the adaptability of single-frame stripe projection to the environment, and improve the reconstruction accuracy while ensuring the real-time of single-frame projection.

[0192] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0193] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the claims.

Claims

1. A method for three-dimensional reconstruction using single-frame circular stripe structured light, characterized in that, Applied to a structured light 3D reconstruction system, the structured light 3D reconstruction system includes a projection device and an image acquisition device; The method includes: A pre-set standard circular stripe image is obtained, the standard circular stripe image including a circular stripe pattern; the center of the circular stripe pattern coincides with the pole of the projection plane of the projection device; The standard circular stripe image is projected onto the object to be reconstructed using the projection device, and the standard circular stripe image is modulated by the object to be reconstructed. The image acquisition device acquires a deformed stripe image of the object to be reconstructed, the deformed stripe image including a deformed stripe pattern; Extract the phase information of the deformed stripe image to obtain truncated phase information; The truncated phase information is unwrapped to obtain the continuous phase information of the target. Based on the target continuous phase information, three-dimensional point cloud data is determined, and based on the three-dimensional point cloud data, a three-dimensional model of the object to be reconstructed is reconstructed.

2. The single-frame circular stripe structured light three-dimensional reconstruction method according to claim 1, characterized in that, The step of extracting the phase information of the deformed stripe image to obtain truncated phase information includes: The deformed stripe image is subjected to qualitative analysis based on spectral energy distribution and quantitative analysis based on spatial frequency distribution to obtain filter parameters; The filter is designed based on the filter parameters, and the phase information of the deformed stripe image is extracted through the filter to obtain truncated phase information.

3. The single-frame circular fringe structured light three-dimensional reconstruction method according to claim 2, characterized in that, The steps of performing qualitative analysis on the deformed stripe image based on spectral energy distribution include: A first expression for the deformed stripe image is determined based on the deformed stripe image; Perform a Fourier transform on the first expression of the deformed stripe image to obtain the first Fourier transform result; Qualitative analysis is performed on the deformed fringe image based on the first Fourier transform result to obtain the spectral energy distribution information of the deformed fringe image.

4. The single-frame circular stripe structured light three-dimensional reconstruction method according to claim 2, characterized in that, The steps for performing quantitative analysis of the deformed fringe image based on spatial frequency distribution include: A second expression for the deformed fringe image is determined based on the deformed fringe image; Perform a Fourier transform on the second expression of the deformed stripe image to obtain the second Fourier transform result; The spatial frequency of the deformed stripe pattern is calculated based on the result of the second Fourier transform; The number of periods of the deformed stripe image is calculated based on the spatial frequency of the deformed stripe pattern.

5. The single-frame circular stripe structured light three-dimensional reconstruction method according to claim 1, characterized in that, The step of unwrapping the truncated phase information to obtain the continuous phase information of the target includes: The truncated phase information is subjected to phase expansion processing to obtain initial continuous phase information; The initial continuous phase information is subjected to a range transformation to obtain the target continuous phase information.

6. The single-frame circular fringe structured light three-dimensional reconstruction method according to claim 1, characterized in that, The step of determining the three-dimensional point cloud data based on the target continuous phase information includes: The radial distance information of the standard circular stripe image is determined based on the target continuous phase information; the radial distance information is used to characterize the distance between each pixel of the standard circular stripe image and the center of the circular stripe pattern; Calculate the polar radius information of the deformed fringe image; the polar radius information is used to characterize the distance between each pixel in the deformed fringe image and the phase pole of the projection plane; The three-dimensional point cloud data is determined based on the radial distance information and the polar diameter information.

7. The single-frame circular stripe structured light three-dimensional reconstruction method according to claim 6, characterized in that, The three-dimensional point cloud data (X) w Y w Z w It is calculated in the following way: in, The coordinates of the optical poles; r p Indicates radial distance information; This indicates the polar radius information corresponding to the phase poles of the projection plane.

8. A single-frame circular stripe structured light three-dimensional reconstruction device, characterized in that, Applied to a structured light 3D reconstruction system, the structured light 3D reconstruction system includes a projection device and an image acquisition device; The single-frame circular stripe structured light 3D reconstruction device includes: A receiving module is used to acquire a pre-set standard circular stripe image, wherein the standard circular stripe image includes a circular stripe pattern; the center of the circular stripe pattern coincides with the pole of the projection plane of the projection device; A projection module is used to project the standard circular stripe image onto the object to be reconstructed through the projection device, and to modulate the standard circular stripe image through the object to be reconstructed; The acquisition module is used to acquire a deformed stripe image of the object to be reconstructed through the image acquisition device, wherein the deformed stripe image includes a deformed stripe pattern; The extraction module is used to extract the phase information of the deformed stripe image to obtain truncated phase information; The unwrapping module is used to unwrap the truncated phase information to obtain the target continuous phase information; The reconstruction module is used to determine three-dimensional point cloud data based on the target continuous phase information, and to reconstruct the three-dimensional model of the object to be reconstructed based on the three-dimensional point cloud data.

9. An electronic device, characterized in that, The electronic device includes a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions, which, when executed by the processor, implement the beam generation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, characterized in that, when the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images