A method and system for three-dimensional reconstruction of moving objects within a large depth range

By projecting non-periodic stripe patterns and using segmented linear transformation functions and depth variance minimization algorithm, the problem of three-dimensional reconstruction of moving objects in a large depth range is solved, and high-precision and fast automatic three-dimensional reconstruction is achieved without the need for additional stripe patterns Or space marking points.

CN114299226BActive Publication Date: 2024-11-29UNIV OF SCI & TECH OF CHINA
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111616298.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-27
Publication Date
2024-11-29
Estimated Expiration
2041-12-27

AI Technical Summary

Technical Problem

The prior art is difficult to effectively reconstruct moving objects in a large depth range (the range of phase domains greater than 2π), especially without adding spatial marking points and without requiring additional stripe patterns.

Method used

By configuring and projecting a non-periodic stripe pattern, the image is decoded to obtain a non-periodic wrapping phase, converted into candidate non-periodic absolute phase, and the correct periodic absolute phase is filtered out through a segmented linear transformation function and depth variance minimization algorithm, and finally combined with the structure The calibration parameters of the optical system are reconstructed in three-dimensionally.

Benefits of technology

Completely automatic, without manual intervention, three-dimensional reconstruction of moving objects in a large depth range is achieved, with high accuracy and fast speed, and no additional stripe patterns or spatial marking points are required.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114299226B_ABST
    Figure CN114299226B_ABST
Patent Text Reader

Abstract

The present application discloses a method and system for 3D reconstruction of moving objects within a large depth range, the method comprising: configuring and projecting a non-periodic stripe pattern; decoding the projected image to obtain a non-periodic wrapped phase, converting the non-periodic wrapped phase into a candidate non-periodic absolute phase; converting the candidate non-periodic absolute phase into a candidate periodic absolute phase through a piecewise linear transformation function, and then screening out the correct periodic absolute phase from the candidate periodic absolute phase through a depth variance minimization algorithm; combining the calibration parameters of the structured light system to obtain the 3D reconstruction result of the object. In this way, no additional stripe pattern is required, and no spatial marker points need to be added, so that fully automatic, human-free optical 3D reconstruction can be achieved with high accuracy and high speed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction of moving objects, and in particular to a method and system for three-dimensional reconstruction of moving objects within a large depth range. Background Art

[0002] Fringe projection profilometry is a high-precision dynamic three-dimensional surface imaging technology that uses active vision mechanisms. By solving the periodic fringe projection pattern, a wrapped phase with phase ambiguity can be obtained. In order to eliminate the phase ambiguity, it is necessary to unfold the wrapped phase and determine the fringe order.

[0003] Phase unwrapping methods are mainly divided into spatial phase unwrapping, temporal phase unwrapping and phase unwrapping based on geometric constraints. Among them, the spatial phase unwrapping method is committed to tracking the path of phase change and obtaining a continuous phase map. However, the uncertainty of the initial phase in the path and the propagation of the phase error jointly reduce the performance of space in phase recovery on complex surfaces. The temporal phase unwrapping method utilizes auxiliary fringe pattern projection, which has high accuracy and high robustness, but the additional projection time reduces the 3D imaging speed. The phase unwrapping method based on geometric constraints establishes a mapping relationship between absolute phase and spatial three-dimensional coordinates, and generates a minimum reference phase plane based on this relationship. Pixel-level phase unwrapping is performed through the reference plane, but when the motion depth range in the phase domain exceeds 2π, phase ambiguity will reappear.

[0004] Therefore, how to effectively perform three-dimensional reconstruction of moving objects within a large depth range (ie, a range where the phase domain is greater than 2π) is a technical problem that needs to be urgently solved by those skilled in the art. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method and system for 3D reconstruction of moving objects within a large depth range, without any additional stripe pattern or any spatial marker points, to achieve fully automatic 3D reconstruction without human intervention. The specific scheme is as follows:

[0006] A method for three-dimensional reconstruction of a moving object within a large depth range, comprising:

[0007] configuring and projecting a non-periodic fringe pattern;

[0008] Decoding the projected image to obtain a non-periodic wrapped phase, and converting the non-periodic wrapped phase into a candidate non-periodic absolute phase;

[0009] The candidate non-periodic absolute phase is converted into a candidate periodic absolute phase by a piecewise linear transformation function, and the periodic absolute phase corresponding to the minimum depth variance term is selected from the candidate periodic absolute phases by a depth variance minimization algorithm as the correct periodic absolute phase;

[0010] Combined with the calibration parameters of the structured light system, the three-dimensional reconstruction result of the object is obtained.

[0011] Preferably, in the above-mentioned method for 3D reconstruction of a moving object within a large depth range provided by an embodiment of the present invention, the configuring and projecting of a non-periodic fringe pattern includes:

[0012] According to a preset equivalent period, a group of non-periodic stripe patterns consisting of a combination of stripes with similar periods are configured;

[0013] The non-periodic fringe pattern is projected onto a surface of an object.

[0014] Preferably, in the above-mentioned method for 3D reconstruction of a moving object within a large depth range provided by an embodiment of the present invention, converting the aperiodic wrapped phase into a candidate aperiodic absolute phase comprises:

[0015] The non-periodic wrapped phase is converted into a non-periodic discontinuous phase using a pre-acquired reference plane phase and a round-up function;

[0016] Obtaining a phase analysis histogram by performing pixel-by-pixel subtraction between the non-periodic discontinuous phase and the reference plane phase;

[0017] Determine a discontinuous portion from the phase analysis histogram, and increase the phase value of the discontinuous portion by 2π to convert it into a non-periodic continuous phase;

[0018] A candidate non-periodic absolute phase is obtained by increasing the non-periodic continuous phase by an integer multiple of 2π.

[0019] Preferably, in the above-mentioned method for three-dimensional reconstruction of a moving object within a large depth range provided by an embodiment of the present invention, the converting of the candidate non-periodic absolute phase into a candidate periodic absolute phase by a piecewise linear transformation function, and screening out the periodic absolute phase corresponding to the minimum depth variance term from the candidate absolute phases as the correct periodic absolute phase by a depth variance minimization algorithm, comprises:

[0020] converting the candidate non-periodic absolute phase into a candidate periodic absolute phase by means of a piecewise linear transformation function between the non-periodic phase and the periodic phase;

[0021] A depth variance minimization algorithm is used to determine the depth variance value corresponding to the global phase order value of the candidate periodic absolute phase, and a minimum depth variance term is selected from the determined depth variance values. The periodic absolute phase corresponding to the minimum depth variance term is used as the correct periodic absolute phase.

[0022] Preferably, in the above-mentioned method for 3D reconstruction of a moving object within a large depth range provided by an embodiment of the present invention, the step of combining the calibration parameters of the structured light system to obtain the 3D reconstruction result of the object includes:

[0023] Combined with the calibration parameters of the structured light system, the correct periodic absolute phase is mapped into a depth distribution map of the object surface to obtain a three-dimensional reconstruction result of the object.

[0024] The embodiment of the present invention further provides a system for 3D reconstruction of moving objects within a large depth range, comprising:

[0025] A pattern configuration module, used for configuring and projecting a non-periodic stripe pattern;

[0026] A phase conversion module, configured to obtain a non-periodic wrapped phase after decoding the projected image, and convert the non-periodic wrapped phase into a candidate non-periodic absolute phase;

[0027] A phase screening module, used for converting the candidate non-periodic absolute phase into a candidate periodic absolute phase through a piecewise linear transformation function, and screening out the periodic absolute phase corresponding to the minimum term of the depth variance from the candidate periodic absolute phase as the correct periodic absolute phase through a depth variance minimization algorithm;

[0028] The 3D reconstruction module is used to obtain the 3D reconstruction result of the object in combination with the calibration parameters of the structured light system.

[0029] Preferably, in the above-mentioned three-dimensional reconstruction system for moving objects within a large depth range provided by an embodiment of the present invention, the pattern configuration module is specifically used to configure a group of non-periodic stripe patterns composed of a combination of multiple stripes with similar periods according to a preset equivalent period; and project the non-periodic stripe pattern onto the surface of the object.

[0030] Preferably, in the above-mentioned three-dimensional reconstruction system for moving objects within a large depth range provided by an embodiment of the present invention, the phase conversion module is specifically used to convert the non-periodic wrapped phase into a non-periodic discontinuous phase by using a pre-acquired reference plane phase and a rounding-up function; obtain a phase analysis histogram by taking a pixel-by-pixel difference between the non-periodic discontinuous phase and the reference plane phase; determine the discontinuous part from the phase analysis histogram, increase the phase value of the discontinuous part by 2π to convert it into a non-periodic continuous phase; and obtain a candidate non-periodic absolute phase by increasing the non-periodic continuous phase by an integer multiple of 2π.

[0031] Preferably, in the above-mentioned three-dimensional reconstruction system for moving objects within a large depth range provided by an embodiment of the present invention, the phase screening module is specifically used to convert the candidate non-periodic absolute phase into a candidate periodic absolute phase with the help of a piecewise linear transformation function between the non-periodic phase and the periodic phase, and use a depth variance minimization algorithm to determine the depth variance value corresponding to the global phase order value of the candidate periodic absolute phase, select the depth variance minimum term from the determined depth variance values, and the periodic absolute phase corresponding to the depth variance minimum term is used as the correct periodic absolute phase.

[0032] Preferably, in the above-mentioned three-dimensional reconstruction system for moving objects within a large depth range provided by an embodiment of the present invention, the three-dimensional reconstruction module is specifically used to map the correct periodic absolute phase into a depth distribution map of the object surface in combination with the calibration parameters of the structured light system to obtain the three-dimensional reconstruction result of the object.

[0033] It can be seen from the above technical scheme that the present invention provides a method for three-dimensional reconstruction of a moving object within a large depth range, including: configuring and projecting a non-periodic stripe pattern; decoding the projected image to obtain a non-periodic wrapped phase, and converting the non-periodic wrapped phase into a candidate non-periodic absolute phase; converting the candidate non-periodic absolute phase into a candidate periodic absolute phase through a piecewise linear transformation function, and screening out the periodic absolute phase corresponding to the minimum depth variance term from the candidate periodic absolute phases as the correct periodic absolute phase through a depth variance minimization algorithm; and obtaining the three-dimensional reconstruction result of the object in combination with the calibration parameters of the structured light system.

[0034] The above-mentioned 3D reconstruction method provided by the present invention can realize fully automatic, non-human-intervention optical 3D reconstruction without any additional stripe pattern or any spatial marker point, with high accuracy and high speed. In addition, the present invention also provides a corresponding system for the 3D reconstruction method of moving objects within a large depth range, which further makes the above-mentioned method more practical, and the system has corresponding advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0036] Figure 1 A flowchart of a method for three-dimensional reconstruction of a moving object within a large depth range provided by an embodiment of the present invention;

[0037] Figure 2A schematic diagram of an example of a non-periodic stripe design provided by an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of a three-dimensional imaging process provided by an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of depth variance statistics corresponding to five candidate non-periodic phase images at three depth positions provided by an embodiment of the present invention;

[0040] Figure 5 The embodiment of the present invention provides Figure 4 The corresponding object surface contour map;

[0041] Figure 6 Surface profile diagrams of a moving object at six depth positions within a large depth range provided by an embodiment of the present invention.

[0042] Figure 7 A schematic diagram of the structure of a system for 3D reconstruction of moving objects within a large depth range provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] The present invention provides a method for three-dimensional reconstruction of moving objects within a large depth range, such as Figure 1 As shown, the following steps are included:

[0045] S101, configuring and projecting a non-periodic stripe pattern;

[0046] S102, decoding the projected image to obtain a non-periodic wrapped phase, and converting the non-periodic wrapped phase into a candidate non-periodic absolute phase;

[0047] S103, converting the candidate non-periodic absolute phase into a candidate periodic absolute phase through a piecewise linear transformation function, and selecting the periodic absolute phase corresponding to the minimum depth variance term from the candidate periodic absolute phases as the correct periodic absolute phase through a depth variance minimization (DVM) algorithm;

[0048] S104 . Acquire a three-dimensional reconstruction result of the object by combining calibration parameters of the structured light system.

[0049] In the above-mentioned three-dimensional reconstruction method for moving objects within a large depth range provided by an embodiment of the present invention, without any additional fringe patterns or any additional spatial marker points, fully automatic and human-free optical three-dimensional reconstruction of moving objects within a large depth range (i.e., a range in which the phase domain is greater than 2π) can be achieved with high accuracy and high speed.

[0050] Furthermore, in a specific implementation, in the above-mentioned three-dimensional reconstruction method for moving objects within a large depth range provided in an embodiment of the present invention, step S101 configures and projects a non-periodic stripe pattern, which may specifically include: first, configuring a group of non-periodic stripe patterns composed of a combination of multiple stripes with similar periods according to a preset equivalent period; then, projecting the non-periodic stripe pattern onto the surface of a complex moving object to obtain three-step phase-shifted non-periodic stripes corresponding to the object.

[0051] Specifically, an equivalent period T is predetermined, that is, the non-periodic fringe pattern takes T as the equivalent period and is composed of N different fringe period components {T1, T2, ..., T n ,...,T N}, various stripe combination sequences can be obtained. The following formula is satisfied:

[0052]

[0053] Wherein, n=1, 2, …, N. Compared with the periodic fringe pattern coding, each 2π phase change interval is unique.

[0054] Furthermore, in a specific implementation, in the above-mentioned three-dimensional reconstruction method for moving objects within a large depth range provided in an embodiment of the present invention, step S102 converts the non-periodic wrapped phase into a candidate non-periodic absolute phase, which may specifically include: first, using a pre-acquired reference plane phase and a rounding function, the non-periodic wrapped phase is converted into a non-periodic discontinuous phase; then, a phase analysis histogram is obtained by subtracting the non-periodic discontinuous phase from the reference plane phase pixel by pixel; a discontinuous part is determined from the phase analysis histogram, and the phase value of the discontinuous part is increased by 2π to convert it into a non-periodic continuous phase; finally, a series of candidate non-periodic absolute phases can be obtained by increasing the non-periodic continuous phase map by integer multiples of 2π.

[0055] Specifically, in the preparation process, the minimum period absolute phase Φ is extracted from the dual-frequency periodic fringes in advance. min , which is the absolute phase of the reference plane. min Applied to a non-periodic fringe pattern, generating a minimum reference phase Ψ min As the relative reference phase of the non-periodic fringes.

[0056] The acquired three-step phase-shifted non-periodic fringes are decoded to obtain the non-periodic wrapped phase Using the minimum reference phase Ψ min Unfolding the non-periodic wrapped phase As shown below:

[0057]

[0058] in, represents the upward rounding function, M is the binary mask representing the discontinuous phase position, and m is the global phase order. The phase Ψ expanded by this formula is discontinuous, and the discontinuous phase Ψ is compared with the minimum reference phase Ψ min The phase analysis histogram is obtained by performing pixel-by-pixel difference calculation. The discontinuous part (M represents the discontinuous area) can be determined according to the phase analysis histogram, and the phase value of the discontinuous part is increased by 2π to convert the non-periodic discontinuous phase into a non-periodic continuous phase. By adding m (m = 0, 1, 2, 3, 4...) 2π to the non-periodic continuous phase map, a series of candidate non-periodic absolute phases Ψ can be obtained.

[0059] The above process can be called a progressive phase recovery strategy. It is called "progressive" because the non-periodic wrapped phase is firstly recovered using the reference phase plane Ψ min and the upward rounding function to obtain a discontinuous phase diagram; then operate on this discontinuous phase diagram, that is, compare the phase diagram with the reference phase plane Ψ min Subtract, so that the pixel area where the discontinuous phase is located can be determined by histogram statistics, and these pixels are added with 2π operation, so that a continuous phase map is obtained. Add 2mπ (m=0,1,2,3,4…) to this continuous phase map, and a series of candidate non-periodic absolute phases can be obtained. Since this process is constantly moving forward, it is named "progressive".

[0060] Further, in the specific implementation, in the above-mentioned method for 3D reconstruction of a moving object within a large depth range provided by an embodiment of the present invention, step S103 converts the candidate non-periodic absolute phase into a candidate periodic absolute phase through a piecewise linear transformation function, and selects the periodic absolute phase corresponding to the minimum depth variance term from the candidate periodic absolute phases as the correct periodic absolute phase through a depth variance minimization algorithm, which may specifically include:

[0061] First, a piecewise linear transformation function g(h,q) between the non-periodic phase and the periodic phase is defined, and the transformation relationship is as follows:

[0062] Φ(x,y)= g(h,q) [Ψ(x,y)]

[0063] Where h represents the number of the current N×T cycle, q represents the order of the stripes in the current cycle, and the range is q=1,2,...,N, (x,y) represents any pixel point on the phase map, Φ(x,y) represents the periodic absolute phase map, and Ψ(x,y) represents the non-periodic absolute phase map.

[0064] Here is a detailed description of g(h,q):

[0065]

[0066] Where L represents the length of the fringe sequence in the x direction. In addition, i and ΔL are defined as:

[0067]

[0068] Through this piecewise linear transformation function, each 2π phase range in the non-periodic phase diagram has a specific transformation parameter, which realizes the transformation from the candidate non-periodic absolute phase to the candidate periodic absolute phase.

[0069] Then, a depth variance minimization algorithm is used to determine the depth variance value corresponding to the global phase order value of the candidate periodic absolute phase, and the depth variance minimum term is selected from the determined depth variance values. The periodic absolute phase corresponding to the depth variance minimum term is used as the correct equivalent periodic absolute phase.

[0070] Specifically, due to the unknown height distribution of the object surface, it is difficult to intuitively distinguish the correct periodic absolute phase image from the incorrect periodic absolute phase image. w ,y w ,z w ) calibrated structured light system, the depth distribution z of the object surface is expressed as follows:

[0071] z=f(Φ,T,P p ,P c )

[0072] Where Φ is the absolute phase of the cycle, T is the fringe period, and P p and P c are the projector and camera projection matrices respectively. Therefore, the conversion error caused by the wrong periodic absolute phase will produce additional distortion on the original depth distribution, so the correct periodic absolute phase is determined by identifying the additional distortion.

[0073] In order to solve the correct global phase order, the present invention proposes a deep variance minimization algorithm (DVM). The global phase order m can be obtained by the following method:

[0074] m=argminF v (m)

[0075] Among them, Fv (m) represents the depth variance value (unit: mm) after the phase image is converted into depth at the mth order. 2 ), F v (m) is defined as:

[0076]

[0077] Where W represents the number of imaging points, and the superscript u indicates that the variance calculation is performed within every 2π range on the surface of the object to ensure statistical accuracy. The minimum depth variance term is selected from multiple candidate global phase orders m as the correct global order to obtain the final correct periodic absolute phase.

[0078] Furthermore, in a specific implementation, in the above-mentioned three-dimensional reconstruction method of a moving object within a large depth range provided in an embodiment of the present invention, step S104 combines the calibration parameters of the structured light system to obtain a three-dimensional reconstruction result of the object, which may specifically include: combining the calibration parameters of the structured light system, mapping the equivalent periodic absolute phase into a depth distribution map of the object surface, and obtaining a three-dimensional reconstruction result of the object.

[0079] Specifically, combined with the calibration parameters of the structured light system, the correct periodic absolute phase can be mapped into the correct depth distribution map of the object surface to achieve three-dimensional reconstruction.

[0080] From the above content, it can be seen that the above-mentioned three-dimensional reconstruction method of a moving object within a large depth range provided by an embodiment of the present invention can specifically include: designing and projecting a group of non-periodic stripe patterns composed of a combination of multiple stripes with similar periods according to a preset equivalent period, so that each 2π phase change interval is unique; for three-dimensional imaging of moving objects within a large depth range of complex objects, a progressive phase recovery strategy is used to convert the acquired object's wrapped phase into a continuous phase and then into a candidate non-periodic absolute phase; a piecewise linear transformation function is used to convert the candidate non-periodic absolute phase into a candidate periodic absolute phase, and the correct periodic absolute phase is determined from the candidate periodic absolute phase by the depth variance minimization method, and then combined with the system marker parameters to achieve high-precision three-dimensional imaging. The entire process does not require any additional stripe patterns, and does not require the addition of any spatial marker points, so that fully automatic and human-free three-dimensional reconstruction of moving objects can be achieved.

[0081] To verify the effectiveness of the present invention, the following specific experiments are performed:

[0082] In the first experiment, Figure 2As shown, a non-periodic stripe sequence with four non-periodic components (62 pixels, 72 pixels, 80 pixels, and 86 pixels) is designed. A DLP projector is used to project three phase-shifted stripes onto a statue with uneven reflectivity, and a camera records the projection pattern. The three-dimensional reconstruction method provided by the present invention is used to perform three-dimensional reconstruction. Figure 3 The three-dimensional imaging process of the three-dimensional reconstruction method provided by the present invention is demonstrated, and the surface contours of all candidate periodic continuous phases are reconstructed within the imaging depth range, among which only one surface contour is correct.

[0083] In the second experiment, the statues were placed at different spatial positions within a large depth range (>2π), and the correct global phase order corresponding to each spatial position was calculated using a depth variance minimization algorithm. Figure 4 Show the depth variation corresponding to the possible phase sequence, and then Figure 5 The correct surface contours of the statue were restored.

[0084] In the third experiment, the dynamic imaging of a moving fan was completed by using the 3D reconstruction method provided by the present invention. Figure 6 The surface profile of the fan at six different depths is restored.

[0085] Based on the same inventive concept, an embodiment of the present invention also provides a three-dimensional reconstruction system for moving objects within a large depth range. Since the principle of solving the problem by this system is similar to that of the aforementioned three-dimensional reconstruction method for moving objects within a large depth range, the implementation of this system can refer to the implementation of the three-dimensional reconstruction method for moving objects within a large depth range, and the repeated parts will not be repeated here.

[0086] In specific implementation, the embodiment of the present invention provides a 3D reconstruction system for moving objects within a large depth range, such as Figure 7 As shown, specifically including:

[0087] A pattern configuration module 701, used to configure and project a non-periodic stripe pattern;

[0088] A phase conversion module 702 is used to obtain a non-periodic wrapped phase after decoding the projected image, and convert the non-periodic wrapped phase into a candidate non-periodic absolute phase;

[0089] The phase screening module 703 is used to convert the candidate non-periodic absolute phase into a candidate periodic absolute phase through a piecewise linear transformation function, and screen out the periodic absolute phase corresponding to the minimum depth variance term from the candidate periodic absolute phases as the correct periodic absolute phase through a depth variance minimization algorithm;

[0090] The three-dimensional reconstruction module 704 is used to obtain a three-dimensional reconstruction result of the object in combination with the calibration parameters of the structured light system.

[0091] In the above-mentioned three-dimensional reconstruction system for moving objects within a large depth range provided by an embodiment of the present invention, through the interaction of the above-mentioned four modules, without any additional stripe patterns and without adding any spatial marker points, fully automatic, human-free optical three-dimensional reconstruction with high accuracy and high speed can be achieved.

[0092] Furthermore, in a specific implementation, in the above-mentioned three-dimensional reconstruction system of a moving object within a large depth range provided in an embodiment of the present invention, the pattern configuration module 701 can be specifically used to configure a group of non-periodic stripe patterns composed of a combination of multiple stripes with similar periods according to a preset equivalent period; and project the non-periodic stripe pattern onto the surface of the object.

[0093] Specifically, the pattern configuration module 701 may first determine an equivalent period T, that is, the non-periodic stripe pattern takes T as the equivalent period and is composed of N different stripe period components {T1, T2, ..., T n ,...,T N}, various stripe combination sequences can be obtained. The following formula is satisfied:

[0094]

[0095] Wherein, n=1, 2, …, N. Compared with the periodic fringe pattern coding, each 2π phase change interval is unique.

[0096] Furthermore, in a specific implementation, in the above-mentioned three-dimensional reconstruction system for moving objects within a large depth range provided in an embodiment of the present invention, the phase conversion module 702 can be specifically used to convert the non-periodic wrapped phase into a non-periodic discontinuous phase by using a pre-acquired reference plane phase and a rounding-up function; obtain a phase analysis histogram by taking a pixel-by-pixel difference between the non-periodic discontinuous phase and the reference plane phase; determine the discontinuous part from the phase analysis histogram, increase the phase value of the discontinuous part by 2π to convert it into a continuous phase, and obtain a series of candidate non-periodic absolute phases by increasing the non-periodic continuous phase map by integer multiples of 2π.

[0097] Specifically, in the preparation process, the minimum period absolute phase Φ is extracted from the dual-frequency periodic fringes in advance. min , which is the absolute phase of the reference plane. min Applied to a non-periodic fringe pattern, generating a minimum reference phase Ψ min As the relative reference phase of the non-periodic fringes.

[0098] The acquired three-step phase-shifted non-periodic fringes are decoded to obtain the non-periodic wrapped phase Using the minimum reference phase Ψ min Unfolding the non-periodic wrapped phase As shown below:

[0099]

[0100] in, represents the rounding function, M is the binary mask representing the discontinuous phase position, and m is the global phase order. The phase Ψ expanded by this formula is discontinuous, so the phase conversion module 702 can be specifically used to convert the discontinuous phase Ψ to the minimum reference phase Ψ min The phase analysis histogram is obtained by performing pixel-by-pixel difference calculation. The discontinuous part can be determined according to the phase analysis histogram, and the phase value of the discontinuous part is increased by 2π to convert the non-periodic discontinuous phase into a non-periodic continuous phase. By adding m (m = 0, 1, 2, 3, 4...) 2π to the non-periodic continuous phase map, a series of candidate non-periodic absolute phases Ψ can be obtained.

[0101] Furthermore, in a specific implementation, in the above-mentioned three-dimensional reconstruction system for moving objects within a large depth range provided in an embodiment of the present invention, the phase screening module 703 can be specifically used to convert the candidate non-periodic absolute phase into a candidate periodic absolute phase with the help of a piecewise linear transformation function between the non-periodic phase and the periodic phase, and use a depth variance minimization algorithm to determine the depth variance value corresponding to the global phase order value of the candidate periodic absolute phase, select the depth variance minimum term from the determined depth variance values, and the periodic absolute phase corresponding to the depth variance minimum term is used as the correct periodic absolute phase.

[0102] Specifically, in the phase screening module 703, a piecewise linear transformation function is used to transform the candidate non-periodic absolute phase into a candidate periodic absolute phase, and a piecewise linear transformation function g(h,q) is defined, and the transformation relationship is as follows:

[0103] Φ(x,y)= g(h,q) [Ψ(x,y)]

[0104] Where h represents the number of the current N×T cycle, q represents the order of the stripes in the current cycle, the range is q=1,2,...,N, (x,y) represents any pixel point on the phase map, and Φ(x,y) represents the periodic absolute phase map.

[0105] Here is a detailed description of g(h,q):

[0106]

[0107] Where L represents the length of the fringe sequence in the x direction. In addition, i and ΔL are defined as:

[0108]

[0109] Through this piecewise linear transformation function, each 2π phase range in the non-periodic phase diagram has a specific transformation parameter, which realizes the transformation from the candidate non-periodic absolute phase to the candidate periodic absolute phase.

[0110] Then use the deep variance minimization algorithm to solve the correct global phase order. The global phase order m can be obtained by the following method:

[0111] m=argminF v (m)

[0112] F v (m) is defined as:

[0113]

[0114] Wherein, W represents the number of imaging points, and the superscript u indicates that the variance calculation is performed within every 2π range on the surface of the object to ensure the accuracy of statistics. The phase screening module 703 can select the minimum depth variance term as the correct global order from multiple candidate global phase orders m to obtain the final correct periodic absolute phase.

[0115] Furthermore, in a specific implementation, in the three-dimensional reconstruction system for moving objects within a large depth range provided in the embodiment of the present invention, the three-dimensional reconstruction module 704 can be specifically used to map the correct periodic absolute phase into a depth distribution map of the object surface in combination with the calibration parameters of the structured light system to obtain a three-dimensional reconstruction result of the object.

[0116] For more specific working processes of the above modules, please refer to the corresponding contents disclosed in the above embodiments, which will not be repeated here.

[0117] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0118] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and configuration constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0119] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0120] In summary, an embodiment of the present invention provides a method for three-dimensional reconstruction of a moving object within a large depth range, including: configuring and projecting a non-periodic stripe pattern; decoding the projected image to obtain a non-periodic wrapped phase, and converting the non-periodic wrapped phase into a candidate non-periodic absolute phase; converting the candidate non-periodic absolute phase into a candidate periodic absolute phase through a piecewise linear transformation function, and then screening out the correct periodic absolute phase from the candidate periodic absolute phase through a depth variance minimization algorithm; combining the calibration parameters of the structured light system to obtain the three-dimensional reconstruction result of the object. This method does not require any additional stripe patterns, and does not require the addition of any spatial markers, and can achieve fully automatic, non-human intervention optical three-dimensional reconstruction with high accuracy and high speed. In addition, the present invention also provides a corresponding system for the three-dimensional reconstruction method of a moving object within a large depth range, which further makes the above method more practical, and the system has corresponding advantages.

[0121] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0122] The above is a detailed introduction to the method and system for three-dimensional reconstruction of moving objects within a large depth range provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for three-dimensional reconstruction of moving objects within a large depth range, characterized in that: include: configuring and projecting a non-periodic fringe pattern; Decoding the projected image to obtain a non-periodic wrapped phase, and converting the non-periodic wrapped phase into a non-periodic discontinuous phase using a pre-acquired reference plane phase and an upward rounding function; Obtaining a phase analysis histogram by performing pixel-by-pixel subtraction between the non-periodic discontinuous phase and the reference plane phase; Determine a discontinuous portion from the phase analysis histogram, and increase the phase value of the discontinuous portion by 2π to convert it into a non-periodic continuous phase; Obtaining a candidate non-periodic absolute phase by increasing the non-periodic continuous phase by an integer multiple of 2π; The candidate non-periodic absolute phase is converted into a candidate periodic absolute phase by a piecewise linear transformation function, and a depth variance minimization algorithm is used to determine a depth variance value corresponding to a global phase order value of the candidate periodic absolute phase, and a depth variance minimum term is selected from the determined depth variance values, and the periodic absolute phase corresponding to the depth variance minimum term is used as the correct periodic absolute phase; Combined with the calibration parameters of the structured light system, the correct periodic absolute phase is mapped into a depth distribution map of the object surface to obtain a three-dimensional reconstruction result of the object.

2. The method for 3D reconstruction of moving objects within a large depth range according to claim 1, characterized in that: The configuring and projecting of the non-periodic fringe pattern comprises: According to a preset equivalent period, a group of non-periodic stripe patterns consisting of a combination of stripes with similar periods are configured; The non-periodic fringe pattern is projected onto a surface of an object.

3. The method for 3D reconstruction of moving objects within a large depth range according to claim 2, characterized in that: The converting the candidate non-periodic absolute phase into a candidate periodic absolute phase by a piecewise linear transformation function comprises: The candidate non-periodic absolute phase is converted into a candidate periodic absolute phase by means of a piecewise linear transformation function between the non-periodic phase and the periodic phase.

4. A three-dimensional reconstruction system for moving objects within a large depth range, characterized in that: include: A pattern configuration module, used for configuring and projecting a non-periodic stripe pattern; A phase conversion module is used to obtain a non-periodic wrapped phase after decoding the projected image, and convert the non-periodic wrapped phase into a non-periodic discontinuous phase by using a pre-acquired reference plane phase and a rounding function; and obtain a phase analysis histogram by subtracting the non-periodic discontinuous phase from the reference plane phase pixel by pixel; Determine a discontinuous portion from the phase analysis histogram, increase the phase value of the discontinuous portion by 2π to convert it into a non-periodic continuous phase; obtain a candidate non-periodic absolute phase by increasing the non-periodic continuous phase by an integer multiple of 2π; A phase screening module is used to convert the candidate non-periodic absolute phase into a candidate periodic absolute phase through a piecewise linear transformation function, and use a depth variance minimization algorithm to determine the depth variance value corresponding to the global phase order value of the candidate periodic absolute phase, select the depth variance minimum term from the determined depth variance values, and the periodic absolute phase corresponding to the depth variance minimum term is used as the correct periodic absolute phase; The three-dimensional reconstruction module is used to map the correct periodic absolute phase into a depth distribution map of the object surface in combination with the calibration parameters of the structured light system to obtain a three-dimensional reconstruction result of the object.

5. The system for 3D reconstruction of moving objects within a large depth range according to claim 4, characterized in that: The pattern configuration module is specifically used to configure a group of non-periodic stripe patterns composed of a combination of multiple stripes with similar periods according to a preset equivalent period; and project the non-periodic stripe pattern onto the surface of the object.

6. The system for 3D reconstruction of moving objects within a large depth range according to claim 5, characterized in that: The phase screening module is specifically used to convert the candidate non-periodic absolute phase into a candidate periodic absolute phase by means of a piecewise linear transformation function between the non-periodic phase and the periodic phase.

Citation Information

Patent Citations

  • Structure light micro-nano structure three-dimensional detection method based on composite grating

    CN109916331A

  • Adaptive structured light three-dimensional measurement system and method based on geometric constraint

    CN110345882A