Large depth-of-field three-dimensional measurement method for phase-guided ray tracing

Abstract

The invention discloses a large depth-of-field three-dimensional measurement method for phase-guided ray tracing, and the method comprises the steps: carrying out the collection of orthogonal fringe patterns displayed on displays at a plurality of placement positions and orthogonal fringes projected on plane targets at a plurality of target positions through a light field camera, and correspondingly constructing a global coordinate system, performing sub-pixel light fitting through a second cost function, constructing a phase-projection light mapping table to map and obtain projection light corresponding to the orthogonal stripes projected on the surface of the to-be-measured object and light of the multi-view camera, and performing fusion calculation to obtain three-dimensional coordinates of the to-be-measured object. According to the method, high-precision and complete three-dimensional reconstruction in a calibration space and outside an extended depth range is realized by utilizing the characteristic of large depth of field of the light field camera and the two-dimensional scanning structured light system and tracking and fusing the projection light and the light of the multi-view camera, and the measurement precision of the three-dimensional light field is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of optical three-dimensional digital imaging, and in particular to a large depth-of-field three-dimensional measurement method using phase-guided ray tracing. Background Art

[0002] Fringe Projection Profilometry (FPP) is widely recognized for its non-contact, high-speed, and high-precision 3D reconstruction capabilities and is indispensable in fields such as industrial inspection, biomedical imaging, and reverse engineering. With the growing demand for accurate 3D positioning and measurement over large spatial ranges in robot-guided assembly, achieving a large depth of field (DOF) while maintaining high measurement accuracy has become a key research direction. Existing FPP systems rely on the imaging mechanism of traditional lenses and are naturally limited by the trade-off between DOF and resolution. Significant depth variations cause the limited DOF of the camera and projector to produce defocus effects, which seriously affects the accuracy of 3D reconstruction of objects over a large depth range.

[0003] Advances in imaging and projection technologies offer new opportunities to overcome the DOF limitations of traditional methods. Light field imaging (LFI), a revolutionary technology, generates a four-dimensional light field by simultaneously recording spatial and angular information. Typical light field cameras achieve multi-view image acquisition by introducing a microlens array (MLA) in front of the image sensor. Compared to conventional cameras, MLAs divide the main aperture into multiple subapertures, so that each pixel under each microlens corresponds to a specific subaperture, significantly extending the depth of field. Furthermore, significant progress has been made in two-dimensional scanning structured light systems, such as micro-electro-mechanical system (MEMS) projection technology. Compared to traditional digital micromirror device (DMD) technology, MEMS projection generates patterns by scanning laser beams and offers advantages such as compactness, light weight, and no focal length limitations. The integration of LFI and MEMS projection technology offers a solution for truly extending the depth of field and achieving high-precision three-dimensional measurement, but it also presents several challenges. First, traditional imaging models cannot accurately characterize systems comprising MEMS projectors and light field cameras. Secondly, due to significant changes in depth range and field of view (FOV), illumination attenuation and image quality degradation exacerbate the difficulty of accurate calibration. Thirdly, brightness unevenness and spatial inconsistency caused by viewpoint changes adversely affect measurement accuracy. Therefore, existing methods for measuring three-dimensional light fields suffer from low accuracy. Summary of the Invention

[0004] The embodiment of the present invention provides a large depth of field three-dimensional measurement method using phase-guided ray tracing, aiming to solve the problem of low accuracy in the measurement methods used for three-dimensional light fields in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a large depth of field three-dimensional measurement method using phase-guided ray tracing, wherein the method includes:

[0006] Obtaining display images acquired by sequentially capturing an orthogonal fringe pattern display of a display at multiple placement positions within a calibration space using a light field camera, and calculating a first orthogonal phase distribution corresponding to each placement position of the display;

[0007] Constructing a global coordinate system corresponding to the first placement position according to the basic parameters of the light field camera and obtaining the pose parameters of the display through optimization using a first cost function;

[0008] Acquiring fringe images obtained by sequentially capturing orthogonal fringes projected by the light field camera onto a planar target placed at a plurality of target positions within the calibration space, and calculating a second orthogonal phase distribution corresponding to each target position; the orthogonal fringes are obtained by projecting a two-dimensional scanning structured light system onto the planar target;

[0009] Based on the placement position of the planar target and the parameters in the first cost function, fitting the sub-pixel light corresponding to the marker point through the second cost function and optimizing the target pose parameters through the third cost function;

[0010] Constructing a phase-projection light mapping table according to a preset phase shift parameter, the first orthogonal phase distribution, the second orthogonal phase distribution, and the second cost function;

[0011] Acquire a target image obtained by capturing the orthogonal fringes projected on the surface of the object to be measured by the light field camera, and calculate a third orthogonal phase distribution corresponding to the object to be measured;

[0012] performing phase matching on the third orthogonal phase distribution according to the phase-projection light mapping table to obtain corresponding projection light and multi-view camera light corresponding to the projection light;

[0013] The projection light and the multi-view camera light are fused and calculated through a fourth cost function to obtain the three-dimensional coordinates corresponding to the object to be measured; the fourth cost function is obtained based on the corresponding configuration of the second cost function, the posture parameters of the display and the posture parameters of the target.

[0014] In a second aspect, an embodiment of the present invention further provides a large depth of field three-dimensional measurement device using phase-guided ray tracing, wherein the device is used to perform the large depth of field three-dimensional measurement method using phase-guided ray tracing as described in the first aspect above, and the device includes:

[0015] a first calculation unit, configured to obtain display images acquired by sequentially capturing orthogonal fringe pattern displays of a display at multiple placement positions in a calibration space by a light field camera, and calculate a first orthogonal phase distribution corresponding to the display at each placement position;

[0016] A first construction unit is configured to construct a global coordinate system corresponding to a first placement position according to basic parameters of the light field camera and obtain pose parameters of the display through optimization using a first cost function;

[0017] a second calculation unit, configured to obtain fringe images obtained by sequentially capturing orthogonal fringes projected by the light field camera onto a planar target placed at a plurality of target positions within the calibration space, and calculate a second orthogonal phase distribution corresponding to each target position; the orthogonal fringes are obtained by projecting a two-dimensional scanning structured light system onto the planar target;

[0018] A pose parameter acquisition unit is configured to, based on the placement position of the planar target and the parameters in the first cost function, fit the sub-pixel light corresponding to the marker point using a second cost function and optimize the pose parameters of the target using a third cost function;

[0019] A second construction unit is configured to construct a phase-projection light mapping table according to a preset phase shift parameter, the first orthogonal phase distribution, the second orthogonal phase distribution, and the second cost function;

[0020] a third computing unit, configured to acquire a target image obtained by capturing the orthogonal fringes projected by the light field camera onto the surface of the object to be measured, and calculate a third orthogonal phase distribution corresponding to the object to be measured;

[0021] a phase matching unit, configured to perform phase matching on the third orthogonal phase distribution according to the phase-projection light mapping table to obtain corresponding projection light and multi-view camera light corresponding to the projection light;

[0022] A fourth calculation unit is used to fuse the projection light and the multi-view camera light through a fourth cost function to obtain the three-dimensional coordinates corresponding to the object to be measured; the fourth cost function is obtained based on the corresponding configuration of the second cost function, the posture parameters of the display and the posture parameters of the target.

[0023] In a third aspect, an embodiment of the present invention further provides a computer device, wherein the device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0024] Memory for storing computer programs;

[0025] The processor is configured to implement the steps of the large depth of field three-dimensional measurement method using phase-guided ray tracing as described in the first aspect above when executing the program stored in the memory.

[0026] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the large depth of field three-dimensional measurement method of phase-guided ray tracing as described in the first aspect above are implemented.

[0027] An embodiment of the present invention provides a large depth-of-field three-dimensional measurement method using phase-guided ray tracing. The method includes: using a light field camera to capture orthogonal fringe patterns displayed on displays at multiple locations and orthogonal fringe patterns projected onto planar targets placed at multiple target locations; constructing a corresponding global coordinate system; performing sub-pixel ray fitting using a second cost function; constructing a phase-to-projection ray mapping table to map the projected rays corresponding to the orthogonal fringe patterns projected on the surface of the object to be measured and performing a fusion calculation to obtain the three-dimensional coordinates of the object to be measured. This method utilizes the large depth of field characteristics of a light field camera and a two-dimensional scanning structured light system. By tracing and fusing projected rays and multi-view camera rays, it achieves high-precision, complete three-dimensional reconstruction within the calibration space and beyond the extended depth range, significantly improving the measurement accuracy of three-dimensional light fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 A flowchart of a method for large depth of field three-dimensional measurement using phase-guided ray tracing provided in an embodiment of the present invention;

[0030] Figure 2 Schematic diagram of an application scenario of the large depth of field three-dimensional measurement method of phase-guided ray tracing provided in an embodiment of the present invention;

[0031] Figure 3 A schematic diagram of another application scenario of the large depth of field three-dimensional measurement method using phase-guided ray tracing provided in an embodiment of the present invention;

[0032] Figure 4 A schematic diagram of another application scenario of the large depth-of-field three-dimensional measurement method using phase-guided ray tracing provided in an embodiment of the present invention;

[0033] Figure 5 A schematic diagram of another application scenario of the 3D reconstruction method based on multi-view fusion provided by an embodiment of the present invention;

[0034] Figure 6 A schematic block diagram of a large depth-of-field three-dimensional measurement device using phase-guided ray tracing provided in an embodiment of the present invention;

[0035] Figure 7 It is a schematic block diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0037] It will be understood that when used in this specification and the appended claims, the terms “comprises” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0038] It should also be understood that the terms used in the present specification are only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0039] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0040] See also Figure 1As shown in the figure, the embodiment of the present invention provides a large depth of field three-dimensional measurement method based on phase-guided ray tracing. The method is applied to a terminal device. The method is executed by an application software installed in the terminal device. The terminal device communicates with a light field camera, and the light field camera captures the image. The terminal device can be a laptop, desktop computer, tablet computer or mobile phone. Figure 1 As shown, the method includes steps S110 to S180.

[0041] S110 , obtaining display images acquired by sequentially capturing orthogonal fringe pattern displays of a display at multiple placement positions in a calibration space using a light field camera, and calculating a first orthogonal phase distribution corresponding to each placement position of the display.

[0042] like Figure 2 As shown, the displays are placed in different positions in sequence in the calibration space, and the displays display an orthogonal fringe pattern at each position. The light field camera synchronously captures the orthogonal fringe patterns displayed on the displays at different positions, and calculates the orthogonal phase distribution of each position as the corresponding first orthogonal phase distribution based on the captured display image.

[0043] S120 , constructing a global coordinate system corresponding to the first placement position according to basic parameters of the light field camera and obtaining pose parameters of the display through optimization using a first cost function.

[0044] Specifically, three coordinate systems can be defined: the global coordinate system (GCS), the LCD local coordinate system (LLCS), and the target local coordinate system (TLCS). The corresponding global coordinate system can be constructed based on the first placement position of the display. The superscripts L and T in the whole text refer to the LCD local coordinate system and the target local coordinate system, respectively. Then (R L ,T L ) and (R T ,T T ) are the rotation matrix and translation vector from the LCD-local coordinate system to the global coordinate system, and the rotation matrix and translation vector from the target coordinate system to the global coordinate system, respectively. The corresponding target coordinate system can be constructed based on the first target position of the planar target.

[0045] In a specific embodiment, step S120 includes sub-steps: constructing a global coordinate system with the first placement position as a reference; selecting a sampled phase reference point in the global coordinate system according to the spatial resolution in the basic parameters; determining the sampling light corresponding to the phase reference point and obtaining the three-dimensional point of the sampling light in the display image of each of the placement positions; transforming the phase reference point according to the global coordinate system to obtain a transformed coordinate point; and performing coincidence calculation on the transformed coordinate point and the phase reference point to obtain the posture parameters of the display through optimization of the first cost function.

[0046] Based on the constructed global coordinate system, the sampling phase reference point (also called sampling point) is selected according to the spatial resolution of the light field camera, the sampling light is determined according to the phase reference point, and the three-dimensional point of the sampling light in the display image at each placement position is further obtained. Figure 2 As shown, the sampling light corresponding to the i-th phase reference point is expressed as It corresponds to different displays (Π1, Π2, ..., Π m ) is a three-dimensional point m is the position number of the display. Take a certain pixel on the light field camera as an example, assuming that the horizontal and vertical phases of the pixel are The width of the stripes in the horizontal and vertical directions of the display are The pixel size of the display is Then the three-dimensional point corresponding to the image point is

[0047] The phase reference points are transformed according to the constructed global coordinate system to obtain the transformed coordinate points. All phase reference points can be expressed as Where P is the total number of sampled rays and M is the total number of display placement positions. The three-dimensional point of the local coordinate system can be converted into the conversion coordinate point corresponding to the global coordinate system is the rotation matrix and translation vector from the local coordinate system to the global coordinate system.

[0048] Further, according to the coincidence relationship between the conversion coordinate point and the phase reference point, the conversion coordinate point and the phase reference point are calculated to obtain the posture parameters of the display through the first cost function optimization. Among them, the conversion coordinate point corresponding to the phase reference point can be expressed as Phase reference point by light and the intersection of the corresponding m-th display plane, since the phase reference point is theoretically The corresponding transformation coordinate point They should be completely consistent, and the coincidence calculation can be performed based on the coincidence relationship between the two sets of coordinate points, thereby constructing a first cost function; the first cost function can be expressed by formula (1).

[0049]

[0050] in, The pose parameters of the display to be optimized, including sampling light parameters and the attitude parameters of each display These parameters can be initialized simply by assigning an initial direction to the ray and setting the pose parameters to zero.

[0051] S130 , obtaining fringe images obtained by sequentially capturing orthogonal fringes projected by the light field camera onto planar targets placed at multiple target positions in the calibration space, and calculating a second orthogonal phase distribution corresponding to each target position.

[0052] Further, such as Figure 3 As shown, a planar target is sequentially placed at different target positions within the same calibration space. Orthogonal fringes are projected and displayed on the planar target. These orthogonal fringes are projected onto the planar target by a two-dimensional scanning structured light system. A light field camera synchronously captures the orthogonal fringes of the planar target at different target positions, and the orthogonal phase distribution corresponding to each target position is calculated as the second orthogonal phase distribution.

[0053] S140. Based on the placement position of the planar target and the parameters in the first cost function, the sub-pixel light corresponding to the marker point is fitted by the second cost function and the pose parameters of the target are obtained by optimization using the third cost function.

[0054] The center coordinates of each planar target under the central viewing angle of the light field camera are extracted, the center coordinates are matched with the first orthogonal phase distribution constructed in step S110, and converted into physical coordinates. The physical coordinates are converted to the global coordinate system using the parameters optimized by the first cost function. The converted coordinates are fitted with sub-pixel rays using the second cost function. These sub-pixel rays can be used to obtain the pose parameters of the target based on the optimization of the third cost function.

[0055] In a specific embodiment, step S140 includes sub-steps: extracting the center coordinates corresponding to the placement positions of each of the planar targets under the central viewing angle of the light field camera; obtaining physical coordinates matching the center coordinates according to the first orthogonal phase distribution; converting the physical coordinates according to the first cost function and the global coordinate system to obtain corresponding target conversion coordinates; performing sub-pixel light fitting on the target conversion coordinates through the second cost function, and obtaining the target pose parameters corresponding to the fitted sub-pixel light through the third cost function.

[0056] Based on the positions of the planar targets, the center coordinates of the placement positions of each planar target under the central viewing angle of the light field camera can be extracted. The center coordinates are matched with the first orthogonal phase distribution (the orthogonal phase of the display in step S110) to convert the center coordinates into physical coordinates that match the first orthogonal phase distribution. The physical coordinates are further transformed using the parameters optimized by the first cost function (primarily based on the posture parameters) to obtain coordinates corresponding to the global coordinate system. These coordinates are named target transformation coordinates. Sub-pixel light fitting is performed on the target transformation coordinates using the second cost function, and the pose parameters of the target corresponding to the fitted sub-pixel light are obtained using the third cost function.

[0057] In a specific embodiment, the target conversion coordinates are subjected to sub-pixel light fitting through the second cost function, and the pose parameters of the target corresponding to the sub-pixel light obtained by fitting are obtained through the third cost function, including the sub-steps of: performing sub-pixel light fitting on the target conversion coordinates through the second cost function to obtain sub-pixel light corresponding to all marker points; converting the marker points according to the global coordinate system to obtain corresponding marker conversion points; calculating the overlap of the marker points and the marker conversion points according to the parameters in the first cost function, and obtaining the corresponding pose parameters through fitting the third cost function.

[0058] Specifically, the sub-pixel light fitting is performed on the target transformation coordinates through the second cost function, which can be expressed by formula (2):

[0059]

[0060] l=(A T A) -1 (A T b)

[0061] Among them, (X i ,Y i ,Z i )(i=1,2,...,Q) are three-dimensional points, Q is the number of three-dimensional points involved in fitting the straight line, is the light parameter. The sub-pixel light corresponding to all the landmark points can be obtained through sub-pixel light fitting. n,k (n=1,2,...,N;k=1,2,...,K), N is the total number of target postures, and K is the total number of landmark points.

[0062] The center coordinates of the marker point can be expressed as follows: The landmark points are transformed by the global coordinate system; specifically, using The three-dimensional points of the target coordinate system can be converted into three-dimensional coordinates of the global coordinate system. is the rotation matrix and translation vector from the target coordinate system to the global coordinate system. The corresponding marker transformation point can be obtained by transforming the marker point to the global coordinate system, which can be expressed as The transition point is marked by the light ray l n,k and the corresponding nth target plane.

[0063] In theory and They should be completely consistent, so the coincidence calculation of the landmark points and the landmark conversion points can be performed according to the parameters in the first cost function, and the corresponding pose parameters can be obtained by fitting the third cost function. The third cost function can be expressed by formula (3):

[0064]

[0065] in, is the target posture parameter to be optimized (ie, the position parameter), and the other parameters in the third cost function can be obtained from the first cost function.

[0066] S150 , constructing a phase-projection light mapping table according to a preset phase shift parameter, the first orthogonal phase distribution, the second orthogonal phase distribution, and the second cost function.

[0067] According to the preset phase shift parameters, and the first orthogonal phase distribution, the second orthogonal phase distribution and the second cost function obtained in the above steps, a phase-projection light mapping table can be constructed.

[0068] In a specific embodiment, step S150 includes the following sub-steps: calculating an orthogonal phase distribution corresponding to the first orthogonal phase distribution based on the phase shift parameter; obtaining, based on the phase shift parameter and the second orthogonal phase distribution, a three-dimensional point on a planar target at a central viewing angle of a light field camera that matches the phase shift parameter; fitting the three-dimensional point based on the second cost function to obtain a corresponding projection light; and constructing a corresponding phase-projection light mapping table based on a matching relationship between the orthogonal phase distribution and the projection light.

[0069] Calculate the orthogonal phase distribution corresponding to the first orthogonal phase distribution according to the set phase shift parameters h s and v s are the horizontal and vertical phases respectively, and S is the total number of projected rays. Figure 4As shown, according to the preset phase in the phase shift parameter, a three-dimensional point consistent with the preset phase is matched on the plane target of the central viewing angle of the light field camera in step S130, and the projection light corresponding to the three-dimensional point is fitted by formula (2). According to the matching relationship between the orthogonal phase distribution and the projection light, a corresponding phase-projection light mapping table (LUT) is established, and each orthogonal phase in the phase-projection light mapping table corresponds to a projection light.

[0070] S160 , obtaining a target image obtained by capturing the orthogonal fringes projected by the light field camera on the surface of the object to be measured, and calculating a third orthogonal phase distribution corresponding to the object to be measured.

[0071] A two-dimensional scanning structured light system projects orthogonal stripes onto the surface of the object to be measured, and the orthogonal stripes projected on the surface of the object to be measured are collected by a light field camera to obtain a target image; the orthogonal phase distribution of the object to be measured is calculated based on the corresponding target image as the corresponding third orthogonal phase distribution.

[0072] S170 , performing phase matching on the third orthogonal phase distribution according to the phase-projection light mapping table to obtain corresponding projection light and multi-view camera light corresponding to the projection light.

[0073] Phase matching is performed on the third orthogonal phase distribution through the phase-projection light mapping table to obtain projection light, and multi-view fitting calculation is further performed on the projection light to obtain multi-view camera light corresponding to the projection light.

[0074] In a specific embodiment, step S170 includes the sub-steps of: mapping the third orthogonal phase distribution according to the phase-projection ray mapping table to obtain matching projection rays; obtaining homonymous points corresponding to the projection rays at each viewing angle of the light field camera; and determining the camera rays corresponding to each of the homonymous points as multi-view camera rays.

[0075] like Figure 5 As shown, according to the phase-projection light mapping table, the third orthogonal phase distribution is found The same points are regarded as synonyms, and each synonym corresponds to a projection ray l in the phase-projection ray mapping table. s ; Further identify the same-name point x under each viewing angle of the light field camera i (i=1,2,...,V), where V is the total number of view angles. Based on the above sub-pixel ray fitting calculation method, sub-pixel ray fitting is performed for each homonymous point at different view angles to determine the camera ray corresponding to each homonymous point. The camera ray of each homonymous point can then be used as the multi-view camera ray.

[0076] S180 , performing a fusion calculation on the projection light and the multi-view camera light using a fourth cost function to obtain three-dimensional coordinates corresponding to the object to be measured.

[0077] The projection light obtained in the above steps is fused with the multi-view camera light through the fourth cost function to obtain the corresponding three-dimensional coordinates, which are the precise coordinates of the object to be measured in the three-dimensional light field; wherein the fourth cost function is obtained based on the corresponding configuration of the second cost function, the posture parameters of the display, and the posture parameters of the target. The three-dimensional coordinates corresponding to the object to be measured can be specifically expressed by formula (4):

[0078]

[0079] X=(A T WA) -1 A T Wb

[0080] Where W is the weight matrix, each weight w=w c w b , w c is the cosine of the angle between the incident light and the reflected light, w b To adjust the system, I n is the modulation pattern captured by the camera, δ n =2π(n-1) / N is the phase shift amount, and N is the phase shift number.

[0081] In a specific embodiment, before step S180, the method further includes the steps of: obtaining fitting parameters obtained by fitting the posture parameters of the display and the posture parameters of the target by the second cost function; and setting parameters of the fourth cost function according to the fitting parameters.

[0082] Furthermore, before executing step S180, the parameters of the fourth cost function can also be set. Specifically, the first cost function and the third cost function are optimized to obtain optimized parameters, and the optimized parameters can be fitted by the second cost function to obtain fitting parameters. The parameters of the fourth cost function can be set according to the obtained fitting parameters.

[0083] Compared with the existing technology, the present invention has the following advantages: the present invention provides a large depth of field three-dimensional measurement method based on phase-guided ray tracing, which can accurately characterize two-dimensional scanning structured light systems and light field cameras. It not only supports three-dimensional reconstruction across the calibration space, but also effectively integrates the multi-view information of the light field camera, significantly improving the overall measurement range and accuracy.

[0084] The phase-guided ray tracing method for large-depth-of-field three-dimensional measurement disclosed in the above-mentioned embodiments includes: using a light field camera to capture orthogonal fringe patterns displayed on displays at multiple locations and orthogonal fringe patterns projected onto planar targets placed at multiple target locations; constructing a corresponding global coordinate system; performing sub-pixel ray fitting using a second cost function; constructing a phase-to-projection ray mapping table to map the projected rays corresponding to the orthogonal fringe patterns projected on the surface of the object to be measured and performing a fusion calculation to obtain the three-dimensional coordinates of the object to be measured. This method utilizes the large depth of field characteristics of a light field camera and a two-dimensional scanning structured light system. By tracing and fusing projected rays and multi-view camera rays, it achieves high-precision, complete three-dimensional reconstruction within the calibration space and beyond the extended depth range, significantly improving the measurement accuracy of three-dimensional light fields.

[0085] The present invention also provides a phase-guided ray tracing 3D measurement device with a large depth of field. The phase-guided ray tracing 3D measurement device with a large depth of field can be configured in a terminal device. The phase-guided ray tracing 3D measurement device with a large depth of field is used to perform any embodiment of the phase-guided ray tracing 3D measurement method with a large depth of field. Specifically, please refer to Figure 6 , Figure 6 A schematic block diagram of a large depth-of-field three-dimensional measurement device using phase-guided ray tracing provided in an embodiment of the present invention.

[0086] like Figure 6 As shown, the large depth of field three-dimensional measurement device 100 using phase-guided ray tracing includes a first calculation unit 110, a first construction unit 120, a second calculation unit 130, a posture parameter acquisition unit 140, a second construction unit 150, a third calculation unit 160, a phase matching unit 170 and a fourth calculation unit 180.

[0087] The first calculation unit 110 is configured to obtain display images acquired by sequentially capturing orthogonal fringe pattern displays of a display at multiple placement positions in a calibration space using a light field camera, and calculate a first orthogonal phase distribution corresponding to each placement position of the display.

[0088] The first construction unit 120 is configured to construct a global coordinate system corresponding to the first placement position according to basic parameters of the light field camera and obtain pose parameters of the display through optimization using a first cost function.

[0089] The second calculation unit 130 is used to obtain fringe images obtained by sequentially capturing orthogonal fringes projected by the light field camera onto a planar target placed at multiple target positions in the calibration space, and calculate a second orthogonal phase distribution corresponding to each target position; the orthogonal fringes are obtained by projecting the two-dimensional scanning structured light system onto the planar target.

[0090] The pose parameter acquisition unit 140 is used to fit the sub-pixel light corresponding to the marker point through the second cost function based on the placement position of the planar target and the parameters in the first cost function, and to obtain the pose parameters of the target through optimization through the third cost function.

[0091] The second constructing unit 150 is configured to construct a phase-projection light mapping table according to a preset phase shift parameter, the first orthogonal phase distribution, the second orthogonal phase distribution, and the second cost function.

[0092] The third calculation unit 160 is configured to obtain a target image obtained by capturing the orthogonal fringes projected by the light field camera onto the surface of the object to be measured, and calculate a third orthogonal phase distribution corresponding to the object to be measured.

[0093] The phase matching unit 170 is configured to perform phase matching on the third orthogonal phase distribution according to the phase-projection light mapping table to obtain corresponding projection light and multi-view camera light corresponding to the projection light.

[0094] The fourth calculation unit 180 is used to fuse the projection light and the multi-view camera light through a fourth cost function to obtain the three-dimensional coordinates corresponding to the object to be measured; the fourth cost function is obtained based on the corresponding configuration of the second cost function, the posture parameters of the display and the posture parameters of the target.

[0095] The phase-guided ray tracing-based large-depth-of-field three-dimensional measurement device provided in an embodiment of the present invention utilizes the phase-guided ray tracing-based large-depth-of-field three-dimensional measurement method described above. A light field camera is used to capture orthogonal fringe patterns displayed on displays at multiple locations and orthogonal fringes projected onto planar targets placed at multiple target locations. A corresponding global coordinate system is constructed, and sub-pixel light fitting is performed using a second cost function. A phase-to-projection ray mapping table is constructed to map the projected light rays corresponding to the orthogonal fringes projected on the surface of the object to be measured and the multi-view camera rays, which are then fused and calculated to obtain the three-dimensional coordinates of the object to be measured. This method utilizes the large depth of field characteristics of a light field camera and a two-dimensional scanning structured light system. By tracking and fusing projected light rays with multi-view camera rays, it achieves high-precision, complete three-dimensional reconstruction within the calibration space and beyond the extended depth range, significantly improving the measurement accuracy of three-dimensional light fields.

[0096] The above-mentioned large depth of field three-dimensional measurement device of phase guided ray tracing can be realized in the form of a computer program. The computer program can be used in Figure 7 Runs on the computer device shown.

[0097] See also Figure 7 , Figure 7is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device may be a terminal device for executing a large depth of field three-dimensional measurement method using phase-guided ray tracing to perform three-dimensional reconstruction of an image in an imaging device and to achieve three-dimensional measurement of an object to be measured.

[0098] See Figure 7 The computer device 500 includes a processor 502 , a memory, and a communication interface 505 connected via a communication bus 501 , wherein the memory may include a storage medium 503 and an internal memory 504 .

[0099] The storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 may perform a large depth of field three-dimensional measurement method using phase-guided ray tracing. The storage medium 503 may be a volatile storage medium or a non-volatile storage medium.

[0100] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500.

[0101] The internal memory 504 provides an environment for the operation of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a large depth of field three-dimensional measurement method using phase-guided ray tracing.

[0102] The communication interface 505 is used for network communication, such as providing data information transmission. Those skilled in the art will understand that Figure 7 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention and does not constitute a limitation on the computer device 500 to which the solution of the present invention is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0103] The processor 502 is configured to run a computer program 5032 stored in the memory to implement corresponding functions in the above-mentioned phase-guided ray tracing method for three-dimensional measurement with a large depth of field.

[0104] Those skilled in the art will understand that Figure 7 The embodiment of the computer device shown in the figure does not constitute a limitation on the specific composition of the computer device. In other embodiments, the computer device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the computer device may only include a memory and a processor. In such an embodiment, the structure and function of the memory and processor are the same as those in the figure. Figure 7 The embodiments shown are consistent and will not be described again here.

[0105] It should be understood that in the embodiment of the present invention, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0106] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be volatile or non-volatile. The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps included in the aforementioned phase-guided ray tracing method for three-dimensional measurement with a large depth of field.

[0107] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0108] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, or units with the same function may be combined into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be an electrical, mechanical or other form of connection.

[0109] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

[0110] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0111] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a computer-readable storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned computer-readable storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0112] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A phase-guided ray tracing method for large depth of field three-dimensional measurement, characterized in that: The method comprises: Obtaining display images acquired by sequentially capturing an orthogonal fringe pattern display of a display at multiple placement positions within a calibration space using a light field camera, and calculating a first orthogonal phase distribution corresponding to each placement position of the display; Constructing a global coordinate system corresponding to the first placement position according to the basic parameters of the light field camera and obtaining the pose parameters of the display through optimization using a first cost function; Acquiring fringe images obtained by sequentially capturing orthogonal fringes projected by the light field camera onto a planar target placed at a plurality of target positions within the calibration space, and calculating a second orthogonal phase distribution corresponding to each target position; the orthogonal fringes are obtained by projecting a two-dimensional scanning structured light system onto the planar target; Based on the placement position of the planar target and the parameters in the first cost function, fitting the sub-pixel light corresponding to the marker point through the second cost function and optimizing the target pose parameters through the third cost function; Constructing a phase-projection light mapping table according to a preset phase shift parameter, the first orthogonal phase distribution, the second orthogonal phase distribution, and the second cost function; Acquire a target image obtained by capturing the orthogonal fringes projected on the surface of the object to be measured by the light field camera, and calculate a third orthogonal phase distribution corresponding to the object to be measured; performing phase matching on the third orthogonal phase distribution according to the phase-projection light mapping table to obtain corresponding projection light and multi-view camera light corresponding to the projection light; The projection light and the multi-view camera light are fused and calculated through a fourth cost function to obtain the three-dimensional coordinates corresponding to the object to be measured; the fourth cost function is obtained based on the corresponding configuration of the second cost function, the posture parameters of the display and the posture parameters of the target.

2. The large depth of field three-dimensional measurement method based on phase-guided ray tracing according to claim 1, characterized in that: The step of constructing a global coordinate system corresponding to the first placement position according to the basic parameters of the light field camera and obtaining the pose parameters of the display through optimization using a first cost function includes: Construct a global coordinate system based on the first placement position; Selecting a sampling phase reference point in the global coordinate system according to the spatial resolution in the basic parameters; Determining a sampling light corresponding to the phase reference point and obtaining a three-dimensional point of the sampling light in the display image at each of the placement positions; transforming the phase reference point according to the global coordinate system to obtain a transformed coordinate point; The conversion coordinate point and the phase reference point are overlapped and calculated to obtain the posture parameters of the display through optimization of a first cost function.

3. The large depth of field three-dimensional measurement method using phase-guided ray tracing according to claim 1, characterized in that: The method of fitting the sub-pixel light corresponding to the marker point by the second cost function based on the placement position of the planar target and the parameters in the first cost function and obtaining the target's pose parameters by optimizing the third cost function includes: Extracting the center coordinates of the circle corresponding to the placement position of each of the planar targets under the central viewing angle of the light field camera; Acquire physical coordinates matching the circle center coordinates according to the first orthogonal phase distribution; transforming the physical coordinates according to the first cost function and the global coordinate system to obtain corresponding target transformation coordinates; Sub-pixel light fitting is performed on the target conversion coordinates using a second cost function, and the pose parameters of the target corresponding to the fitted sub-pixel light are obtained using a third cost function.

4. The large depth of field three-dimensional measurement method using phase-guided ray tracing according to claim 3, characterized in that: The sub-pixel light fitting is performed on the target conversion coordinates by using the second cost function, and the pose parameters of the target corresponding to the sub-pixel light obtained by fitting are obtained by using the third cost function, including: Performing sub-pixel light fitting on the target transformation coordinates using the second cost function to obtain sub-pixel light corresponding to all marker points; Convert the marker points according to the global coordinate system to obtain corresponding marker conversion points; The overlap calculation of the landmark point and the landmark conversion point is performed according to the parameters in the first cost function, and the corresponding posture parameters are obtained by fitting the third cost function.

5. The large depth of field three-dimensional measurement method using phase-guided ray tracing according to claim 1, characterized in that: The phase-projection light mapping table is constructed according to the preset phase shift parameter, the first orthogonal phase distribution, the second orthogonal phase distribution and the second cost function, including: Calculating a quadrature phase distribution corresponding to the first quadrature phase distribution according to the phase shift parameter; According to the phase shift parameter and the second orthogonal phase distribution, obtaining a three-dimensional point on a planar target that matches the phase shift parameter with the central viewing angle of the light field camera; Fitting the three-dimensional points according to the second cost function to obtain corresponding projection rays; According to the matching relationship between the orthogonal phase distribution and the projection light, a corresponding phase-projection light mapping table is constructed.

6. The large depth of field three-dimensional measurement method using phase-guided ray tracing according to claim 1, characterized in that: The performing phase matching on the third orthogonal phase distribution according to the phase-projection light mapping table to obtain corresponding projection light and multi-view camera light corresponding to the projection light includes: Mapping the third orthogonal phase distribution according to the phase-projection light mapping table to obtain matching projection light; Obtaining the same-name points corresponding to the projection light at each viewing angle of the light field camera; The camera rays corresponding to the points with the same name are determined as the multi-view camera rays.

7. The large depth of field three-dimensional measurement method using phase-guided ray tracing according to claim 1, characterized in that: Before fusing the projection light and the multi-view camera light through a fourth cost function to obtain the three-dimensional coordinates corresponding to the object to be measured, the method further includes: Obtaining fitting parameters obtained by fitting the posture parameters of the display and the posture parameters of the target using a second cost function; Parameters of the fourth cost function are set according to the fitting parameters.

8. A phase-guided ray tracing three-dimensional measurement device with a large depth of field, characterized in that: The phase-guided ray tracing three-dimensional measurement device with a large depth of field is used to perform the phase-guided ray tracing three-dimensional measurement method with a large depth of field according to any one of claims 1 to 7, and the device comprises: a first calculation unit, configured to obtain display images acquired by sequentially capturing orthogonal fringe pattern displays of a display at multiple placement positions in a calibration space by a light field camera, and calculate a first orthogonal phase distribution corresponding to the display at each placement position; A first construction unit is configured to construct a global coordinate system corresponding to a first placement position according to basic parameters of the light field camera and obtain pose parameters of the display through optimization using a first cost function; a second calculation unit, configured to obtain fringe images obtained by sequentially capturing orthogonal fringes projected by the light field camera onto a planar target placed at a plurality of target positions within the calibration space, and calculate a second orthogonal phase distribution corresponding to each target position; the orthogonal fringes are obtained by projecting a two-dimensional scanning structured light system onto the planar target; A pose parameter acquisition unit is configured to, based on the placement position of the planar target and the parameters in the first cost function, fit the sub-pixel light corresponding to the marker point using a second cost function and optimize the pose parameters of the target using a third cost function; A second construction unit is configured to construct a phase-projection light mapping table according to a preset phase shift parameter, the first orthogonal phase distribution, the second orthogonal phase distribution, and the second cost function; a third computing unit, configured to acquire a target image obtained by capturing the orthogonal fringes projected by the light field camera onto the surface of the object to be measured, and calculate a third orthogonal phase distribution corresponding to the object to be measured; a phase matching unit, configured to perform phase matching on the third orthogonal phase distribution according to the phase-projection light mapping table to obtain corresponding projection light and multi-view camera light corresponding to the projection light; A fourth calculation unit is used to fuse the projection light and the multi-view camera light through a fourth cost function to obtain the three-dimensional coordinates corresponding to the object to be measured; the fourth cost function is obtained based on the corresponding configuration of the second cost function, the posture parameters of the display and the posture parameters of the target.

9. A computer device, characterized in that: The device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; The processor is configured to implement the steps of the large depth of field three-dimensional measurement method using phase-guided ray tracing according to any one of claims 1 to 7 when executing a program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the large depth of field three-dimensional measurement method according to any one of claims 1 to 7 are implemented.

Images