Multi-view projection-imaging three-dimensional reconstruction method

Abstract

The three-dimensional reconstruction method of multi-view projection-imaging solves the problem of poor consistency of reconstruction results caused by calibration error and system noise, and belongs to the field of machine vision measurement. The method comprises the following steps: a projection unit projects a preset sequence of stripe coded images to a calibration board in sequence after receiving a synchronous trigger signal of the imaging unit; the position and rotating posture of the calibration board in the three-dimensional space are moved; the imaging unit and N p projection units are triggered in sequence under each posture of the calibration board until N b sets of posture sequences of the calibration board are acquired; the imaging unit and the projection unit are calibrated; the external posture parameters of the imaging unit and each projection unit are optimized based on geometric consistency constraints, the problem of poor consistency of reconstruction results is solved, the distortion-free three-dimensional coordinates of single-direction structured light projection are calculated, the depth measurement values of each projection unit p i are fused to obtain the depth information Z of the object to be detected at (X, Y), and the final three-dimensional point cloud data (X, Y, Z) is generated.

Description

Technical Field

[0001] This invention relates to a three-dimensional reconstruction method using multi-view projection-imaging, belonging to the field of machine vision measurement. Background Technology

[0002] Active 3D measurement technology is now widely used in industrial measurement and visual inspection fields, such as 3D measurement and defect detection of printed circuit boards, 3D printing, and reverse engineering. Commonly used methods include Time-of-Flight (TOF), structured light methods, and stereo vision. Among these, active 3D reconstruction technology based on structured light does not require contact with the object being measured. It overcomes the problem of missing features on smooth, textureless surfaces by actively projecting structured light sources (points, stripes, speckles, etc.) onto the object, offering advantages such as high measurement accuracy, strong anti-interference ability, and high real-time performance. It is widely used for precise 3D measurement of complex object surfaces. 3D reconstruction systems based on single-view projection-imaging units introduce shadow problems due to overlapping fields of view limitations. Furthermore, as the requirements for measurement accuracy in inspection tasks increase, the combination of multi-view projection units and imaging units has been widely applied and researched in recent years. Due to calibration errors and system measurement errors, existing methods yield measurement results with significant differences in multiple directions, increasing the computational difficulty of the fusion process and reducing system reliability. In addition, in order to account for the effect of lens distortion of the projection unit, the projection unit needs to project stripe images in two orthogonal directions at the same time, which reduces the reconstruction speed of the system.

[0003] Therefore, the consistency calibration process and reconstruction method of multi-view projection-imaging unit system are problems that urgently need to be solved. Summary of the Invention

[0004] To address the issue of inconsistent reconstruction results caused by calibration errors and system noise, this invention provides a three-dimensional reconstruction method based on multi-view projection-imaging.

[0005] This invention provides a three-dimensional reconstruction method based on multi-view projection-imaging. The method is implemented using a multi-view imaging system and a calibration board. The multi-view imaging system includes an imaging unit, an industrial control computer, an image acquisition card, and an N... p N projection units p ≥2, imaging unit is opposite to calibration plate, N p A projection unit is uniformly distributed on a circle centered on the imaging unit, with the projection direction facing the calibration plate. The calibration plate includes S feature marker circles. An industrial control computer controls the imaging unit to capture image sequences on the calibration plate via an image acquisition card. The method of the present invention includes:

[0006] S1. After receiving the synchronization trigger signal from the imaging unit, the projection unit projects a preset stripe-coded image sequence onto the calibration plate in sequence. It moves the calibration plate according to its position and rotation in three-dimensional space. Under each calibration plate orientation, the imaging unit and N... p Each projection unit is triggered sequentially until N is obtained. b Group calibration board attitude sequence;

[0007] S2. Establish the imaging unit model and N p A projection unit model, utilizing N b The calibration plate attitude sequence is used to evaluate the imaging unit model and N. p The intrinsic and extrinsic parameters of each projection unit model are calibrated.

[0008] S3. Establish a global geometric consistency constraint model for the relative pose of the calibration plate under the view of multiple projection units, and jointly optimize the external parameters of the imaging unit and each projection unit.

[0009] S4. Replace the calibration plate with the object to be detected, and calculate the projection unit p based on the optimized imaging unit and the external parameters of each projection unit. i Distortion-free pixel coordinates on the normalized imaging plane use Obtain the three-dimensional coordinates (X, Y, Z) of the single-view imaging unit-projection unit pair in the normalized imaging plane coordinate system. i Z i Represents the projection unit p i The depth measurement values ​​are fused from the projection units p. i The depth measurement value is used to obtain the depth information Z of the object to be detected at (X,Y), and the final three-dimensional point cloud data (X,Y,Z) is generated.

[0010] Preferably, S3 includes:

[0011] S31. Calculate based on the parameters calibrated in S2. and The initial value;

[0012] In N b Group calibration plate b j Choose any set as the reference calibration plate b ref ; Indicates the calibration plate b of different projection units j The relative poses between them This indicates that the imaging unit is relative to the reference calibration plate b. ref The relative pose;

[0013] S32, according to and Given initial values, optimize the global geometric consistency constraint model and solve for the optimal solution. and

[0014] The global geometric consistency constraint model is as follows:

[0015]

[0016] Represents the mapping relationship between three-dimensional spatial coordinates and two-dimensional image coordinates; ||·|| 2 Represents the L2 loss function; This indicates the pixel coordinates of the feature marker circle captured by the imaging unit; express In projection unit p i Equiphase pixel coordinates on; Let K represent the three-dimensional physical coordinates of the s-th feature circle, s∈S; c Indicates the internal parameters of the imaging unit. Indicates the internal parameters of the projection unit. Indicates the distortion parameters of the projection unit;

[0017] S33, then based on the optimization calculate Represents the projection unit p i Relative to reference calibration plate b ref The homogeneous transformation matrix;

[0018] S34, Calculate the world coordinate system O w Relative imaging unit physical coordinate system O c rotation matrix Translation matrix

[0019]

[0020]

[0021] in, and For optimization The elements in and For optimization The elements in

[0022] Preferably, S31 includes:

[0023] Calculate in different projection units p i N from the perspective b The pose of the calibration plate relative to the reference calibration plate is:

[0024]

[0025] Represents the projection unit p i Relative to reference calibration plate b ref The homogeneous transformation matrix, Represents the projection unit p i Compared to calibration board b j The homogeneous transformation matrix in N b Group calibration plate b j Choose any set as the reference calibration plate b ref , Determined based on the external parameters of each calibrated projection unit model;

[0026] Will As a calculation Initial value;

[0027] The initial value is determined based on the external parameters of the calibrated imaging unit model.

[0028] As a preferred embodiment, in S32, the one determined in S31 is... and Using these as initial values, the LM algorithm is used to optimize the global geometric consistency constraint model, and the optimal solution is obtained. and

[0029] As a preferred option, in S33, according to the optimization... Minimize reprojection error using the PnP algorithm

[0030] Preferably, S4 includes:

[0031] S41, according to and Calculate the rotation matrix from the projection unit to the imaging unit. Translation matrix from projection unit to imaging unit

[0032]

[0033]

[0034] S42. Establish an orthogonal-perspective hybrid nonlinear epipolar constraint model:

[0035]

[0036]

[0037] In the formula, r1 and r2 represent the values ​​obtained by S41. The first and second row elements, t1 and t2, represent the translation vectors obtained by S41. The elements in the first and second rows, Represents the distortion coordinates on the normalized imaging plane in the projection unit coordinate system, (x c ,y c () represents the normalized pixel coordinates of the imaging unit. Represents the normalized pixel coordinates of the projection unit. express corresponding It must lie on the polynomial curve superior;

[0038] Replace the calibration plate with the object to be tested to obtain (x) c ,y c ) and normalized plane distortion coordinates of single-direction projection elements Based on polynomial curves The mapping relationship is used to obtain the normalized plane distortion coordinates of the single-direction projection unit.

[0039] use and The distortion relationship is used to calculate the distortion-free pixel coordinates of the projection unit on the normalized imaging plane.

[0040] This represents the coordinates of the projection unit in a distortion-free ideal image on the normalized plane.

[0041] S43. Establish the three-dimensional coordinates (X, Y, Z) of the single-view imaging-projection unit pair in the normalized imaging plane coordinate system. i Z i Represents the projection unit p i The depth measurement value;

[0042] S44, merge all projection units p i The depth measurement value is used to obtain the depth information Z of the object to be detected at (X,Y), and the final three-dimensional point cloud data (X,Y,Z) is generated.

[0043] Preferably, in S43, the three-dimensional coordinates (X,Y,Z) i They are respectively:

[0044]

[0045]

[0046]

[0047] In the formula, and Representing projection unit p respectively i attitude matrix relative to the world coordinate system and The corresponding element in and These represent the lens in the physical coordinate system O of the imaging unit. c Chinese x c and y c Magnification in direction, (u c ,v c () represents the two-dimensional projected pixel coordinates of a point in three-dimensional space within the pixel coordinate system of the imaging unit. It is the distortion center of the image in the pixel coordinate system of the imaging unit.

[0048] Preferably, in S44, the depth information Z of the object to be detected at (X,Y) is:

[0049]

[0050] In the formula, Z f (X,Y) represents the multi-view fusion depth value at (X,Y). W represents the set of effective imaging-projection unit indices at (X,Y). k (X,Y) represents the reliability weight of the imaging-projection unit for depth information. The reliability weight includes at least one of the modulation and intensity of the surface of the object to be detected.

[0051] The beneficial effects of this invention are as follows: Based on an external attitude optimization method that establishes multi-view geometric consistency constraints through multiple sets of calibration board attitude sequence images, this invention can effectively reduce the consistency differences in reconstruction results caused by calibration board errors and system noise. This invention proposes a distortion-free 3D reconstruction model using unidirectional structured light projection, establishing an orthogonal-perspective hybrid nonlinear epipolar constraint model. Compared to traditional linear epipolar constraint models and bidirectional fringe projection reconstruction models, this model simplifies the number of projection-encoded fringes while ensuring the accuracy of the reconstruction measurement results. Attached Figure Description

[0052] Figure 1 This is a flowchart of the three-dimensional reconstruction method of the present invention;

[0053] Figure 2 This is a schematic diagram of the relative pose to be optimized during the calibration process of this invention;

[0054] Figure 3 This is a schematic diagram of the imaging model of the two projection units and the imaging unit of the present invention;

[0055] Figure 4 This is an example of reconstruction for three types of objects to be detected according to the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0058] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0059] The multi-view projection-imaging three-dimensional reconstruction method of this embodiment is based on a multi-view imaging system and a calibration plate. The multi-view imaging system of this embodiment is as follows: Figure 2 As shown, it includes an imaging unit, an industrial control computer, an image acquisition card, and N. p Each projection unit has an imaging unit that can be configured with dual telecentric lenses to capture images of the calibration plate and the workpiece to be inspected, and to generate synchronous trigger signals to each projection unit. The projection unit in this embodiment uses a tilting imaging lens. p ≥2, N in this embodiment p =4, the imaging unit is opposite to the calibration plate, and the four projection units are evenly distributed on the circumference of the imaging unit with the projection direction facing the calibration plate. The calibration plate includes S feature marker circles. The industrial control computer controls the imaging unit to capture the image sequence on the calibration plate through the image acquisition card; the three-dimensional reconstruction method of this embodiment includes:

[0060] Step 1: Simultaneously with the start of exposure, the imaging unit sends a synchronization trigger signal to the projection unit. Upon receiving the synchronization trigger signal, the projection unit sequentially projects a preset stripe-coded image sequence onto the calibration plate. The trigger signal controls the triggering of the projection unit via a trigger selector, meaning only one projection unit operates at a time. The position and rotation of the calibration plate in three-dimensional space are adjusted via the stage. Under each calibration plate orientation, the imaging unit and N... p Each projection unit is triggered sequentially until N is obtained. b Group calibration board attitude sequence;

[0061] Step 2: Establish the imaging unit model and N p A projection unit model, utilizing N b The calibration plate attitude sequence is used to evaluate the imaging unit model and N. p The intrinsic and extrinsic parameters of each projection unit model are calibrated.

[0062] In this embodiment, the imaging unit model and N p The calibration of the intrinsic and extrinsic parameters of each projection element model can be performed using traditional methods or the method described in this embodiment. Step 2 of this embodiment includes:

[0063] Step 21: The intrinsic and extrinsic parameters of the imaging unit model include... An imaging unit model is established based on the relationship between three-dimensional spatial coordinates and imaging unit pixel coordinates:

[0064]

[0065] In the formula, [X,Y,Z] represents a spatial point relative to the world coordinate system O. w Three-dimensional physical coordinates and These represent the lens in the physical coordinate system O of the imaging unit. c Chinese x c and y c Magnification in direction, (u c ,v c () represents the two-dimensional projected pixel coordinates of a point in three-dimensional space within the pixel coordinate system of the imaging unit. The distortion center of the image in the pixel coordinate system of the imaging unit. and Representing the world coordinate system O w Relative imaging unit physical coordinate system O c rotation matrix and translation matrix, K c Indicates the internal parameters of the imaging unit;

[0066] Step 22: The intrinsic and extrinsic parameters of the projection element model include: and Based on three-dimensional spatial coordinates and projection unit p i The relationship between pixel coordinates is used to establish the projection unit model:

[0067]

[0068]

[0069] In the formula, Indicates the scale factor. and For the effective focal length, The origin of the image plane in the pixel coordinate system of the projection unit. and Representing the world coordinate system O w Relative to projection unit p i Physical coordinate system The rotation and translation matrices, Represents the projection unit pi Internal parameters;

[0070] Step 23, based on N b Given the three-dimensional physical coordinates and two-dimensional projected pixel coordinates in the calibration board attitude sequence, K is solved using a two-step calibration method and an imaging unit model. c and N b The external pose of the calibration plate relative to the physical coordinate system O of the imaging unit c rotation and translation parameter sequences and

[0071] Based on N b The known 3D physical coordinates and 2D projected pixel coordinates in the attitude sequence of the calibration board are solved using a two-step calibration method and a projection element model, respectively. and N b The external pose of the calibration board relative to the projection unit p i Physical coordinate system rotation and translation parameter sequences and

[0072] According to the parameter sequence and Sure;

[0073] Based on N b Given the known 3D physical coordinates and 2D projected pixel coordinates in the calibration board's attitude sequence, the distortion parameter vector is solved based on the distortion relationship.

[0074] The distortion relationship in this embodiment is as follows:

[0075]

[0076] Represents the coordinates of an ideal, distortion-free image on the normalized plane. This represents the distortion coordinates on the normalized imaging plane of the projection unit. Represents radial distance, {k n} represents the radial distortion factor, and ρ1 and ρ2 both represent the tangential distortion factors; considering the tilt of the imaging plane caused by the tilted lens, τ x ,τ y These represent the tilt angles of the tilted imaging plane along the x and y directions, respectively.

[0077] Distortion parameter vector k pi =[k1,k2,k3,ρ1,ρ2,τ] x ,τ y ].

[0078] Step 3: Establish a global geometric consistency constraint model for the relative pose of the calibration board based on the viewpoints of multiple projection units, and jointly optimize the external parameters of the imaging unit and each projection unit, including:

[0079] Step 31: Calculate based on the parameters calibrated in Step 2. and The initial value; for ease of description, the geometric transformation process of the physical coordinates of points in three-dimensional space is described. Equivalent representation as a homogeneous matrix Generally refers to the rigid geometric transformation relationship between the β-representing coordinate system and the α-representing coordinate system, calculated in different projection elements p i N from the perspective b The pose of the calibration plate relative to the reference calibration plate is:

[0080]

[0081] In N b Group calibration plate b j Choose any set as the reference calibration plate b ref ; Indicates the calibration plate b under different projection units j The relative poses between them This indicates that the imaging unit is relative to the reference calibration plate b. ref The relative pose; Represents the projection unit p i Relative to reference calibration plate b ref The homogeneous transformation matrix, Represents the projection unit p i Compared to calibration board b j The homogeneous transformation matrix in N b Group calibration plate b j Choose any set as the reference calibration plate b ref , Determined based on the external parameters of each calibrated projection unit model;

[0082] The average of the relative poses between the calibration plates of different projection units. As a calculation Initial value;

[0083] The initial value is determined based on the external parameters of the calibrated imaging unit model.

[0084] Step 32, according to and Given initial values, optimize the global geometric consistency constraint model, and use the LM algorithm to solve for the optimal solution. and

[0085] This implementation method optimizes the relative poses between calibration plates and the external poses of imaging units based on globally consistent multi-view geometric constraints. The globally consistent geometric constraint model is as follows:

[0086]

[0087] Represents the mapping relationship between three-dimensional spatial coordinates and two-dimensional image coordinates; ||·| 2 Represents the L2 loss function; This indicates the coordinates of the feature marker circle pixels captured by the imaging unit; express In projection unit p i Equiphase pixel coordinates on Based on capturing N b The attitude sequence of the calibration board is calculated. Let K represent the three-dimensional physical coordinates of the s-th feature circle, s∈S; c Indicates the internal parameters of the imaging unit. Indicates the internal parameters of the projection unit. Indicates the distortion parameters of the projection unit;

[0088] Step 33, then based on the optimization calculate This process utilizes the PnP algorithm to minimize the reprojection error calculation;

[0089] Represents the projection unit p i Relative to reference calibration plate b ref The homogeneous transformation matrix;

[0090] Step 34: Combine the optimized external pose parameters of the imaging unit as described above. and external pose parameters of each projection unit All external pose parameters are relative to the same reference calibration plate, which facilitates subsequent 3D coordinate calculations and establishes the world coordinate system O. w Defined as the physical coordinate system O of the imaging unit c In z c The extension of direction then leads to the world coordinate system O w Relative imaging unit physical coordinate system O c rotation matrix Translation matrix

[0091]

[0092]

[0093] in, and For optimization The elements in and For optimization The elements in

[0094] Step 4: Replace the calibration plate with the object to be detected, and calculate the projection unit p based on the optimized imaging unit and the external parameters of each projection unit. i Distortion-free pixel coordinates on the normalized imaging plane use Obtain the three-dimensional coordinates (X, Y, Z) of the single-view imaging unit-projection unit pair in the normalized imaging plane coordinate system. i Z i Represents the projection unit p i The depth measurement values ​​are fused from the projection units p. i The depth measurement value is used to obtain the depth information Z of the object to be detected at (X,Y), and the final 3D point cloud data (X,Y,Z) is generated, including:

[0095] Step 41, according to and Calculate the rotation matrix from the projection unit to the imaging unit. Translation matrix from projection unit to imaging unit

[0096]

[0097]

[0098] Step 42: Establish an orthogonal-perspective hybrid nonlinear epipolar constraint model, which describes the geometric constraints that the pixel coordinates on the image plane of the projection unit and the imaging unit should satisfy:

[0099]

[0100]

[0101] In the formula, r1 and r2 represent the values ​​obtained by S41. The first and second row elements, t1 and t2, represent the translation vectors obtained by S41. The elements in the first and second rows, Represents the distortion coordinates on the normalized imaging plane in the projection unit coordinate system, (x c ,y c () represents the normalized pixel coordinates of the imaging unit. Represents the normalized pixel coordinates of the projection unit. It means (x) c,y c ) corresponding It must lie on the polynomial curve superior;

[0102] Replace the calibration plate with the object to be tested to obtain (x) c ,y c ) and normalized plane distortion coordinates of single-direction projection elements Based on polynomial curves The mapping relationship is used to obtain the normalized plane distortion coordinates of the single-direction projection unit.

[0103] use and The distortion relationship is used to calculate the distortion-free pixel coordinates of the projection unit on the normalized imaging plane.

[0104] This represents the coordinates of the projection unit in a distortion-free ideal image on the normalized plane.

[0105] Step 43: Establish a triangulation reconstruction model of a single-view imaging-projection unit pair in the normalized imaging plane coordinate system to obtain the three-dimensional coordinates (X,Y,Z). i Z i Represents the projection unit p i The depth measurement value, the triangulation reconstruction model is:

[0106]

[0107]

[0108]

[0109] In the formula, and Representing projection unit p respectively i attitude matrix relative to the world coordinate system and The corresponding element in and These represent the lens in the physical coordinate system O of the imaging unit. c Chinese x c and y c Magnification in direction, (u c ,v c () represents the two-dimensional projected pixel coordinates of a point in three-dimensional space within the pixel coordinate system of the imaging unit. It is the distortion center of the image in the pixel coordinate system of the imaging unit.

[0110] Step 44: Merge all projection units p iThe depth measurement value is used to obtain the depth information Z of the object to be detected at (X,Y), and the final 3D point cloud data (X,Y,Z) is generated. The depth information Z of the object to be detected at (X,Y) is as follows:

[0111]

[0112] In the formula, Z f (X,Y) represents the multi-view fusion depth value at (X,Y). W represents the set of effective imaging-projection unit indices at (X,Y). k (X,Y) represents the reliability weight of the imaging-projection unit for depth information. The reliability weight includes at least one of the modulation and intensity of the surface of the object to be detected.

[0113] This implementation first calibrates the geometric parameters of the imaging unit and each projection unit, then optimizes the external pose parameters of the imaging unit and each projection unit based on geometric consistency constraints to solve the problem of poor consistency of reconstruction results in the multi-directional projection structured light reconstruction system. Finally, it calculates the distortion-free 3D reconstruction model of unidirectional structured light projection, thereby improving the reconstruction speed while ensuring the accuracy of 3D reconstruction.

[0114] Figure 4 Images (a)-(d) show the depth maps calculated by each imaging-projection unit, image (e) shows the fused depth map information, and image (f) shows the rendered mesh image. Clearly, the method proposed in this invention can effectively eliminate shadow and noise areas, improving the stability and accuracy of the 3D reconstruction results.

[0115] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other embodiments.

Claims

1. A method of multi-view projection-imaging three-dimensional reconstruction, characterized in that, The method is realized based on a multi-view imaging system and a calibration board, the system comprises an imaging unit, an industrial computer, an image acquisition card and N p projection units, N p ≥2, the imaging unit is opposite to the calibration board, the N p projection units are uniformly distributed on a circumference with the imaging unit as the center and the projection directions are towards the calibration board, the calibration board comprises S feature mark circles, the industrial computer controls the imaging unit to capture a sequence of images on the calibration board through the image acquisition card; the method comprises: S1, after receiving the imaging unit synchronization trigger signal, the projection unit projects the preset sequence of fringe encoding images to the calibration plate in sequence, and the moving calibration plate changes the position and rotation attitude in the three-dimensional space, and the imaging unit and N p projection units are triggered in sequence until the N b group of calibration plate attitude sequences are acquired; S2, establish an imaging unit model and N p projection unit models, calibrate the internal and external parameters of the imaging unit model and the N b projection unit models using N p sets of calibration board pose sequences; S3, a global geometric consistency constraint model based on the relative poses of the calibration board under the perspective of the multiple projection units is established, and the external parameters of the imaging unit and each projection unit are jointly optimized; S4. Replace the calibration plate with the object to be detected, and calculate the projection unit p based on the optimized imaging unit and the external parameters of each projection unit. i Distortion-free pixel coordinates on the normalized imaging plane use Obtain the three-dimensional coordinates (X, Y, Z) of the single-view imaging unit-projection unit pair in the normalized imaging plane coordinate system. i Z i Represents the projection unit p i The depth measurement values ​​are fused from the projection units p. i The depth measurement value is used to obtain the depth information Z of the object to be detected at (N,Y), and the final three-dimensional point cloud data (X,Y,Z) is generated.

2. The multi-view projection-imaging three-dimensional reconstruction method of claim 1, wherein, The S3 includes: S31, calculating the parameters calibrated according to S2 and initial values; In N b group of calibration boards b j Any one of the groups is selected as a reference calibration board b ref ; represent the relative poses between the calibration boards b j under different projection units, represent the relative poses of the imaging units with respect to the reference calibration board b ref ; S32、According to and initial values, optimize the global geometric consistency constraint model to solve the optimal and The global geometric consistency constraint model is: denotes the mapping relationship between three-dimensional space coordinates and two-dimensional image coordinates;‖·‖ 2 denotes the L2 loss function; denotes the feature mark circle pixel coordinates captured by the imaging unit; denotes denotes the equal phase pixel coordinates on the projection unit p i ; denotes the three-dimensional physical coordinates of the s-th feature mark circle, s∈S;K c denotes the imaging unit internal parameter, denotes the projection unit internal parameter, denotes the distortion parameter of the projection unit; S33、again according to the optimized computing representing the homographic matrix of the projection unit p i with respect to the reference calibration board b ref the homographic matrix of the reference calibration board b S34, compute world coordinate system O w relative to the imaging unit physical coordinate system O c rotation matrix of O and translation matrix wherein and are optimized elements in, and are optimized elements in, 3. The multi-view projection-imaging three-dimensional reconstruction method of claim 2, wherein, The S31 includes: The calculation is performed for different projection units p i N under different view angles b The pose of the group calibration board relative to the reference calibration board is: a homogeneous transformation matrix of the projection unit p i with respect to the reference calibration board b ref , a homogeneous transformation matrix of the projection unit p i with respect to the calibration board b j , in the N b groups of calibration boards b j any one group is selected as the reference calibration board b ref , determined according to the calibrated external parameters of each projection unit model; will be described below. The present application will be described below. As the initial value of the algorithm ; The initial values are determined in accordance with the external parameters of the calibrated imaging unit model.

4. The multi-view projective-imaging three-dimensional reconstruction method of claim 3, wherein, In the S32, the global geometric consistency constraint model is optimized using the L-M algorithm to solve the optimal and As the initial value, the global geometric consistency constraint model is optimized using the L-M algorithm to solve the optimal and 5. The multi-view projection-imaging three-dimensional reconstruction method of claim 4, wherein, In the S33, the Minimizing the re-projection error calculation using PnP algorithm 6. The multi-view projective-imaging three-dimensional reconstruction method of claim 2, wherein S4 including: S41、According to and calculating a rotation matrix of the projection unit to the imaging unit and a translation matrix of the projection unit to the imaging unit S42, an orthogonal-perspective hybrid nonlinear epipolar constraint model is established: where r1, r2 denote the first two elements of the vector obtained in S41 where t1, t2 denote the translation vector obtained in S41 of the first two elements of the vector denote the distorted coordinates on the normalized imaging plane in the projection unit coordinate system, (x c ,y c ) denote the normalized pixel coordinates of the imaging unit, denote the normalized pixel coordinates of the projection unit, denote the coordinates of the point (x c ,y c ) on the polynomial curve which must lie on the polynomial curve ​ Replace the calibration board with the object to be detected to obtain (x c ,y c ) and the single-direction projection unit normalized plane distortion coordinates Based on the mapping relationship of the polynomial curve , obtain the single-direction projection unit normalized plane distortion coordinates Utilizing and the distortion relationship, the non-distorted pixel coordinates of the projection unit on the normalized imaging plane are calculated represents the undistorted ideal image coordinates of the projection unit on the normalized plane; S43, establish the three-dimensional coordinates (X, Y, Z) of the single-view imaging-projection unit pair under the normalized imaging plane coordinate system i ), Z i represents the depth measurement value of the projection unit p i ​ S44, fusing the depth measurement values of the respective projection units p i to obtain depth information Z of the object to be detected at (X, Y), and generate final three-dimensional point cloud data (X, Y, Z).

7. The multi-view projection-imaging three-dimensional reconstruction method of claim 6, wherein, In S43, the three-dimensional coordinates (X, Y, Z i ) are respectively: wherein and respectively represent the corresponding elements in the pose matrix of the projection unit p i with respect to the world coordinate system and respectively represent the corresponding elements in the pose matrix of the imaging unit O and respectively represent the magnification of the lens in the x c and y c directions in the physical coordinate system of the imaging unit O c , (u c , v f ) represent the two-dimensional projection pixel coordinates of the three-dimensional space point in the pixel coordinate system of the imaging unit, and is the distortion center of the image in the pixel coordinate system of the imaging unit.

8. The multi-view projection-imaging three-dimensional reconstruction method of claim 6, wherein, In S44, the depth information Z of the object to be detected at (X, Y) is: wherein Z f (X,Y) represents a multi-view fused depth value at (X,Y), (X,Y) represents a set of valid imaging-projection cell indices at (X,Y), W k (X,Y) represents a reliability weight of the imaging-projection cell pair on the depth information, the reliability weight comprising at least one of a modulation, an intensity of a surface of an object to be detected.

9. The multi-view projection-imaging three-dimensional reconstruction method of claim 1, wherein S2 including: S21, the internal and external parameters of the imaging unit model are determined by K c , and the imaging unit model is established where [X, Y, Z] represents the three-dimensional physical coordinates of a certain spatial point relative to the world coordinate system O w , and are the magnification factors of the lens in the x c and y c directions of the imaging unit physical coordinate system O c , (u c , v c ) represents the two-dimensional projection pixel coordinates of the three-dimensional spatial point in the imaging unit pixel coordinate system, is the distortion center of the image in the imaging unit pixel coordinate system, and respectively represent the rotation matrix and the translation matrix of the world coordinate system O w relative to the imaging unit physical coordinate system O c , and K c represents the internal parameters of the imaging unit; S22, the internal and external parameters of the projection unit model are: and establishing the projection unit model: wherein denotes the scale factor, and is the effective focal length, is the origin of the image plane in the projection unit pixel coordinate system, and denote the rotation and translation matrix of the world coordinate system O w with respect to the projection unit p i physical coordinate system , denotes the intrinsic parameters of the projection unit p i . S23, based on N b The three-dimensional physical coordinates and two-dimensional projection pixel coordinates of the known sequence of the group calibration board posture are respectively solved by using a two-step calibration method and an imaging unit model c and N b The rotation and translation parameter sequence of the external posture of the group calibration board relative to the physical coordinate system O c of the imaging unit and Based on N b The three-dimensional physical coordinates and two-dimensional projection pixel coordinates of the known sequence of group calibration board poses are solved using a two-step calibration method and a projection unit model And N b The external pose of the group calibration board relative to the projection unit p i The rotation and translation parameter sequence of the physical coordinate system O pi of the group calibration board And According to the parameter sequence And Determination; N-based b The three-dimensional physical coordinates and two-dimensional projection pixel coordinates of the known group calibration board posture sequence are solved according to the distortion relationship 10. The multi-view projection-imaging three-dimensional reconstruction method of claim 6 or 9, wherein, The distortion relationship is: denotes the normalized plane without distortion ideal image coordinates, denotes the distorted coordinates on the normalized imaging plane of the projection unit, denotes the radial distance, {k n} denotes the radial distortion factor, and both ρ1 and ρ2 denote the tangential distortion factors; τ x , τ y denote the tilt angles of the tilted imaging plane along the x, y directions, respectively; distortion parameter vector k pi = [k1, k2, k3, p1, p2, t x , t y ].

Images