Three-dimensional reconstruction method and system for dynamic stereoscopic vision of linear structured light

Abstract

The invention belongs to the technical field of three-dimensional reconstruction, and provides a three-dimensional reconstruction method and system for linear structured light dynamic stereoscopic vision. The method comprises the following steps: carrying out stereoscopic vision device push-scan along a preset direction to obtain a plurality of stripe images corresponding to a target; combining the perspective projection imaging model of the left camera with the light plane equation of the line-structured light to obtain a stripe center three-dimensional coordinate of the stripe center of the line-structured light; calculating a relative pose of adjacent positions, and calculating an absolute pose of the camera at each position based on a global optimization algorithm; and finally, unifying the three-dimensional coordinates of the centers of the stripes to a reference coordinate system based on the absolute poses so as to realize three-dimensional reconstruction. The method overcomes the limitation of the static stereoscopic vision measurement range, and is suitable for three-dimensional reconstruction of the surface of a large device.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional reconstruction, and in particular to a three-dimensional reconstruction method and system of line structured light dynamic stereoscopic vision. Background Art

[0002] Three-dimensional reconstruction technology refers to the technology of restoring the three-dimensional shape of an object from a two-dimensional image. It has important applications in urban planning, historical site protection, virtual reality, augmented reality, and medical imaging.

[0003] In the prior art, there are three main methods for achieving three-dimensional reconstruction of target surfaces, namely: a three-dimensional reconstruction method based on speckle patterns, a three-dimensional reconstruction method based on deep learning, and a three-dimensional reconstruction method based on structured light. Among them, the three-dimensional reconstruction method based on speckle patterns relies on high-density three-dimensional point clouds, and therefore has the defect of low efficiency in the reconstruction process. Although the three-dimensional reconstruction method based on deep learning reduces the calculation time of three-dimensional point clouds, the reconstruction quality is highly dependent on a comprehensive training data set; but in practical applications, a comprehensive training data set is often not effectively obtained.

[0004] The 3D reconstruction method based on structured light is a 3D reconstruction method that has developed rapidly with the improvement of image processing technology and the accuracy and efficiency of various photoelectric sensors. It not only avoids the above-mentioned defects of the first two methods in the reconstruction process, but also has the advantages of high reconstruction efficiency and high reliability of results. However, the existing 3D reconstruction method based on structured light is a static visual measurement method, so it has the defect of limited visual measurement range, which makes it only suitable for surface 3D reconstruction of small-sized targets, but cannot be well applied to large-sized targets. Summary of the invention

[0005] The present invention aims to provide a three-dimensional reconstruction method and system of line structured light dynamic stereoscopic vision to improve the technical problem that the existing three-dimensional reconstruction method based on line structured light has a limited visual measurement range and is not applicable to the three-dimensional reconstruction of large-size targets.

[0006] To achieve the above object, the present invention proposes the following technical solutions:

[0007] In a first aspect, the present technical solution provides a three-dimensional reconstruction method of line structured light dynamic stereoscopic vision, comprising:

[0008] Controlling the stereoscopic vision device to push-scan along a preset direction to acquire a plurality of frames of stripe images covering all positions on the target surface; wherein the stereoscopic vision device comprises a right camera, a laser generator, and a left camera which are sequentially arranged with respect to the target surface; and the stripe images corresponding to each position on the target surface include a right stripe image acquired by the right camera and a left stripe image acquired by the left camera;

[0009] The right fringe image at each position is processed based on the intrinsic parameter matrix of the right camera and the extrinsic parameter matrix of the right camera relative to the left camera, and the left fringe image at each position is processed based on the intrinsic parameter matrix of the left camera; then the processed right fringe image and left fringe image are input into the Steger algorithm to extract the sub-pixel coordinates of the fringe image at each position;

[0010] The sub-pixel coordinates are introduced into the left camera perspective projection model, and the pre-constructed light plane equation is established to obtain the three-dimensional coordinates of the fringe center of the fringe image at each position;

[0011] Extract several groups of sub-pixel coordinates with the same name in the adjacent frame stripe images at each pair of adjacent positions, and introduce them into the left camera perspective projection model to obtain the rough relative pose at each pair of adjacent positions, and iteratively optimize each rough relative pose based on the object space reprojection error minimization function to obtain the corresponding optimal relative pose; at the same time, obtain the relative pose of each device of the stereoscopic vision device at each position compared with the stereoscopic vision device at the initial position; wherein the sub-pixel coordinates with the same name refer to the pixel coordinates corresponding to the same position in the adjacent frame stripe images;

[0012] Taking the left camera coordinate system at the initial position as the reference coordinate system, bringing the relative posture of each of the devices into the corresponding optimal relative posture to obtain the absolute posture of the fringe image at each position compared to the reference coordinate system;

[0013] Based on each absolute position, the three-dimensional coordinates of the center of each stripe are converted into the reference coordinate system to achieve three-dimensional reconstruction of the target surface.

[0014] Further, the method includes obtaining the light plane equation; including:

[0015] Define the three-dimensional coordinates of a chessboard angle in the corresponding chessboard coordinate system as (x w ,y w ,z w ), where z w = 0; the projection of the chessboard angle on the left camera image plane is denoted as g(u g ,v g ), and then obtain the left camera perspective projection imaging model as:

[0016]

[0017] Among them, K is the intrinsic parameter matrix of the left camera, f x 、f y are the focal lengths of the left camera in the x and y directions, c x 、c y are the principal points of the left camera in the x and y directions respectively, λ is the scale factor, r chessis the calibration rotation matrix of the chessboard coordinate system compared to the left camera coordinate system, t chess is the calibration translation matrix of the chessboard coordinate system compared to the left camera coordinate system;

[0018] For a given number of chessboard corners with different rotation angles and different translation positions, the following definition matrix is ​​constructed: H = K[R chess t chess ] and h=[h 11 ,h 12 ,h 13 ,h 14 ,h 21 ,h 22 ,h 23 ,h 24 ,h 31 ,h 32 ,h 33 ,h 34 ] T , where h ij (i=1,2,3; j=1,2,3,4) represents the elements of the i-th row and j-th column of the matrix H; and then the definition matrix is ​​substituted into the left camera perspective projection imaging model to obtain:

[0019]

[0020] At this time, for a given number of n chessboard corners, the matrix equation Ah = 0 can be obtained, where A is given by Vertical stacking, a i are the first two rows of the matrix equation; performing singular value decomposition on the matrix equation to obtain rough values ​​of the calibration rotation matrix and the calibration translation matrix;

[0021] Create a total error optimization objective function:

[0022]

[0023] Using the rough values ​​of the calibration rotation matrix and the calibration translation matrix as initial values, iterative calculation is performed to obtain the precise values ​​of the calibration rotation matrix and the calibration translation matrix; wherein x' is a mapping function of the chessboard corner image coordinates;

[0024] Select any two points on the projection line of the laser stripe on the chessboard plane in the camera coordinate system to construct the projection line equation: L = M a M b T -M b M a T ; and place the chessboard plane in different directions to obtain the first determined linear equation Then perform singular value decomposition to obtain the normal vector of the structured light plane; where M a is the coordinate of the first point on the projection line, M b is the coordinate of the second point on the projection line, is the vertical stack of the projection line equation L, n is the normal vector of the structured light plane;

[0025] The light plane equation is constructed based on the structured light plane normal vector.

[0026] Furthermore, the method extracts several groups of sub-pixel coordinates of the same name in the adjacent frame fringe images at each pair of adjacent positions, introduces them into the left camera perspective projection model to obtain the rough relative pose at each pair of adjacent positions, and iteratively optimizes each rough relative pose based on the reprojection error minimization function of the object space to obtain the corresponding optimal relative pose; including:

[0027] The deformation processing of the left camera perspective projection model is as follows:

[0028]

[0029] Among them, λ is the scale factor, K is the intrinsic parameter matrix of the camera, (u jL ,v jL ) is the image coordinate of the center of the fringe at the jth position, (x wi ,y wi ,z wi ) is the three-dimensional coordinate of the center of the fringe at the i-th position; is the relative rotation matrix of the j-th position compared to the i-th position, is the relative translation matrix of the j-th position compared to the i-th position;

[0030] Introduce several groups of sub-pixel coordinates with the same name in each pair of adjacent frame stripe images into the deformed left camera perspective projection model, and define Q = K -1 [u jL ,v jL ,1] T ,M=[R j i T j i ],m=[m 11 ,m 12 ,m 13 ,m 14 ,m 21 ,m 22 ,m 23 ,m 24 ,m 31 ,m 32 ,m 33 ,m 34 ]T , where m ab (a=1,2,3;b=1,2,3,4) represents the element in the ath row and bth column of the matrix M; after rearrangement, we get:

[0031]

[0032] Where Q(k) (k = 1, 2, 3) represents the kth element of the matrix Q, x wi ,y wi ,z wi The 3D coordinates of the center of the stripe at the i-th position; at this time, when n pairs of points of the same name between adjacent frames are given, the second overdetermined linear equation Bm=0 can be obtained, where B is given by Vertical stacking, b i are the first two rows of the sorted matrix; perform singular value decomposition on it to obtain the rough relative pose at each pair of adjacent positions;

[0033] The minimization function of the reprojection error of the k-th point of the same name at the i-th position and the j-th position in the object space is established as:

[0034]

[0035] The rough relative pose is used as the initial value for iterative calculation to obtain the corresponding optimal relative pose; are the three-dimensional coordinates of the kth feature point at the jth position and the i-th position respectively.

[0036] Furthermore, the left camera coordinate system at the initial position is used as the reference coordinate system, and the relative posture of each device is brought into the corresponding optimal relative posture to obtain the absolute posture of the fringe image at each position compared to the reference coordinate system; including:

[0037] Based on standard rotation relationship The first relative rotation relationship between the optimal relative postures is obtained as follows:

[0038]

[0039] in, R i is the absolute rotation matrix at the i-th position, R j is the absolute rotation matrix at the jth position; R i , R j The k-th column vector of ; is the relative rotation matrix of the j-th position compared to the i-th position;

[0040] Based on standard rotation relationship The second relative rotation relationship between the relative positions of each device is obtained as follows:

[0041]

[0042] in, is the rotation matrix of the binocular module at the i-th position relative to the initial position, is the rotation matrix of the binocular module at the jth position relative to the initial position; is a unit matrix, R0 is the rotation matrix of the binocular module at the initial position;

[0043] Substituting the second relative rotation relationship into the first relative rotation relationship, and solving it by using the least square method;

[0044] When it is determined that the solution of the least squares method is a unit orthogonal matrix, the solution is used as the absolute rotation matrix in the absolute pose; otherwise, a forced orthogonal equation is introduced as follows to re-estimate the absolute rotation matrix:

[0045]

[0046] in, is the forced orthogonal matrix corresponding to the absolute rotation matrix of the ith position, F is the Frobenius norm, and I is the identity matrix.

[0047] Furthermore, the left camera coordinate system at the initial position is used as the reference coordinate system, and the relative posture of each device is brought into the corresponding optimal relative posture to obtain the absolute posture of the fringe image at each position compared to the reference coordinate system; including:

[0048] Get the standard translation relationship As the first relative translation relationship between the optimal relative postures;

[0049] Among them, T i is the absolute translation matrix at the ith position, T j is the absolute translation matrix at the jth position, is the relative translation matrix of the j-th position compared to the i-th position;

[0050] Based on standard translation relationship The second relative translation relationship between the relative positions of each device is obtained as follows:

[0051]

[0052] in, is the rotation matrix of the binocular module at the i-th position relative to the initial position, is the rotation matrix of the binocular module at the jth position relative to the initial position; T0 is the translation matrix of the binocular module at the initial position;

[0053] Substituting the second relative translation relationship into the first relative translation relationship, we get:

[0054]

[0055] The least squares method is used to solve the problem to obtain the absolute translation matrix in the absolute pose.

[0056] In a second aspect, the technical solution provides a three-dimensional reconstruction system of line structured light dynamic stereoscopic vision, comprising:

[0057] An image acquisition module, used to control the stereoscopic vision device to push-scan along a preset direction to acquire a plurality of frames of stripe images covering all positions on the target surface; wherein the stereoscopic vision device comprises a right camera, a laser generator, and a left camera which are sequentially arranged with respect to the target surface; and the stripe images corresponding to each position on the target surface include a right stripe image acquired by the right camera and a left stripe image acquired by the left camera;

[0058] A pixel coordinate acquisition module is used to process the right fringe image at each position based on the intrinsic parameter matrix of the right camera and the extrinsic parameter matrix of the right camera relative to the left camera, and to process the left fringe image at each position based on the intrinsic parameter matrix of the left camera; and then input the processed right fringe image and left fringe image into the Steger algorithm to extract the sub-pixel coordinates of the fringe image at each position;

[0059] A fringe center extraction module, used to introduce the sub-pixel coordinates into the left camera perspective projection model and establish a pre-constructed light plane equation to obtain the fringe center three-dimensional coordinates of the fringe image at each position;

[0060] A relative posture acquisition module is used to extract several groups of sub-pixel coordinates of the same name in adjacent frame stripe images at each pair of adjacent positions, and introduce them into the left camera perspective projection model to obtain a rough relative posture at each pair of adjacent positions, and iteratively optimize each rough relative posture based on the object space reprojection error minimization function to obtain the corresponding optimal relative posture; and simultaneously obtain the relative posture of each device of the stereoscopic vision device at each position compared with the stereoscopic vision device at the initial position; wherein the sub-pixel coordinates of the same name refer to the pixel coordinates corresponding to the same position in the adjacent frame stripe images;

[0061] An absolute posture acquisition module, used to use the left camera coordinate system at the initial position as a reference coordinate system, and bring the relative posture of each device into a corresponding optimal relative posture to obtain the absolute posture of the fringe image at each position compared to the reference coordinate system;

[0062] The three-dimensional reconstruction module is used to transform the three-dimensional coordinates of the center of each stripe into the reference coordinate system based on each absolute posture to achieve three-dimensional reconstruction of the target surface.

[0063] Further, including:

[0064] The model acquisition module is used to define the three-dimensional coordinates of a chessboard angle in the corresponding chessboard coordinate system as (x w ,y w ,z w ), where z w = 0; the projection of the chessboard angle on the left camera image plane is denoted as g(u g ,v g ), and then obtain the left camera perspective projection imaging model as:

[0065]

[0066] Among them, K is the intrinsic parameter matrix of the left camera, f x 、f y are the focal lengths of the left camera in the x and y directions, c x 、c y are the principal points of the left camera in the x and y directions respectively, λ is the scale factor, r chess is the calibration rotation matrix of the chessboard coordinate system compared to the left camera coordinate system, t chess is the calibration translation matrix of the chessboard coordinate system compared to the left camera coordinate system;

[0067] The rough value acquisition module is used to construct the following definition matrix for a given number of chessboard corners with different rotation angles and different translation positions: H = K[R chess t chess ] and h=[h 11 ,h 12 ,h 13 ,h 14 ,h 21 ,h 22 ,h 23 ,h 24 ,h 31 ,h 32 ,h 33 ,h 34 ] T , where h ij (i=1,2,3; j=1,2,3,4) represents the elements of the i-th row and j-th column of the matrix H; and then the definition matrix is ​​substituted into the left camera perspective projection imaging model to obtain:

[0068]

[0069] At this time, for a given number of n chessboard corners, the matrix equation Ah = 0 can be obtained, where A is given by Vertical stacking, a iare the first two rows of the matrix equation; performing singular value decomposition on the matrix equation to obtain rough values ​​of the calibration rotation matrix and the calibration translation matrix;

[0070] The precise value acquisition module is used to create the total error optimization objective function:

[0071]

[0072] Using the rough values ​​of the calibration rotation matrix and the calibration translation matrix as initial values, iterative calculation is performed to obtain the precise values ​​of the calibration rotation matrix and the calibration translation matrix; wherein x' is a mapping function of the chessboard corner image coordinates;

[0073] The normal vector acquisition module is used to select any two points on the projection line of the laser stripe on the chessboard plane in the camera coordinate system to construct the projection line equation: L = M a M b T -M b M a T ; and place the chessboard plane in different directions to obtain the first determined linear equation Then perform singular value decomposition to obtain the normal vector of the structured light plane; where M a is the coordinate of the first point on the projection line, M b is the coordinate of the second point on the projection line, is the vertical stack of the projection line equation L, n is the normal vector of the structured light plane;

[0074] A light plane equation acquisition module is used to construct the light plane equation based on the structured light plane normal vector.

[0075] Furthermore, the relative posture acquisition module includes:

[0076] The deformation processing unit is used to deform the left camera perspective projection model:

[0077]

[0078] Among them, λ is the scale factor, K is the intrinsic parameter matrix of the camera, (u jL ,v jL ) is the image coordinate of the center of the fringe at the jth position, (x wi ,y wi ,z wi ) is the three-dimensional coordinate of the center of the fringe at the i-th position; is the relative rotation matrix of the j-th position compared to the i-th position, is the relative translation matrix of the j-th position compared to the i-th position;

[0079] A calculation unit is used to introduce several groups of sub-pixel coordinates of the same name in the adjacent frame stripe images at each pair of adjacent positions into the left camera perspective projection model after deformation processing, and define Q = K -1 [u jL ,v jL ,1] T ,M=[R j i T j i ],m=[m 11 ,m 12 ,m 13 ,m 14 ,m 21 ,m 22 ,m 23 ,m 24 ,m 31 ,m 32 ,m 33 ,m 34 ] T , where m ab (a=1,2,3;b=1,2,3,4) represents the element in the ath row and bth column of the matrix M; after rearrangement, we get:

[0080]

[0081] Where Q(k) (k = 1, 2, 3) represents the kth element of the matrix Q, x wi ,y wi ,z wi The 3D coordinates of the center of the stripe at the i-th position; at this time, when n pairs of points of the same name between adjacent frames are given, the second overdetermined linear equation Bm=0 can be obtained, where B is given by Vertical stacking, b i are the first two rows of the sorted matrix; perform singular value decomposition on it to obtain the rough relative pose at each pair of adjacent positions;

[0082] The optimization unit is used to minimize the reprojection error of the k-th point of the same name in the object space at the i-th position and the j-th position:

[0083]

[0084] The rough relative pose is used as the initial value for iterative calculation to obtain the corresponding optimal relative pose; are the three-dimensional coordinates of the kth feature point at the jth position and the i-th position respectively.

[0085] In a third aspect, the technical solution provides an electronic device, comprising at least one processor, wherein the processor is coupled to a memory, wherein a computer program is stored in the memory, and wherein the computer program is configured to execute the method described when executed by the processor.

[0086] In a fourth aspect, the present technical solution provides a computer-readable storage medium having a computer program stored thereon, and the computer program implements the described method when executed by a computer.

[0087] Beneficial effects:

[0088] It can be seen from the above technical solutions that the technical solution of the present invention provides a three-dimensional reconstruction method of line structured light dynamic stereoscopic vision to improve the technical problem that the existing visual measurement range is limited and cannot be applied to the three-dimensional reconstruction of large-size targets.

[0089] In order to overcome the limitation of the static stereoscopic vision measurement range based on line structured light and expand the visual measurement range, the present technical solution performs push-scanning dynamic control on the stereoscopic vision device so that several stripe images corresponding to the entire large-scale target object surface can be obtained. And correspondingly, the specific three-dimensional reconstruction process is set as follows: First, considering that when the surface of a large-scale target object is reconstructed based on line structured light, the number of corresponding stripe images is larger, and then the three-dimensional coordinates of the center of the line structured light stripe are obtained by using the triangulation method commonly used in the existing three-dimensional reconstruction of line structured light, there will be defects such as high computational overhead and low efficiency of coordinate extraction. Therefore, the present technical solution proposes to fuse the perspective projection model of the left camera and the plane equation of the line structured light to calculate the three-dimensional coordinates of the center of the line structured light stripe. At this time, it is not necessary to match the line structured light stripes on the left stripe image and the right stripe image, thereby improving the calculation speed and reducing the calculation overhead. Secondly, the method abandons the pixel-level reprojection error minimization method, and adopts the object space reprojection error minimization principle to obtain the fine relative posture between adjacent frames to avoid the introduction of local distortion visual errors, improve the estimation of camera posture and three-dimensional reconstruction accuracy, and improve the robustness of image noise processing. Furthermore, considering the motion process of dynamic stereo vision itself, especially for complex surface conditions such as large-scale objects, noise will inevitably be introduced, thereby affecting the accuracy of 3D reconstruction. Therefore, when obtaining absolute pose, this application abandons the traditional step-by-step transfer method to calculate absolute pose, and adopts a global optimization method to avoid cumulative errors and improve the accuracy of 3D reconstruction of dynamic stereo vision. Based on the above design, this application can achieve high-efficiency and high-precision 3D reconstruction of large-scale complex target object surfaces.

[0090] It should be appreciated that all combinations of the foregoing concepts, as well as additional concepts described in greater detail below, may be considered to be part of the inventive subject matter of the present disclosure, provided such concepts are not mutually inconsistent.

[0091] The foregoing and other aspects, embodiments and features of the present invention can be more fully understood from the following description in conjunction with the accompanying drawings. Other additional aspects of the present invention, such as the features and / or beneficial effects of the exemplary embodiments, will be apparent from the following description or learned from the practice of the specific embodiments according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] The drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component shown in various figures may be represented by the same reference numeral. For clarity, not every component is labeled in every figure. Embodiments of various aspects of the present invention will now be described by way of example and with reference to the accompanying drawings, in which:

[0093] Figure 1 is a flow chart of the three-dimensional reconstruction method of line structured light dynamic stereoscopic vision described in this embodiment;

[0094] Figure 2 is a flow chart for obtaining the light plane equation described in this embodiment;

[0095] Figure 3 is a flowchart of obtaining the relative posture described in this embodiment;

[0096] Figure 4 This is a flow chart of obtaining the absolute rotation matrix in the absolute posture described in this embodiment;

[0097] Figure 5 This is a flow chart for obtaining the absolute translation matrix in the absolute posture described in this embodiment;

[0098] Figure 6 It is a structural block diagram of the three-dimensional reconstruction system of line structured light dynamic stereoscopic vision described in this embodiment;

[0099] Figure 7 is a structural block diagram of the electronic device described in this embodiment. DETAILED DESCRIPTION

[0100] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meaning understood by people with general skills in the field to which the present invention belongs.

[0101] The words "first", "second" and similar words used in the specification and claims of this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, unless the context clearly indicates otherwise, the singular form of "a", "an" or "the" and other similar words do not indicate a quantitative limitation, but rather indicate the presence of at least one. Words such as "include" or "comprise" and the like mean that the elements or objects appearing before "include" or "comprise" cover the features, wholes, steps, operations, elements and / or components listed after "include" or "comprise", and do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0102] With the improvement of image processing technology and the accuracy and efficiency of various photoelectric sensors, the three-dimensional reconstruction system based on line structured light has developed rapidly. Its advantages of wide application range and high reliability have been gradually highlighted in industrial applications, and therefore have received extensive attention and research from scientific researchers. However, although the three-dimensional reconstruction method based on structured light stereo vision is easy to extract stripes under normal conditions, the existing three-dimensional reconstruction method based on structured light is a static visual measurement method, and therefore has the defect of limited visual measurement range, which makes it only suitable for surface three-dimensional reconstruction of small-sized targets, but cannot be well applied to large-sized targets. Based on this, the present embodiment aims to provide a three-dimensional reconstruction method of line structured light dynamic stereo vision to improve the above technical defects.

[0103] The following is a detailed introduction to the three-dimensional reconstruction method of line structured light dynamic stereoscopic vision described in this embodiment in conjunction with the accompanying drawings.

[0104] Combination Figure 1 As shown, the method comprises the following steps:

[0105] Step S102: Control the stereoscopic vision device to push-scan along a preset direction to obtain a plurality of frames of stripe images covering all positions on the target surface.

[0106] In this embodiment, the stereoscopic vision device includes a right camera, a laser generator, and a left camera that are sequentially arranged relative to the target surface. Specifically, the preset push-scan direction is the horizontal direction. The stripe image corresponding to each position of the target surface includes a right stripe image obtained by the right camera and a left stripe image obtained by the left camera.

[0107] Step S104: extracting the sub-pixel coordinates of the fringe image at each position.

[0108] In the specific implementation, first, the right fringe image at each position is processed based on the intrinsic parameter matrix of the right camera and the extrinsic parameter matrix of the right camera relative to the left camera, and the left fringe image at each position is processed based on the intrinsic parameter matrix of the left camera. Then, the processed right fringe image and left fringe image are input into the Steger algorithm to extract the sub-pixel coordinates of the fringe image at each position.

[0109] Specifically, the intrinsic parameter matrices of the two cameras and the extrinsic parameter matrix of the right camera relative to the left camera are obtained by Zhang's calibration method. Specifically, the extrinsic parameter matrix of the right camera relative to the left camera includes: a rotation matrix of the right camera relative to the left camera, and a translation matrix of the right camera relative to the left camera.

[0110] Step S106: introducing the sub-pixel coordinates into the left camera perspective projection model, and establishing the pre-constructed light plane equation in conjunction with the equation to obtain the three-dimensional coordinates of the fringe center of the fringe image at each position.

[0111] In specific implementation, the left camera perspective projection model in this step is expressed as:

[0112]

[0113] Among them, λ is the scale factor, K is the intrinsic parameter matrix of the left camera, (u,v) is the sub-pixel coordinate of the fringe image; (x w ,y w ,z w ) is the three-dimensional coordinate of the center point of the stripe in the left camera coordinate system.

[0114] The light plane equation is expressed as:

[0115] AX w +BY w +CZ w +D=0;

[0116] Among them, [A,B,C,D] T is the normal vector of the light plane equation.

[0117] By linking them together, we can obtain the three-dimensional coordinates of the fringe center of the fringe image at each position:

[0118]

[0119] As a specific implementation, this embodiment uses a single camera and a laser line generator joint calibration method to calculate the light plane equation of the line structured light. Figure 2 As shown, the following steps are included:

[0120] Step S202: Obtain the left camera perspective projection imaging model.

[0121] Specifically, the three-dimensional coordinates of a chessboard angle in the corresponding chessboard coordinate system are defined as (x w ,y w ,z w ), where z w = 0; the projection of the chessboard angle on the left camera image plane is denoted as g(u g ,v g ), and then obtain the left camera perspective projection imaging model as:

[0122]

[0123] Among them, K is the intrinsic parameter matrix of the left camera, f x 、f y are the focal lengths of the left camera in the x and y directions, c x 、c y are the principal points of the left camera in the x and y directions respectively, λ is the scale factor, r chess is the calibration rotation matrix of the chessboard coordinate system compared to the left camera coordinate system, t chess is the calibration translation matrix of the chessboard coordinate system compared to the left camera coordinate system.

[0124] Step S204: calibrate the rough values ​​of the rotation matrix and the translation matrix.

[0125] For a given number of chessboard corners with different rotation angles and different translation positions, the following definition matrix is ​​constructed: H = K[R chess t chess ] and h=[h 11 ,h 12 ,h 13 ,h 14 ,h 21 ,h 22 ,h 23 ,h 24 ,h 31 ,h 32 ,h 33 ,h 34 ] T , where h ij (i=1,2,3; j=1,2,3,4) represents the elements of the i-th row and j-th column of the matrix H; and then the definition matrix is ​​substituted into the left camera perspective projection imaging model to obtain:

[0126]

[0127] At this time, for a given number of n chessboard corners, the matrix equation Ah = 0 can be obtained, where A is given by Vertical stacking, a iare the first two rows of the matrix equation; singular value decomposition is performed on the matrix equation to obtain rough values ​​of the calibration rotation matrix and the calibration translation matrix.

[0128] Step S206: Create a total error optimization objective function:

[0129]

[0130] The rough values ​​of the calibration rotation matrix and the calibration translation matrix are used as initial values ​​to perform iterative calculation to obtain the precise values ​​of the calibration rotation matrix and the calibration translation matrix; wherein, x' is the mapping function of the chessboard corner image coordinates.

[0131] Step S208: Obtain the structured light plane normal vector.

[0132] Specifically, select any two points on the projection line of the laser stripe on the chessboard plane in the camera coordinate system to construct the projection line equation: L = M a M b T -M b M a T ; and place the chessboard plane in different directions to obtain the first determined linear equation Then perform singular value decomposition to obtain the normal vector of the structured light plane; where M a is the coordinate of the first point on the projection line, M b is the coordinate of the second point on the projection line, is the vertical stack of the projection line equation L, and n is the normal vector of the structured light plane.

[0133] Step S210: construct the light plane equation based on the structured light plane normal vector.

[0134] At this time, when the light plane equation is acquired based on steps S202 to S210, since there is no need to match the line structured light fringes on the left and right images, the calculation speed is improved.

[0135] Step S108, obtaining the rough relative pose at each pair of adjacent positions based on the left camera perspective projection model, and optimizing it to obtain the corresponding optimal relative pose; and the relative pose of each device of the stereoscopic vision device at each position compared to the stereoscopic vision device at the initial position.

[0136] In the specific implementation, the relative pose is obtained in the following way: first, extract several groups of sub-pixel coordinates with the same name in the adjacent frame stripe images at each pair of adjacent positions, and introduce them into the left camera perspective projection model to obtain the rough relative pose at each pair of adjacent positions; secondly, iteratively optimize each rough relative pose based on the object space reprojection error minimization function to obtain the corresponding optimal relative pose. The sub-pixel coordinates with the same name refer to the pixel coordinates corresponding to the same position in the adjacent frame stripe images.

[0137] Further, combined with Figure 3 As shown, the following steps are included:

[0138] Step S10802: deforming the left camera perspective projection model by:

[0139]

[0140] Among them, λ is the scale factor, K is the intrinsic parameter matrix of the camera, (u jL ,v jL ) is, (x wi ,y wi ,z wi )for, is the relative rotation matrix of the j-th position compared to the i-th position, is the relative translation matrix of the j-th position compared to the i-th position.

[0141] Step S10804: Obtain the rough relative pose at each pair of adjacent positions.

[0142] Introduce several groups of sub-pixel coordinates with the same name in each pair of adjacent frame stripe images into the deformed left camera perspective projection model, and define Q = K -1 [u jL ,v jL ,1] T ,M=[R j i T j i ],m=[m 11 ,m 12 ,m 13 ,m 14 ,m 21 ,m 22 ,m 23 ,m 24 ,m 31 ,m 32 ,m 33 ,m 34 ] T , where m ab(a=1,2,3;b=1,2,3,4) represents the element in the ath row and bth column of the matrix M; after rearrangement, we get:

[0143]

[0144] Where Q(k) (k = 1, 2, 3) represents the kth element of the matrix Q, x wi ,y wi ,z wi The 3D coordinates of the center of the stripe at the i-th position; at this time, when n pairs of points of the same name between adjacent frames are given, the second overdetermined linear equation Bm=0 can be obtained, where B is given by Vertical stacking, b i are the first two rows of the sorted matrix; perform singular value decomposition on it to obtain the coarse relative pose at each pair of adjacent positions.

[0145] Step S10806: Obtain the optimal relative pose at each pair of adjacent positions.

[0146] The minimization function of the reprojection error of the k-th point of the same name at the i-th position and the j-th position in the object space is established as:

[0147]

[0148] The rough relative pose is used as the initial value for iterative calculation to obtain the corresponding optimal relative pose; are the three-dimensional coordinates of the kth feature point at the jth position and the i-th position respectively.

[0149] At this time, based on steps S10802 to S10806, the object space reprojection error minimization principle is used to obtain the fine relative pose between adjacent frames. Since it can more directly process the three-dimensional geometric relationship, it improves the estimation ability of the camera pose and the three-dimensional reconstruction accuracy, and is more robust to image noise. At the same time, it can better adapt to changes when processing large-scale scenes.

[0150] Step S110: Using the left camera coordinate system at the initial position as a reference coordinate system, the relative postures of the devices are brought into corresponding optimal relative postures to obtain the absolute posture of the fringe image at each position compared to the reference coordinate system.

[0151] As a specific implementation method, Figure 4 As shown, the absolute rotation matrix in the absolute pose is obtained as follows:

[0152] Step S11002: Based on the standard rotation relationship The first relative rotation relationship between the optimal relative postures is obtained as follows:

[0153]

[0154] in, R i is the absolute rotation matrix at the i-th position, R j is the absolute rotation matrix at the jth position; R i , R j The k-th column vector of ; is the relative rotation matrix of the j-th position relative to the i-th position.

[0155] Step S11004: Based on the standard rotation relationship The second relative rotation relationship between the relative positions of each device is obtained as follows:

[0156]

[0157] in, is the rotation matrix of the binocular module at the i-th position relative to the initial position, is the rotation matrix of the binocular module at the jth position relative to the initial position; is a unit matrix, and R0 is the rotation matrix of the binocular module at the initial position.

[0158] Step S11006: Substitute the second relative rotation relationship into the first relative rotation relationship, and solve it using the least squares method.

[0159] Step S11008: When it is determined that the solution of the least squares method is a unit orthogonal matrix, the solution is used as the absolute rotation matrix in the absolute posture; otherwise, a forced orthogonal equation is introduced as follows to re-estimate the absolute rotation matrix:

[0160]

[0161] in, is the forced orthogonal matrix corresponding to the absolute rotation matrix of the ith position, F is the Frobenius norm, and I is the identity matrix.

[0162] Similarly, as a specific implementation method, combined with Figure 5 As shown, the absolute translation matrix in the absolute pose is obtained as follows:

[0163] Step S11022: Obtain standard translation relationship As the first relative translation relationship between the optimal relative postures;

[0164] Among them, T i is the absolute translation matrix at the ith position, T j is the absolute translation matrix at the jth position, is the relative translation matrix of the j-th position compared to the i-th position.

[0165] Step S11024: Based on the standard translation relationship The second relative translation relationship between the relative positions of each device is obtained as follows:

[0166]

[0167] in, is the rotation matrix of the binocular module at the i-th position relative to the initial position, is the rotation matrix of the binocular module at the jth position relative to the initial position; T0 is the translation matrix of the binocular module at the initial position.

[0168] Step S11026, substituting the second relative translation relationship into the first relative translation relationship to obtain:

[0169]

[0170] The least square method is used to solve it. Since the absolute translation matrix does not need to satisfy such an orthogonal constraint, the solution here is used as the absolute translation matrix in the absolute pose.

[0171] At this time, based on step S110, and specifically steps S11002 to S11008, steps S11022 to S11026 will realize the acquisition of the absolute pose at each position (specifically, including the absolute rotation matrix and the absolute translation matrix in the absolute pose) based on the global optimization algorithm. Compared with the step-by-step transfer method used in the prior art to acquire the absolute pose, the cumulative error is effectively reduced, thereby avoiding the influence of noise and improving the three-dimensional reconstruction accuracy of dynamic stereoscopic vision.

[0172] At this point, continue with the following steps to achieve surface 3D reconstruction:

[0173] Step S112: based on each absolute posture, convert the three-dimensional coordinates of the center of each stripe into the reference coordinate system to achieve three-dimensional reconstruction of the target surface.

[0174] From the above, it can be seen that in the three-dimensional reconstruction based on line structured light, this embodiment firstly abandons the widely used two-dimensional parallax, and calculates the three-dimensional coordinates of the line structured light stripes at various positions by fusing the camera perspective projection imaging model and the line structured light plane equation; secondly, by solving the rotation and translation of dynamic stereo vision at various positions relative to the reference position, the three-dimensional reconstruction of the target surface is realized; this method abandons the pixel-level reprojection error minimization method, and adopts the object space reprojection error minimization principle to obtain the fine relative pose between adjacent frames; because dynamic stereo vision will introduce noise during the movement process, the absolute pose optimal solution of each position is calculated through global optimization. As a result, the dynamic stereo vision three-dimensional reconstruction method of this embodiment can be reconstructed with high precision and high efficiency, which is suitable for large-size device surface reconstruction.

[0175] The above program can be run in the processor, or it can also be stored in the memory (or computer-readable storage medium), which includes permanent and non-permanent, removable and non-removable media. Information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include temporary computer-readable media, such as modulated data signals and carrier waves.

[0176] These computer programs can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in one or more blocks can be implemented by different modules corresponding to different steps.

[0177] Based on the above method, this embodiment also provides a three-dimensional reconstruction system of line structured light dynamic stereo vision. Figure 6 As shown, the system includes the following functional modules:

[0178] An image acquisition module is used to control a stereoscopic vision device to push-scan along a preset direction to acquire a plurality of frames of stripe images covering all positions on a target surface; wherein the stereoscopic vision device comprises a right camera, a laser generator, and a left camera which are sequentially arranged with respect to the target surface; and the stripe images corresponding to each position on the target surface include a right stripe image acquired by the right camera and a left stripe image acquired by the left camera.

[0179] The pixel coordinate acquisition module is used to process the right side stripe image at each position based on the intrinsic parameter matrix of the right camera and the extrinsic parameter matrix of the right camera relative to the left camera, and to process the left side stripe image at each position based on the intrinsic parameter matrix of the left camera; and then input the processed right side stripe image and left side stripe image into the Steger algorithm to extract the sub-pixel coordinates of the stripe image at each position.

[0180] The fringe center extraction module is used to introduce the sub-pixel coordinates into the left camera perspective projection model and to establish a pre-constructed light plane equation to obtain the fringe center three-dimensional coordinates of the fringe image at each position.

[0181] The relative posture acquisition module is used to extract several groups of sub-pixel coordinates with the same name in the adjacent frame stripe images at each pair of adjacent positions, and introduce them into the left camera perspective projection model to obtain the rough relative posture at each pair of adjacent positions, and iteratively optimize each rough relative posture based on the reprojection error minimization function of the object space to obtain the corresponding optimal relative posture; at the same time, the relative posture of each device of the stereoscopic vision device at each position compared with the stereoscopic vision device at the initial position is obtained; wherein the sub-pixel coordinates with the same name refer to the pixel coordinates corresponding to the same position in the adjacent frame stripe images.

[0182] The absolute posture acquisition module is used to use the left camera coordinate system at the initial position as the reference coordinate system, and bring the relative posture of each device into the corresponding optimal relative posture to obtain the absolute posture of the fringe image at each position compared with the reference coordinate system.

[0183] The three-dimensional reconstruction module is used to transform the three-dimensional coordinates of the center of each stripe into the reference coordinate system based on each absolute posture to achieve three-dimensional reconstruction of the target surface.

[0184] Since the system is used to implement the steps of the above method, those have already been described and will not be repeated here.

[0185] For example, in order to obtain the light plane equation, the system further includes:

[0186] The model acquisition module is used to define the three-dimensional coordinates of a chessboard angle in the corresponding chessboard coordinate system as (x w ,y w ,z w ), where zw = 0; the projection of the chessboard angle on the left camera image plane is denoted as g(u g ,v g ), and then obtain the left camera perspective projection imaging model as:

[0187]

[0188] Among them, K is the intrinsic parameter matrix of the left camera, f x 、f y are the focal lengths of the left camera in the x and y directions, c x 、c y are the principal points of the left camera in the x and y directions respectively, λ is the scale factor, r chess is the calibration rotation matrix of the chessboard coordinate system compared to the left camera coordinate system, t chess is the calibration translation matrix of the chessboard coordinate system compared to the left camera coordinate system.

[0189] The rough value acquisition module is used to construct the following definition matrix for a given number of chessboard corners with different rotation angles and different translation positions: H = K[R chess t chess ] and h=[h 11 ,h 12 ,h 13 ,h 14 ,h 21 ,h 22 ,h 23 ,h 24 ,h 31 ,h 32 ,h 33 ,h 34 ] T , where h ij (i=1,2,3; j=1,2,3,4) represents the elements of the i-th row and j-th column of the matrix H; and then the definition matrix is ​​substituted into the left camera perspective projection imaging model to obtain:

[0190]

[0191] At this time, for a given number of n chessboard corners, the matrix equation Ah = 0 can be obtained, where A is given by Vertical stacking, a i are the first two rows of the matrix equation; singular value decomposition is performed on the matrix equation to obtain rough values ​​of the calibration rotation matrix and the calibration translation matrix.

[0192] The precise value acquisition module is used to create the total error optimization objective function:

[0193]

[0194] The rough values ​​of the calibration rotation matrix and the calibration translation matrix are used as initial values ​​to perform iterative calculation to obtain the precise values ​​of the calibration rotation matrix and the calibration translation matrix; wherein x' is a mapping function of the chessboard corner image coordinates.

[0195] The normal vector acquisition module is used to select any two points on the projection line of the laser stripe on the chessboard plane in the camera coordinate system to construct the projection line equation: L = M a M b T -M b M a T ; and place the chessboard plane in different directions to obtain the first determined linear equation Then perform singular value decomposition to obtain the normal vector of the structured light plane; where M a is the coordinate of the first point on the projection line, M b is the coordinate of the second point on the projection line, is the vertical stack of the projection line equation L, and n is the normal vector of the structured light plane.

[0196] The light plane equation acquisition module is used to construct the light plane equation based on the structured light plane normal vector.

[0197] For example, the relative posture acquisition module includes:

[0198] The deformation processing unit is used to deform the left camera perspective projection model:

[0199]

[0200] Among them, λ is the scale factor, K is the intrinsic parameter matrix of the camera, (u jL ,v jL ) is the image coordinate of the center of the fringe at the jth position, (x wi ,y wi ,z wi ) is the three-dimensional coordinate of the center of the fringe at the i-th position; is the relative rotation matrix of the j-th position compared to the i-th position, is the relative translation matrix of the j-th position compared to the i-th position.

[0201] A calculation unit is used to introduce several groups of sub-pixel coordinates of the same name in the adjacent frame stripe images at each pair of adjacent positions into the left camera perspective projection model after deformation processing, and define Q = K -1 [u jL ,v jL ,1] T ,M=[R ji T j i ],m=[m 11 ,m 12 ,m 13 ,m 14 ,m 21 ,m 22 ,m 23 ,m 24 ,m 31 ,m 32 ,m 33 ,m 34 ] T , where m ab (a=1,2,3;b=1,2,3,4) represents the element in the ath row and bth column of the matrix M; after rearrangement, we get:

[0202]

[0203] Where Q(k) (k = 1, 2, 3) represents the kth element of the matrix Q, x wi ,y wi ,z wi The 3D coordinates of the center of the stripe at the i-th position; at this time, when n pairs of points of the same name between adjacent frames are given, the second overdetermined linear equation Bm=0 can be obtained, where B is given by Vertical stacking, b i are the first two rows of the sorted matrix; perform singular value decomposition on it to obtain the coarse relative pose at each pair of adjacent positions.

[0204] The optimization unit is used to minimize the reprojection error of the k-th point of the same name in the object space at the i-th position and the j-th position:

[0205]

[0206] The rough relative pose is used as the initial value for iterative calculation to obtain the corresponding optimal relative pose; are the three-dimensional coordinates of the kth feature point at the jth position and the i-th position respectively.

[0207] Combination Figure 7 As shown, this embodiment further provides an electronic device, comprising at least one processor, the processor being coupled to a memory, the memory storing a computer program, and the computer program being configured to execute the method when being executed by the processor.

[0208] Similarly, this embodiment also provides a computer-readable storage medium, characterized in that a computer program is stored thereon, and the computer program implements the method when executed by a computer.

[0209] Since the system, the electronic device and the computer-readable storage medium are all constructed based on the method, they also have the technical advantage of being suitable for three-dimensional reconstruction of large-sized objects in practical applications.

[0210] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. A person with ordinary knowledge in the technical field to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the definition of the claims.

Claims

1. A three-dimensional reconstruction method of line structured light dynamic stereo vision, characterized in that: include: Controlling the stereoscopic vision device to push-scan along a preset direction to acquire a plurality of frames of stripe images covering all positions on the target surface; wherein the stereoscopic vision device comprises a right camera, a laser generator, and a left camera which are sequentially arranged with respect to the target surface; and the stripe images corresponding to each position on the target surface include a right stripe image acquired by the right camera and a left stripe image acquired by the left camera; The right fringe image at each position is processed based on the intrinsic parameter matrix of the right camera and the extrinsic parameter matrix of the right camera relative to the left camera, and the left fringe image at each position is processed based on the intrinsic parameter matrix of the left camera; then the processed right fringe image and left fringe image are input into the Steger algorithm to extract the sub-pixel coordinates of the fringe image at each position; The sub-pixel coordinates are introduced into the left camera perspective projection model, and the pre-constructed light plane equation is established to obtain the three-dimensional coordinates of the fringe center of the fringe image at each position; Extract several groups of sub-pixel coordinates with the same name in the adjacent frame stripe images at each pair of adjacent positions, and introduce them into the left camera perspective projection model to obtain the rough relative pose at each pair of adjacent positions, and iteratively optimize each rough relative pose based on the object space reprojection error minimization function to obtain the corresponding optimal relative pose; at the same time, obtain the relative pose of each device of the stereoscopic vision device at each position compared with the stereoscopic vision device at the initial position; wherein the sub-pixel coordinates with the same name refer to the pixel coordinates corresponding to the same position in the adjacent frame stripe images; Taking the left camera coordinate system at the initial position as the reference coordinate system, bringing the relative posture of each of the devices into the corresponding optimal relative posture to obtain the absolute posture of the fringe image at each position compared to the reference coordinate system; Based on each absolute position, the three-dimensional coordinates of the center of each stripe are converted into the reference coordinate system to achieve three-dimensional reconstruction of the target surface.

2. The three-dimensional reconstruction method of line structured light dynamic stereoscopic vision according to claim 1, characterized in that: The method comprises obtaining the light plane equation; comprising: Define the three-dimensional coordinates of a chessboard angle in the corresponding chessboard coordinate system as (x w ,y w ,z w ), where z w = 0; the projection of the chessboard angle on the left camera image plane is denoted as g(u g ,v g ), and then obtain the left camera perspective projection imaging model as: Among them, K is the intrinsic parameter matrix of the left camera, f x 、f y are the focal lengths of the left camera in the x and y directions, c x 、c y are the principal points of the left camera in the x and y directions respectively, λ is the scale factor, r chess is the calibration rotation matrix of the chessboard coordinate system compared to the left camera coordinate system, t chess is the calibration translation matrix of the chessboard coordinate system compared to the left camera coordinate system; For a given number of chessboard corners with different rotation angles and different translation positions, the following definition matrix is ​​constructed: H = K[R chess t chess ] and h=[h 11 ,h 12 ,h 13 ,h 14 ,h 21 ,h 22 ,h 23 ,h 24 ,h 31 ,h 32 ,h 33 ,h 34 ] T , where h ij (i=1,2,3; j=1,2,3,4) represents the elements of the i-th row and j-th column of the matrix H; and then the definition matrix is ​​substituted into the left camera perspective projection imaging model to obtain: At this time, for a given number of n chessboard corners, the matrix equation Ah = 0 can be obtained, where A is given by Vertical stacking, a i are the first two rows of the matrix equation; performing singular value decomposition on the matrix equation to obtain rough values ​​of the calibration rotation matrix and the calibration translation matrix; Create a total error optimization objective function: Using the rough values ​​of the calibration rotation matrix and the calibration translation matrix as initial values, iterative calculation is performed to obtain the precise values ​​of the calibration rotation matrix and the calibration translation matrix; wherein x' is a mapping function of the chessboard corner image coordinates; Select any two points on the projection line of the laser stripe on the chessboard plane in the camera coordinate system to construct the projection line equation: L = M a M b T -M b M a T ; and place the chessboard plane in different directions to obtain the first determined linear equation Then perform singular value decomposition to obtain the normal vector of the structured light plane; where M a is the coordinate of the first point on the projection line, M b are the coordinates of the second point on the projection line, is the vertical stack of the projection line equation L, n is the normal vector of the structured light plane; The light plane equation is constructed based on the structured light plane normal vector.

3. The three-dimensional reconstruction method of line structured light dynamic stereoscopic vision according to claim 1, characterized in that: The method extracts several groups of sub-pixel coordinates of the same name from the adjacent frame fringe images at each pair of adjacent positions, introduces them into the left camera perspective projection model to obtain the rough relative pose at each pair of adjacent positions, and iteratively optimizes each rough relative pose based on the reprojection error minimization function of the object space to obtain the corresponding optimal relative pose; including: The deformation processing of the left camera perspective projection model is as follows: Among them, λ is the scale factor, K is the intrinsic parameter matrix of the camera, (u jL ,v jL ) is the image coordinate of the center of the fringe at the jth position, (x wi ,y wi ,z wi ) is the three-dimensional coordinate of the center of the fringe at the i-th position; is the relative rotation matrix of the j-th position compared to the i-th position, is the relative translation matrix of the j-th position compared to the i-th position; Introduce several groups of sub-pixel coordinates with the same name in each pair of adjacent frame stripe images into the deformed left camera perspective projection model, and define Q = K -1 [u jL ,v jL ,1] T ,M=[R j i T j i ],m=[m 11 ,m 12 ,m 13 ,m 14 ,m 21 ,m 22 ,m 23 ,m 24 ,m 31 ,m 32 ,m 33 ,m 34 ] T , where m ab (a=1,2,3;b=1,2,3,4) represents the element in the ath row and bth column of the matrix M; after rearrangement, we get: Where Q(k) (k = 1, 2, 3) represents the kth element of the matrix Q, x wi ,y wi ,z wi The 3D coordinates of the center of the stripe at the i-th position; at this time, when n pairs of points of the same name between adjacent frames are given, the second overdetermined linear equation Bm=0 can be obtained, where B is given by Vertical stacking, b i are the first two rows of the sorted matrix; perform singular value decomposition on it to obtain the rough relative pose at each pair of adjacent positions; The minimization function of the reprojection error of the k-th point of the same name at the i-th position and the j-th position in the object space is established as: The rough relative pose is used as the initial value for iterative calculation to obtain the corresponding optimal relative pose; are the three-dimensional coordinates of the kth feature point at the jth position and the i-th position respectively.

4. The three-dimensional reconstruction method of line structured light dynamic stereoscopic vision according to claim 1, characterized in that: The method uses the left camera coordinate system at the initial position as the reference coordinate system, and brings the relative posture of each device into the corresponding optimal relative posture to obtain the absolute posture of the fringe image at each position compared with the reference coordinate system; including: Based on standard rotation relationship The first relative rotation relationship between the optimal relative postures is obtained as follows: in, R i is the absolute rotation matrix at the i-th position, R j is the absolute rotation matrix at the jth position; R i , R j The k-th column vector of ; is the relative rotation matrix of the j-th position compared to the i-th position; Based on standard rotation relationship The second relative rotation relationship between the relative positions of each device is obtained as follows: in, is the rotation matrix of the binocular module at the i-th position relative to the initial position, is the rotation matrix of the binocular module at the jth position relative to the initial position; is a unit matrix, R0 is the rotation matrix of the binocular module at the initial position; Substituting the second relative rotation relationship into the first relative rotation relationship, and solving it by using the least square method; When it is determined that the solution of the least squares method is a unit orthogonal matrix, the solution is used as the absolute rotation matrix in the absolute pose; otherwise, a forced orthogonal equation is introduced as follows to re-estimate the absolute rotation matrix: in, is the forced orthogonal matrix corresponding to the absolute rotation matrix of the ith position, F is the Frobenius norm, and I is the identity matrix.

5. The three-dimensional reconstruction method of line structured light dynamic stereoscopic vision according to claim 1, characterized in that: The method uses the left camera coordinate system at the initial position as the reference coordinate system, and brings the relative posture of each device into the corresponding optimal relative posture to obtain the absolute posture of the fringe image at each position compared with the reference coordinate system; including: Get the standard translation relationship As the first relative translation relationship between the optimal relative postures; Among them, T i is the absolute translation matrix at the ith position, T j is the absolute translation matrix at the jth position, is the relative translation matrix of the j-th position compared to the i-th position; Based on standard translation relationship The second relative translation relationship between the relative positions of each device is obtained as follows: in, is the rotation matrix of the binocular module at the i-th position relative to the initial position, is the rotation matrix of the binocular module at the jth position relative to the initial position; T0 is the translation matrix of the binocular module at the initial position; Substituting the second relative translation relationship into the first relative translation relationship, we get: The least squares method is used to solve the problem to obtain the absolute translation matrix in the absolute pose.

6. A three-dimensional reconstruction system of line structured light dynamic stereo vision, characterized in that: include: An image acquisition module, used to control the stereoscopic vision device to push-scan along a preset direction to acquire a plurality of frames of stripe images covering all positions on the target surface; wherein the stereoscopic vision device comprises a right camera, a laser generator, and a left camera which are sequentially arranged with respect to the target surface; and the stripe images corresponding to each position on the target surface include a right stripe image acquired by the right camera and a left stripe image acquired by the left camera; A pixel coordinate acquisition module is used to process the right fringe image at each position based on the intrinsic parameter matrix of the right camera and the extrinsic parameter matrix of the right camera relative to the left camera, and to process the left fringe image at each position based on the intrinsic parameter matrix of the left camera; and then input the processed right fringe image and left fringe image into the Steger algorithm to extract the sub-pixel coordinates of the fringe image at each position; A fringe center extraction module, used to introduce the sub-pixel coordinates into the left camera perspective projection model and establish a pre-constructed light plane equation to obtain the fringe center three-dimensional coordinates of the fringe image at each position; A relative posture acquisition module is used to extract several groups of sub-pixel coordinates of the same name in adjacent frame stripe images at each pair of adjacent positions, and introduce them into the left camera perspective projection model to obtain a rough relative posture at each pair of adjacent positions, and iteratively optimize each rough relative posture based on the object space reprojection error minimization function to obtain the corresponding optimal relative posture; and simultaneously obtain the relative posture of each device of the stereoscopic vision device at each position compared with the stereoscopic vision device at the initial position; wherein the sub-pixel coordinates of the same name refer to the pixel coordinates corresponding to the same position in the adjacent frame stripe images; An absolute posture acquisition module, used to use the left camera coordinate system at the initial position as a reference coordinate system, and bring the relative posture of each device into a corresponding optimal relative posture to obtain the absolute posture of the fringe image at each position compared to the reference coordinate system; The three-dimensional reconstruction module is used to transform the three-dimensional coordinates of the center of each stripe into the reference coordinate system based on each absolute posture to achieve three-dimensional reconstruction of the target surface.

7. The three-dimensional reconstruction system of line structured light dynamic stereoscopic vision according to claim 6, characterized in that: include: The model acquisition module is used to define the three-dimensional coordinates of a chessboard angle in the corresponding chessboard coordinate system as (x w ,y w ,z w ), where z w = 0; the projection of the chessboard angle on the left camera image plane is denoted as g(u g ,v g ), and then obtain the left camera perspective projection imaging model as: Among them, K is the intrinsic parameter matrix of the left camera, f x 、f y are the focal lengths of the left camera in the x and y directions, c x 、c y are the principal points of the left camera in the x and y directions respectively, λ is the scale factor, r chess is the calibration rotation matrix of the chessboard coordinate system compared to the left camera coordinate system, t chess is the calibration translation matrix of the chessboard coordinate system compared to the left camera coordinate system; The rough value acquisition module is used to construct the following definition matrix for a given number of chessboard corners with different rotation angles and different translation positions: H = K[R chess t chess ] and h=[h 11 ,h 12 ,h 13 ,h 14 ,h 21 ,h 22 ,h 23 ,h 24 ,h 31 ,h 32 ,h 33 ,h 34 ] T , where h ij (i=1,2,3; j=1,2,3,4) represents the elements of the i-th row and j-th column of the matrix H; and then the definition matrix is ​​substituted into the left camera perspective projection imaging model to obtain: At this time, for a given number of n chessboard corners, the matrix equation Ah = 0 can be obtained, where A is given by Vertical stacking, a i are the first two rows of the matrix equation; performing singular value decomposition on the matrix equation to obtain rough values ​​of the calibration rotation matrix and the calibration translation matrix; The precise value acquisition module is used to create the total error optimization objective function: Using the rough values ​​of the calibration rotation matrix and the calibration translation matrix as initial values, iterative calculation is performed to obtain the precise values ​​of the calibration rotation matrix and the calibration translation matrix; wherein x' is a mapping function of the chessboard corner image coordinates; The normal vector acquisition module is used to select any two points on the projection line of the laser stripe on the chessboard plane in the camera coordinate system to construct the projection line equation: L = M a M b T -M b M a T ; and place the chessboard plane in different directions to obtain the first determined linear equation Then perform singular value decomposition to obtain the normal vector of the structured light plane; where M a is the coordinate of the first point on the projection line, M b are the coordinates of the second point on the projection line, is the vertical stack of the projection line equation L, n is the normal vector of the structured light plane; A light plane equation acquisition module is used to construct the light plane equation based on the structured light plane normal vector.

8. The three-dimensional reconstruction system of line structured light dynamic stereoscopic vision according to claim 6, characterized in that: The relative posture acquisition module comprises: The deformation processing unit is used to deform the left camera perspective projection model: Among them, λ is the scale factor, K is the intrinsic parameter matrix of the camera, (u jL ,v jL ) is the image coordinate of the center of the fringe at the jth position, (x wi ,y wi ,z wi ) is the three-dimensional coordinate of the center of the fringe at the i-th position; is the relative rotation matrix of the j-th position compared to the i-th position, is the relative translation matrix of the j-th position compared to the i-th position; A calculation unit is used to introduce several groups of sub-pixel coordinates of the same name in the adjacent frame stripe images at each pair of adjacent positions into the left camera perspective projection model after deformation processing, and define Q = K -1 [u jL ,v jL ,1] T ,M=[R j i T j i ],m=[m 11 ,m 12 ,m 13 ,m 14 ,m 21 ,m 22 ,m 23 ,m 24 ,m 31 ,m 32 ,m 33 ,m 34 ] T , where m ab (a=1,2,3;b=1,2,3,4) represents the element in the ath row and bth column of the matrix M; after rearrangement, we get: Where Q(k) (k = 1, 2, 3) represents the kth element of the matrix Q, x wi ,y wi ,z wi The 3D coordinates of the center of the stripe at the i-th position; at this time, when n pairs of points of the same name between adjacent frames are given, the second overdetermined linear equation Bm=0 can be obtained, where B is given by Vertical stacking, b i are the first two rows of the sorted matrix; perform singular value decomposition on it to obtain the rough relative pose at each pair of adjacent positions; The optimization unit is used to minimize the reprojection error of the k-th point of the same name in the object space at the i-th position and the j-th position: The rough relative pose is used as the initial value for iterative calculation to obtain the corresponding optimal relative pose; are the three-dimensional coordinates of the kth feature point at the jth position and the i-th position respectively.

9. An electronic device, characterized in that: The method comprises at least one processor, wherein the processor is coupled to a memory, wherein a computer program is stored in the memory, and wherein the computer program is configured to execute the method according to any one of claims 1 to 5 when executed by the processor.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 5 is implemented.

Images