Multi-view multi-laser-ray 3D model reconstruction method

Abstract

The invention relates to a multi-view multi-laser-ray 3D model reconstruction method, which comprises an imaging assembly and an upper computer electrically connected with the imaging assembly, and is characterized in that the imaging assembly comprises a multi-beam laser used for emitting multiple beams of laser; a swinging assembly is arranged right in front of a light path of the multi-beam laser in a swinging manner and is used for transmitting multiple beams of laser emitted by the multi-beam laser to an object to be measured in a planar form; at least three 2D cameras are arranged on the peripheral side of the swing assembly in an array mode. The multi-view multi-laser-ray 3D model reconstruction method has the advantages of being high in precision, multifunctional, efficient in data processing, wide in application prospect and the like, the problems existing in a traditional 3D model reconstruction method can be effectively solved, and the requirements of the modern industry and the science and technology field for 3D model reconstruction are met.

Description

Technical Field

[0001] The present invention relates to the technical field of 3D model reconstruction, and in particular to a method for realizing 3D model reconstruction by using multiple laser lines. Background Art

[0002] In the current 3D model reconstruction technology, traditional methods often have some limitations. For example, monocular or binocular vision methods may not be accurate enough in obtaining object depth information, especially for objects with complex shapes or poor surface textures. Some laser scanning-based methods may have limited scanning range or low efficiency when processing large amounts of data. In addition, while existing model reconstruction technologies are performing three-dimensional reconstruction of objects, it is difficult to efficiently detect multiple aspects such as object size, shape, and residues, and the processing efficiency is low.

[0003] In view of this, a new, more efficient and accurate 3D model reconstruction method is needed to overcome these problems. Summary of the invention

[0004] The purpose of the present invention is to provide a multi-eye and multi-laser line 3D model reconstruction method with the advantages of high precision, multi-function, efficient data processing and broad application prospects, which can effectively solve the problems existing in traditional 3D model reconstruction methods and meet the needs of modern industry and science and technology for 3D model reconstruction, so as to solve the above-mentioned technical problems.

[0005] In order to realize the above technical scheme, the technical scheme of the present invention is as follows: A multi-eye multi-laser line 3D model reconstruction method is mainly implemented based on an imaging component and a host computer. The imaging component includes a multi-beam laser and a swing component. The multi-beam laser is used to emit multiple beams of laser light. The swing component can be swingably arranged in front of the optical path of the multi-beam laser. Its function is to transmit multiple beams of laser light to the object to be measured in a planar form. At least three 2D cameras are arranged in an array on the side of the swing component. These three 2D cameras image the object to be measured with the laser projected. The resulting images are sent to the host computer. These images are fused into a three-dimensional depth of field image through a specific fusion method, thereby completing the 3D model reconstruction.

[0006] Furthermore, the multi-beam laser: as the laser emission source of the entire system, can emit multiple laser beams at the same time, providing sufficient laser light for subsequent scanning and imaging. This multi-beam design increases the laser coverage and improves scanning efficiency.

[0007] Furthermore, the swing component (MEMS galvanometer): The swing component in the present method adopts a MEMS galvanometer, which has the advantages of small size and fast response speed. The MEMS galvanometer can transmit the light emitted by the multi-beam laser to the surface of the object to be tested in the shape of a surface for scanning. Specifically, it can arrange the light emitted by the multi-beam laser into a surface at equal intervals and then project it onto the surface of the object to be tested for scanning. This equal-interval arrangement helps to improve the uniformity and accuracy of the scan, so that the surface of the object can be more comprehensively and accurately irradiated by the laser, providing high-quality scanning data for subsequent 3D model reconstruction.

[0008] Furthermore, at least three 2D cameras are arranged in an array around the swing component. This multi-eye design can observe the object to be measured from different angles. Each 2D camera can capture the image information of the object surface under laser irradiation. The image information from multiple angles complements each other, providing a rich source of data for the host computer to reconstruct the 3D model. Moreover, these three 2D cameras are not only used to reconstruct the 3D model, but also can simultaneously detect the size, shape or residue, realizing multi-functional application in the 3D model reconstruction process and improving the detection efficiency.

[0009] Furthermore, a laser line pickup unit is electrically connected between the imaging component and the host computer, and its main function is to pick up the laser lines imaged on the three 2D cameras and send them to the host computer to complete 3D reconstruction. The laser line pickup unit plays a key role in data transmission and preprocessing in the entire system. It can accurately extract the laser line information in the 2D camera image, reduce the amount of data transmission, and improve the efficiency and accuracy of the host computer in processing data, which helps to complete 3D model reconstruction more quickly and accurately.

[0010] The invention also discloses: a multi-eye multi-laser line 3D model reconstruction method comprises the following steps:

[0011] S1. Laser scanning and image acquisition: Start the multi-beam laser and emit multiple laser beams. The laser beams are projected onto the surface of the object to be tested in the form of a surface arranged at equal intervals through the action of the swing component (MEMS galvanometer) for scanning. At the same time, at least three 2D cameras on the periphery work synchronously to shoot the object to be tested from different angles to obtain images of the object surface under laser irradiation. During the shooting process, the parameters of the 2D camera (such as focal length, exposure time, etc.) can be pre-adjusted according to the characteristics of the object to be tested and the environmental conditions to ensure that a clear and accurate image is obtained.

[0012] S2. Laser line pickup and data transmission: The laser line pickup unit picks up the laser line information in the image formed by the three 2D cameras. In this process, the laser line pickup unit can accurately identify and extract the laser line from the complex image background through specific algorithms and circuit designs, and then send the data containing the laser line information to the host computer. During the data transmission process, a high-speed and stable communication protocol is used to ensure the integrity and timeliness of the data.

[0013] S3, 3D model fusion and reconstruction: After the host computer receives the laser line data from the laser line pickup unit, it fuses the image information collected by at least three 2D cameras into a three-dimensional depth of field image through a specific fusion algorithm. This fusion process is based on the principle of multi-view geometry, and uses the relative position relationship between different cameras and the projection information of the laser line on the surface of the object to calculate the three-dimensional coordinates of each point on the surface of the object. The specific fusion algorithm may include steps such as feature point matching and triangulation. Through these steps, the 3D model of the object to be tested is gradually constructed. At the same time, in this process, the host computer can also analyze and detect the size and shape of the object based on the image information obtained by the three 2D cameras, and by analyzing the reflection of the laser line on the surface of the object, detect whether there are abnormal conditions such as residues on the surface of the object.

[0014] S4. Model optimization and output: After the initial completion of the 3D model reconstruction, the host computer can optimize the model.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] 1) The present invention has high-precision 3D model reconstruction: multi-eye multi-laser line design: through the cooperation of multi-beam lasers and multiple 2D cameras, object information is obtained from multiple angles and multiple laser line scanning directions. This multi-dimensional data acquisition method greatly improves the accuracy of model reconstruction, especially for the details of objects with complex shapes, it can be restored more accurately, reducing the model error caused by a single perspective or a small number of laser line scanning. Uniformly scanned oscillating component: The oscillating component (MEMS galvanometer) arranges the laser into a surface at equal intervals and then projects it onto the surface of the object for scanning, ensuring the uniformity of the scan. This uniform scanning method allows every part of the object surface to receive sufficient laser irradiation and information collection, further improving the quality and accuracy of the 3D model, and making the reconstructed 3D model closer to the true shape of the object.

[0017] 2) The present invention has multifunctional detection capabilities: the three 2D cameras can be used to detect size, shape or residue while being used to reconstruct the 3D model. This multifunctional design allows multiple aspects of information to be obtained during a single scan and data collection process without the need for additional detection equipment and processes, thereby improving detection efficiency and saving time and cost. In particular, in application scenarios such as quality inspection in industrial production, it is possible to simultaneously achieve three-dimensional model reconstruction and quality inspection of products, which has important practical value.

[0018] 3) Efficient data processing and transmission of the present invention: The setting of the laser line pickup unit effectively improves the efficiency of data processing and transmission. It can accurately pick up laser line information from the 2D camera image, reduce the amount of data transmitted to the host computer, and avoid the occupation of system resources by a large amount of useless data. At the same time, through the optimized data transmission method, the timeliness and integrity of the data are guaranteed, so that the host computer can process the data quickly and accurately, thereby improving the speed and efficiency of the entire 3D model reconstruction.

[0019] 4) Broad application prospects of the present invention: The present 3D model reconstruction method can be applied to a variety of fields, such as 3D object modeling, industrial inspection, reverse engineering, virtual reality, etc. In the field of 3D object modeling, high-quality 3D models can be created quickly and accurately for various complex objects; in industrial inspection, 3D inspection and quality assessment of products can be achieved; in reverse engineering, 3D models of the physical objects can be quickly obtained for subsequent design improvements; in the field of virtual reality, high-quality 3D model resources are provided for the creation of virtual scenes, which has broad application prospects and market value.

[0020] In summary, the multi-eye multi-laser line 3D model reconstruction method of the present invention has the advantages of high precision, multi-function, efficient data processing and broad application prospects. It can effectively solve the problems existing in traditional 3D model reconstruction methods and meet the needs of modern industry and science and technology for 3D model reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] To further illustrate the various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, a person of ordinary skill in the art should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0022] Figure 1 This is a schematic diagram of the structure of the multi-eye multi-laser line 3D model reconstruction system. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0025] Please see attached Figure 1 As shown: a multi-eye multi-laser line 3D model reconstruction method, comprising an imaging component 1 and a host computer 2 electrically connected to the imaging component 1, wherein the imaging component 1 comprises a multi-beam laser 11 for emitting multiple laser beams; a swing component 12 is swingably provided in front of the optical path of the multi-beam laser 11, for transmitting the multiple laser beams emitted by the multi-beam laser 11 to the object to be measured in a planar form; at least three 2D cameras 4 are arrayed on the peripheral side of the swing component 12;

[0026] Wherein: at least three images taken by the 2D cameras are sent to the host computer 2, through X=(A T A) -1 A T b fused into a 3D depth image

[0027] Specifically, in the process of multi-eye laser 3D reconstruction, multiple cameras are combined into binocular cameras in sequence. Specifically, these cameras are paired in pairs to form multiple binocular measurement systems. In this way, the principle of binocular vision can be used to obtain the 3D information of the object from different perspectives, thereby achieving more accurate and comprehensive 3D reconstruction. According to the principle of binocular camera epipolar geometric constraints, a 3D point X is set in space, and the projection points on the left and right cameras are m and m′, which are called image matching points and are recorded as Then m′ is located on the epipolar line corresponding to m, and m is located on the epipolar line lm' corresponding to m′, that is, m′∈l′m, m∈lm'. The basic matrix can describe the correspondence between the image points and the epipolar lines of the binocular camera. Let the two camera matrices be K, K′, and the phase planes of the two cameras be represented as I, I′ respectively, then Backprojection epipolar The parametric equation is:

[0028] X(s)=K + m+sC,s∈(-∞,+∞)

[0029] Where K + is the generalized inverse of K, that is, KK +=I, C is the optical center of the first camera, that is, KC = 0, so we can get:

[0030] l′ m =e′m′=(K′C)(K′X(s))=(K′C)((K′Pm+sK′C))

[0031] =(K′C)((K′Km)=[e′] + K′K + m

[0032] That is l′ m =[e′] + K'K + m, so the fundamental matrix between the two cameras F = [e′] × K′K +

[0033] Assume that the intrinsic matrix of the left camera is K, the memory matrix of the right camera is K′, and the transformation relationship from the right camera to the left camera is [R, t]. Therefore, the coordinate transformation relationship between the right camera and the left camera is X'=RX+t, where R is the rotation matrix and t is the translation vector. The projection matrix of the left and right cameras can be expressed as:

[0034] P=K(I,0),P′=K′(R,t)

[0035] according to e′=K′t, so:

[0036]

[0037] According to the method proposed above, the basic matrix between the binocular camera systems formed by all possible pairwise combinations can be calculated in detail. This process involves an accurate geometric relationship analysis of each pair of cameras to ensure that the corresponding points in the three-dimensional space can be correctly projected onto their respective two-dimensional image planes. In this way, the basic matrix between each pair of cameras can be obtained, which will describe the relative position and posture between the cameras. The calculation of these basic matrices provides a solid foundation for the subsequent extreme matching based on the basic matrix, enabling more accurate and robust feature matching in complex visual scenes. This matching not only helps to improve the accuracy of three-dimensional reconstruction, but also provides important geometric constraints for other tasks in computer vision, such as motion estimation and scene understanding.

[0038] When performing multi-camera laser matching, first detect the image coordinates of the laser center point on each camera, and perform initial matching based on the basic matrix between the two cameras calculated above. Suppose the basic matrix on the first two cameras is F 12 , the first binocular system has a laser point p0 on the left camera according to the basic matrix F12 , multiple matching points q can be found i (i=0...n), and satisfies:

[0039] p0F 12 q i (i=0...n)=0

[0040] Similarly, in the second binocular system consisting of two cameras, let the basic matrix of its calculation be F 13 , the first binocular system has a laser point p0 on the left camera according to the basic matrix F 13 , multiple matching points q can be found i ′(i=0...m), and satisfies:

[0041] p0F 12 q i ′(i=0...m)=0

[0042] Similarly, for the third binocular system consisting of two cameras, let the basic matrix of its calculation be F 23 , according to the matched q i (i=0...n) and q′ i (i=0...n), satisfying:

[0043] q i (i=0...n)F 23 q′ i (i=0...m)=0

[0044] According to the above method, the matching relationship of all lasers can be calculated, and finally the three-dimensional reconstruction of multi-line lasers can be completed according to the matching points. Suppose the point pair matched each time is and According to the camera parameter matrix

[0045]

[0046] in, are the camera matrix parameters of the left and right cameras respectively; λ1 and λ2 are non-zero scale factors, (u1, v1)~(u2, v2) are the corresponding two-dimensional coordinates of the matching points, (x w ,y w ,z w ) is the coordinate of the three-dimensional point. Substituting it into the equation, the equation corresponding to the two matching points in the three-dimensional space is:

[0047] Ax=b

[0048] in According to the least squares principle, we can get:

[0049] X=(A T A)-1 A T b

[0050] Here X(x, y, z) is the three-dimensional coordinate of the laser image point.

[0051] Based on the above embodiment, the swing component 12 is a MEMS galvanometer; the light emitted by the multi-beam laser 11 is transmitted to the surface of the object to be measured in the shape of a surface through the MEMS galvanometer for scanning.

[0052] Based on the above embodiment, the MEMS galvanometer arranges the light emitted by the multi-beam laser 11 into a plane at equal intervals and then projects it onto the surface of the object to be measured for scanning.

[0053] Based on the above embodiment, the three 2D cameras are used to reconstruct the 3D model and detect the size, shape or residue at the same time.

[0054] Based on the above embodiment, a laser line pickup unit 3 is electrically connected between the imaging component 1 and the host computer 2, for picking up the laser lines imaged on the three 2D cameras and sending them to the host computer 2 to complete 3D reconstruction.

[0055] The present invention also discloses a multi-eye multi-laser line 3D model reconstruction method comprising the following steps:

[0056] S1. Laser scanning and image acquisition: Start the multi-beam laser and emit multiple laser beams. The laser beams are projected onto the surface of the object to be tested in the form of a surface arranged at equal intervals through the action of the swing component (MEMS galvanometer) for scanning. At the same time, at least three 2D cameras on the periphery work synchronously to shoot the object to be tested from different angles to obtain images of the object surface under laser irradiation. During the shooting process, the parameters of the 2D camera (such as focal length, exposure time, etc.) can be pre-adjusted according to the characteristics of the object to be tested and the environmental conditions to ensure that a clear and accurate image is obtained.

[0057] S2. Laser line pickup and data transmission: The laser line pickup unit picks up the laser line information in the image formed by the three 2D cameras. In this process, the laser line pickup unit can accurately identify and extract the laser line from the complex image background through specific algorithms and circuit designs, and then send the data containing the laser line information to the host computer. During the data transmission process, a high-speed and stable communication protocol is used to ensure the integrity and timeliness of the data.

[0058] S3, 3D model fusion and reconstruction: After the host computer receives the laser line data from the laser line pickup unit, it fuses the image information collected by at least three 2D cameras into a three-dimensional depth of field image through a specific fusion algorithm. This fusion process is based on the principle of multi-view geometry, and uses the relative position relationship between different cameras and the projection information of the laser line on the surface of the object to calculate the three-dimensional coordinates of each point on the surface of the object. The specific fusion algorithm may include steps such as feature point matching and triangulation. Through these steps, the 3D model of the object to be tested is gradually constructed. At the same time, in this process, the host computer can also analyze and detect the size and shape of the object based on the image information obtained by the three 2D cameras, and by analyzing the reflection of the laser line on the surface of the object, detect whether there are abnormal conditions such as residues on the surface of the object.

[0059] S4. Model optimization and output: After the initial completion of the 3D model reconstruction, the host computer can optimize the model.

[0060] The multi-eye multi-laser line 3D model reconstruction method of the present invention has the advantages of high precision, multi-function, efficient data processing and broad application prospects. It can effectively solve the problems existing in traditional 3D model reconstruction methods and meet the needs of modern industry and science and technology for 3D model reconstruction.

[0061] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment, it is not used to limit the present invention. Any person skilled in the art should be able to use the technical content disclosed above to make equivalent embodiments that are equivalent changes by slight changes or modifications without departing from the scope of the technical solution of the present invention. However, any brief modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A multi-lens multi-laser line 3D model reconstruction method, comprising an imaging component (1) and a host computer (2) electrically connected to the imaging component (1), characterized in that: The imaging component (1) comprises a multi-beam laser (11) for emitting multiple beams of laser light; a swing component (12) is swingably provided in front of the optical path of the multi-beam laser (11) for transmitting the multiple beams of laser light emitted by the multi-beam laser (11) onto the object to be measured in a planar form; at least three 2D cameras (4) are arrayed on the circumference of the swing component (12); Wherein: at least three images captured by the 2D cameras are sent to the host computer (2), through X = (A T A) -1 A T b Fusion into a three-dimensional depth of field image.

2. The multi-lens multi-laser line 3D model reconstruction method according to claim 1, characterized in that: The swing component (12) is a MEMS oscillator; the light emitted by the multi-beam laser (11) is transmitted to the surface of the object to be measured in the shape of a surface through the MEMS oscillator for scanning.

3. The multi-lens multi-laser line 3D model reconstruction method according to claim 2, characterized in that: The MEMS oscillating mirror arranges the light emitted by the multi-beam laser (11) into a plane at equal intervals and then projects the light onto the surface of the object to be measured for scanning.

4. The multi-lens multi-laser line 3D model reconstruction method according to claim 1, characterized in that: The three 2D cameras are used to reconstruct the 3D model and detect the size, shape or residue at the same time.

5. The multi-lens multi-laser line 3D model reconstruction method according to claim 1, characterized in that: A laser line pickup unit (3) is electrically connected between the imaging component (1) and the host computer (2), and is used to pick up the laser lines imaged on the three 2D cameras and send them to the host computer (2) to complete 3D reconstruction.

6. The multi-eye multi-laser line 3D model reconstruction method according to claim 1, characterized in that: The multi-eye multi-laser line 3D model reconstruction method includes the following steps: S1. Laser scanning and image acquisition: Start the multi-beam laser to emit multiple laser beams. The laser beams are projected onto the surface of the object to be measured in the form of a surface arranged at equal intervals through the action of the swing component for scanning. At the same time, at least three 2D cameras on the side work synchronously to shoot the object to be measured from different angles to obtain images of the object surface under laser irradiation. S2, laser line picking and data transmission: the laser line picking unit picks up the laser line information in the image formed by three 2D cameras; S3, 3D model fusion and reconstruction: After receiving the laser line data from the laser line pickup unit, the host computer fuses the image information collected by at least three 2D cameras into a three-dimensional depth of field image through a specific fusion algorithm; S4. Model optimization and output: After the initial completion of the 3D model reconstruction, the host computer can optimize the model.

Images