Color three-dimensional scanner and three-dimensional reconstruction method

Abstract

The invention provides a color three-dimensional scanner and a three-dimensional reconstruction method, the color three-dimensional scanner comprises two imaging systems in rigid connection, a laser and a processor, and each imaging system comprises a lens and a color sensor; the laser is fixedly arranged between the two imaging systems; the color sensor comprises a plurality of monochromatic spectrum sensors arranged in an array; the color sensor and the laser are connected with the processor; the laser is used for emitting laser of a first monochromatic spectrum to the surface of a target object under the control of the processor; and the processor is used for acquiring the two groups of laser line images and the two groups of mark point images acquired by the two imaging systems, and performing three-dimensional reconstruction on the target object according to the two groups of laser line images and the two groups of mark point images to obtain a texture-free three-dimensional model of the target object. The imaging system formed by the color sensor is adopted to collect the mark point image and the laser line image at the same time, the influence of laser line aliasing on mark point extraction is reduced, and the measurement precision of the color three-dimensional scanner is improved.

Description

Technical Field

[0001] This invention relates to the field of 3D scanning technology, and more specifically, to a color 3D scanner and a 3D reconstruction method. Background Technology

[0002] A 3D scanner is a precision device integrating optics, mechanics, electronics, and algorithms, possessing high-precision 3D reconstruction capabilities. Typically, a 3D scanner simultaneously captures landmarks in a scene and laser lines projected onto the target surface. Then, based on triangulation principles, it obtains the 3D information of the laser lines and, based on landmark registration, calculates inter-frame poses for 3D point fusion. Finally, continuous scanning completes the reconstruction of the target surface. Therefore, the key features of a 3D scanner include: 1) two or more rigidly connected imaging components supporting the triangulation principle; 2) a laser capable of projecting various laser line distribution patterns; and 3) supplementary lighting distributed around the lens to ensure the clarity of landmarks in images under various environmental conditions.

[0003] Existing 3D scanners use supplementary lights and lasers with similar wavelengths. The aim is to maintain the sharpness of both marker points and laser lines when simultaneously capturing them on the same panchromatic sensor, ensuring the accuracy of both marker point and laser line extraction, and thus preserving 3D measurement accuracy. However, during scanning, when the laser line passes through or closely follows the edge of a marker point, the overlap between the two reduces the smoothness of the marker point's edge in the image, thereby decreasing the accuracy of marker point extraction and ultimately affecting the accuracy of 3D point measurement. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of the prior art by providing a color 3D scanner and a 3D reconstruction method, so that an imaging system composed of color sensors can simultaneously acquire marker point images and laser line images, thereby reducing the impact of laser line aliasing on marker point extraction and improving the measurement accuracy of the color 3D scanner.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a color 3D scanner, the color 3D scanner comprising: two imaging systems rigidly connected, a laser, and a processor, each imaging system comprising: a lens and a color sensor; the laser is fixedly disposed between the two imaging systems; the color sensor comprises: a plurality of monochromatic spectral sensors arranged in an array; both the color sensor and the laser are connected to the processor; The laser is used to emit a first monochromatic laser spectrum onto the surface of a target object under the control of the processor; The processor is used to acquire two sets of laser line images and two sets of marker point images collected by the two imaging systems, and to perform three-dimensional reconstruction of the target object based on the two sets of laser line images and the two sets of marker point images to obtain a textureless three-dimensional model of the target object. The two sets of laser line images are laser line images acquired by the monochromatic spectral sensor corresponding to the first monochromatic spectrum in the color sensors of the two imaging systems, and the two sets of marker point images are marker point images acquired by the monochromatic spectral sensor corresponding to the second monochromatic spectrum in the color sensors of the two imaging systems.

[0006] In an optional embodiment, each imaging system further includes a dual-channel filter disposed between the lens and the color sensor, the dual-channel filter being a filter for the first monochromatic spectrum and a filter for the second monochromatic spectrum.

[0007] In an optional embodiment, each imaging system further includes a dual-channel fill light disposed around the lens, the dual-channel fill light being connected to the processor, wherein the dual-channel fill light includes a fill light with a second monochromatic spectrum and a white fill light.

[0008] Secondly, embodiments of this application also provide a three-dimensional reconstruction method, applied to the processor in the color three-dimensional scanner described in the first aspect above, the method comprising: Acquire two sets of laser line images and two sets of marker point images acquired by the two imaging systems; The target object is reconstructed in three dimensions based on the two sets of laser line images and the two sets of marker point images to obtain a textureless three-dimensional model of the target object.

[0009] In an optional implementation, the step of performing three-dimensional reconstruction of the target object based on the two sets of laser line images and the two sets of marker point images to obtain a textureless three-dimensional model of the target object includes: Marker points are extracted from the two sets of marker point images to obtain a first three-dimensional coordinate set, wherein the first three-dimensional coordinate set includes the three-dimensional coordinates of the same-name marker points in the two sets of marker point images; Based on the first three-dimensional coordinate set, determine the pose parameters of the two imaging systems; Laser lines are extracted from the two sets of laser line images to obtain a second three-dimensional coordinate set, wherein the second three-dimensional coordinate set includes the three-dimensional coordinates of each laser line in the two sets of laser line images; Based on the pose parameters of the two imaging systems, the camera parameters, and the second three-dimensional coordinate set, the target object is reconstructed in three dimensions to obtain a textureless three-dimensional model of the target object.

[0010] In an optional implementation, determining the pose parameters of the two imaging systems based on the first three-dimensional coordinate set includes: Based on the three-dimensional coordinates of the corresponding markers in the first three-dimensional coordinate set in adjacent frames, the first transformation parameter of the corresponding markers is calculated as the pose parameter of the corresponding imaging system.

[0011] In an optional implementation, the step of performing three-dimensional reconstruction of the target object based on the pose parameters of the two imaging systems, camera parameters, and the second three-dimensional coordinate set to obtain a textureless three-dimensional model of the target object includes: Based on the three-dimensional coordinates of the marker points in the first three-dimensional coordinate set in the first frame, a reference world coordinate system is constructed; Using the reference world coordinate system as a reference, the transformation matrix of the corresponding imaging system is determined. The transformation matrix includes: the second transformation parameters of the corresponding marker points in the first three-dimensional coordinate set. Based on the transformation matrix, pose parameters, and camera parameters of the corresponding imaging system, the three-dimensional coordinates of each laser line in the second three-dimensional coordinate set are transformed to the reference world coordinate system to obtain the transformed second three-dimensional coordinate set. Based on the transformed second three-dimensional coordinate set, the target object is reconstructed in three dimensions to obtain a textureless three-dimensional model of the target object.

[0012] In an optional implementation, the method further includes: Acquire two sets of color texture images captured by the two imaging systems; Based on the camera parameters of the two imaging systems, the surface texture coordinates of the textureless 3D model are mapped to the two sets of color texture images to obtain the color information of the surface texture coordinates; The textureless 3D model is rendered using color information based on the surface texture coordinates.

[0013] In an optional implementation, before acquiring the two sets of laser line images and two sets of marker point images acquired by the two imaging systems, the method further includes: When the color 3D scanner has the dual-channel filter, the laser in the color 3D scanner is controlled to emit a laser line of the first monochromatic spectrum, and the fill light in the fill light of the color 3D scanner is controlled to emit light of the second monochromatic spectrum.

[0014] In an optional implementation, before acquiring the two sets of color texture images captured by the two imaging systems, the method further includes: When the dual-channel filter is removed, the laser in the color 3D scanner is controlled to emit a laser line of the second monochromatic spectrum, and the white fill light in the fill light in the color 3D scanner is controlled to emit light.

[0015] The beneficial effects of this application are: This application provides a color 3D scanner and a 3D reconstruction method. The color 3D scanner includes two rigidly connected imaging systems, a laser, and a processor. Each imaging system includes a lens and a color sensor. The laser is fixedly disposed between the two imaging systems. The color sensor includes multiple monochromatic spectral sensors arranged in an array. Both the color sensor and the laser are connected to the processor. The laser, under the control of the processor, emits a laser beam of a first monochromatic spectrum onto the surface of a target object. The processor acquires two sets of laser line images and two sets of marker point images collected by the two imaging systems, and performs 3D reconstruction of the target object based on the two sets of laser line images and the two sets of marker point images to obtain a textureless 3D model of the target object. The two sets of laser line images are laser line images acquired by the monochromatic spectral sensor corresponding to the first monochromatic spectrum in the two imaging systems, and the two sets of marker point images are marker point images acquired by the monochromatic spectral sensor corresponding to the second monochromatic spectrum in the two imaging systems. The color 3D scanner of this application, using an imaging system composed of color sensors, can simultaneously acquire marker point images and laser line images, reducing the impact of laser line aliasing on marker point extraction and improving the measurement accuracy of the color 3D scanner. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic diagram of a color 3D scanner provided for an embodiment of this application; Figure 2 A monochrome spectral sensor layout diagram of a color sensor provided in this application embodiment; Figure 3 A flowchart illustrating a three-dimensional reconstruction method provided in an embodiment of this application; Figure 4 A flowchart illustrating another three-dimensional reconstruction method provided in an embodiment of this application; Figure 5 A flowchart illustrating yet another three-dimensional reconstruction method provided in this application embodiment; Figure 6 A flowchart illustrating another three-dimensional reconstruction method provided in this application embodiment; Figure 7 This is a schematic diagram of the functional modules of a three-dimensional reconstruction device provided in an embodiment of this application.

[0018] Explanation of key component symbols: 1-Imaging system; 2-Dual-channel filter; 3-Color sensor; 4-Laser; 5-Dual-channel fill light. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0020] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] In the description of this application, it should be noted that if the terms "upper", "lower", etc. appear to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship that the product of this application is usually placed in, it is only for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0022] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0024] To address the problem that existing 3D scanners using complementary lights and lasers with similar wavelengths suffer from reduced edge smoothness of markers in the image due to overlapping laser lines and markers, leading to decreased marker extraction accuracy and consequently affecting the solution of inter-frame pose parameters, ultimately reducing the measurement accuracy of the 3D scanner, this application provides a color 3D scanner. Figure 1 This is a schematic diagram of a color 3D scanner provided in an embodiment of this application. Figure 2 This is a monochrome spectral sensor layout diagram of a color sensor provided in an embodiment of this application. (See diagram below.) Figure 1 As shown, the color 3D scanner includes: two rigidly connected imaging systems 1, a laser 4, and a processor. Each imaging system includes: a lens and a color sensor 3. The laser 4 is fixedly disposed between the two imaging systems 1. The color sensor 3 includes: multiple monochromatic spectral sensors arranged in an array. Both the color sensor 3 and the laser 4 are connected to the processor.

[0025] Laser 4 is used to emit a first monochromatic laser spectrum onto the surface of the target object under the control of the processor.

[0026] The processor is used to acquire two sets of laser line images and two sets of marker point images collected by two imaging systems 1, and to perform three-dimensional reconstruction of the target object based on the two sets of laser line images and two sets of marker point images to obtain a textureless three-dimensional model of the target object.

[0027] Among them, the two sets of laser line images are laser line images collected by the monochromatic spectral sensor corresponding to the first monochromatic spectrum in the color sensor 3 of the two imaging systems 1, and the two sets of marker point images are marker point images collected by the monochromatic spectral sensor corresponding to the second monochromatic spectrum in the color sensor 3 of the two imaging systems 1.

[0028] In this embodiment, under the control of the processor, the laser 4 emits a laser of a specific wavelength toward the surface of the target object to generate a laser line, which helps to obtain depth information of the target object's surface. The two rigidly connected imaging systems 1 can be two cameras to simulate binocular vision and acquire laser line images and marker point images of the target object from two perspectives. The laser line image includes depth information of the target object's surface, and the marker point image includes marker point information of the target object's surface.

[0029] In addition to controlling the laser 4 to emit laser light, the processor is also used to acquire two sets of laser line images and two sets of marker point images collected by the two imaging systems 1. Specifically, the two sets of laser line images and two sets of marker point images are acquired by the color sensors 3 in the two imaging systems 1, and the target object is reconstructed in three dimensions based on the two sets of laser line images and two sets of marker point images to obtain a textureless 3D model of the target object. If the color sensor 3 in one imaging system 1 acquires one set of laser line images and one set of marker point images, then the two imaging systems 1 will acquire two sets of laser line images and two sets of marker point images. Each set of laser line images includes laser line images acquired at multiple times, and each set of marker point images includes marker point images acquired at multiple times.

[0030] like Figure 2 As shown, in each imaging system 1, multiple monochromatic spectral sensors in the color sensor 3 are arranged in an array, specifically using a Bayer pattern. Each monochromatic spectral sensor has a filter on its surface, ensuring that each sensor records only monochromatic information. A complete color image is then generated through interpolation processing. Interpolation methods include bilinear interpolation, adaptive orientation interpolation, and chromatic aberration constrained interpolation. (Example) Figure 2 The first row of the image alternating between red and green spectral sensors, and the next row alternating between green and blue spectral sensors, each sensor only records the grayscale value of its corresponding color. The missing two color components are reconstructed using a neighborhood interpolation algorithm. Additionally, the processor needs to perform processing on the acquired image, including false color suppression, white balance correction, and color bias correction, to recover the true color information of the target object's surface.

[0031] The purpose of laser 4 emitting laser light is to form a laser line on the surface of the target object, similar to a thin beam of light. When the laser line shines on the surface of the target object, it will bend and deform due to the uneven shape of the object, thereby obtaining the depth information of the target object's surface. That is, the laser line image contains the depth information of the target object's surface. If the first monochromatic spectrum is a blue spectrum, the processor controls laser 4 to emit blue laser light, and the laser line image acquired by color sensor 3 is a blue light image.

[0032] Furthermore, the number and distribution pattern of the laser lines emitted by laser 4 can be adjusted according to the surface characteristics of the target object. The selection of the laser wavelength emitted by laser 4 is adapted to the response curve of the first monochromatic spectrum in color sensor 3. If the first monochromatic spectrum is blue and the second monochromatic spectrum is red, then the principle of maximizing the difference in blue and red light responses in color sensor 3 must be met, that is, selecting a wavelength range with the strongest possible response to the blue spectrum and the weakest possible response to the red spectrum. The purpose is to reduce the response of the blue spectrum sensor in color sensor 3 to red light, thereby improving the clarity of the laser lines on the target surface in the laser line image. For example, the wavelength of the first monochromatic spectrum, i.e., the blue wavelength, can be set to 450 nm; there are no further restrictions on the wavelength of the first monochromatic spectrum.

[0033] By pre-setting reflective markers on the surface of the target object and using a second monochromatic spectrum supplementary light, the monochromatic spectrum sensor corresponding to the second monochromatic spectrum on the color sensor 3 can acquire clear marker image. If the second monochromatic spectrum is selected as the red spectrum, the marker image acquired by the color sensor 3 will be a red light image. The function of the pre-set markers is to serve as spatial reference coordinates. In 3D reconstruction, when the surface features of the target object are not obvious, the pre-set markers provide clear artificial features, helping the algorithm to identify the corresponding markers from different perspectives and improve reconstruction accuracy.

[0034] Optionally, each imaging system 1 further includes a dual-channel filter 2 disposed between the lens and the color sensor, wherein the dual-channel filter 2 is a filter for a first monochromatic spectrum and a second monochromatic spectrum.

[0035] Optionally, each imaging system 1 further includes a dual-channel fill light 5 disposed around the lens, the dual-channel fill light 5 being connected to the processor, wherein the dual-channel fill light 5 includes a fill light of a second monochromatic spectrum and a white fill light.

[0036] If each imaging system is equipped with a dual-channel filter 2, and the first monochromatic spectrum is a blue spectrum and the second monochromatic spectrum is a red spectrum, then the function of the dual-channel filter 2 is to filter out light other than red and blue light, so that the red spectrum sensor in the color sensor 3 can collect the marker points on the surface of the target object and obtain the marker point image, i.e., the red light image, and the blue spectrum sensor in the color sensor 3 can collect the laser line and obtain the laser line image, i.e., the blue light image.

[0037] Each imaging system 1 has LED supplementary lights of a second monochromatic spectrum and white light spaced around its lens to ensure that the color 3D scanner can acquire clear images of marker points even in low-light environments. If each imaging system 1 is equipped with a dual-channel filter 2, to improve the acquisition accuracy of marker points on the target object surface, the processor controls the use of supplementary lights of the second monochromatic spectrum for illumination. The wavelength of the second monochromatic spectrum supplementary light is selected to match the response curve of the second monochromatic spectrum in the color sensor 3. If the second monochromatic spectrum is red and the first monochromatic spectrum is blue, the principle of maximizing the difference in red and blue light responses of the color sensor 3 must be met, that is, selecting a wavelength range with the strongest possible response to the red spectrum and the weakest possible response to the blue spectrum. The purpose is to reduce the response of the red spectrum sensor in the color sensor 3 to blue light, thereby further reducing aliasing interference when the blue laser line passes through the marker point and improving the accuracy of marker point extraction. For example, the wavelength of the second monochromatic spectrum, i.e., the red wavelength, can be set to 620nm; there are no further restrictions on the wavelength of the second monochromatic spectrum.

[0038] It should be noted that if each imaging system 1 is not equipped with a dual-channel filter 2, the processor controls the use of a white fill light for fill light, which can obtain clear true color texture of the target object surface in a darker environment, namely red light image, blue light image and green light image, which are used to reconstruct the texture of the textureless 3D model of the target object.

[0039] In summary, this application provides a color 3D scanner, comprising: two rigidly connected imaging systems, a laser, and a processor. Each imaging system includes: a lens and a color sensor; the laser is fixedly disposed between the two imaging systems; the color sensor includes: multiple monochromatic spectral sensors arranged in an array; both the color sensor and the laser are connected to the processor; the laser, under the control of the processor, emits a laser beam of a first monochromatic spectrum onto the surface of a target object; the processor acquires two sets of laser line images and two sets of marker point images collected by the two imaging systems, and performs 3D reconstruction of the target object based on the two sets of laser line images and the two sets of marker point images to obtain a textureless 3D model of the target object; wherein, the two sets of laser line images are laser line images acquired by the monochromatic spectral sensor corresponding to the first monochromatic spectrum in the two imaging systems, and the two sets of marker point images are marker point images acquired by the monochromatic spectral sensor corresponding to the second monochromatic spectrum in the two imaging systems. The color 3D scanner of this application, using an imaging system composed of color sensors, can simultaneously acquire marker point images and laser line images, reducing the impact of laser line aliasing on marker point extraction and improving the measurement accuracy of the color 3D scanner.

[0040] This application also provides a possible implementation of a three-dimensional reconstruction method, applied to the processor in the aforementioned color three-dimensional scanner. The processor acquires two sets of laser line images and two sets of marker point images from two imaging systems, and performs three-dimensional reconstruction of the target object based on the two sets of laser line images and the two sets of marker point images to obtain a textureless three-dimensional model of the target object. Since the marker points and laser lines are in different images, the interference of laser lines on the marker points can be avoided, the extraction accuracy of the marker points can be improved, and the quality of the textureless three-dimensional model of the target object can be improved.

[0041] Before explaining the three-dimensional reconstruction method provided in the embodiments of this application, the embodiments of this application will first explain the calibration and standardization procedures of the two imaging systems.

[0042] Specifically, the two imaging systems acquire two sets of laser line images and two sets of marker point images. If the first monochromatic spectrum is blue and the second monochromatic spectrum is red, it is equivalent to the two imaging systems acquiring two sets of blue light images and two sets of red light images. Therefore, the two imaging systems are considered a red-light binocular imaging system and a blue-light binocular imaging system. When generating the two imaging systems, the red-light and blue-light binocular imaging systems need to be factory calibrated to set their internal and external parameters and the transformation parameters between them.

[0043] Specifically, for a red-light binocular imaging system, the intrinsic and extrinsic parameters of the system need to be obtained, including the intrinsic parameter sets of the left and right virtual cameras in the red-light binocular imaging system. Two sets of lens distortion parameters The set of extrinsic parameters between the two virtual cameras on the left and right Among them, the set of internal parameters Includes: the principal point coordinates of the left and right virtual cameras , and principal distance , Distortion parameter set Depending on the distortion model used, commonly used Brownian distortion models include parametric compensation for the radial and tangential distortion of the lens, including radial distortion parameters. , and tangential distortion parameters , In a red-light binocular imaging system, the extrinsic parameters of the left and right virtual cameras include: the rotation matrix from the right virtual camera to the left virtual camera. Translation matrix Correspondingly, the blue light binocular imaging system also needs to be calibrated to obtain a set of intrinsic parameters. Distortion parameter set and external parameter set Finally, the transformation matrix from the blue light binocular system coordinate system to the red light binocular system coordinate system also needs to be obtained. .

[0044] The calibration process for both the red-light and blue-light binocular imaging systems requires the use of a calibration board. This board is affixed with reflective markers whose coordinates have been measured. During the calibration of the red-light binocular system, the red light supplement light from the color 3D scanner is turned on, and the imaging system photographs the calibration board from different angles and distances, ensuring the markers cover as many locations as possible in the image. Subsequently, the scanner's red light supplement light is turned off, and an external blue light source is turned on. The calibration image of the blue-light binocular system is then photographed from the same locations. Adjustment processing is performed based on the principle of minimizing the reprojection error of the markers to obtain the intrinsic parameter set of the red-light binocular imaging system. Distortion parameter set and external parameter set And the intrinsic parameter set of the blue light binocular imaging system Distortion parameter set and external parameter set and transformation matrix .

[0045] To ensure scanning accuracy of the target object, the extrinsic parameter sets of both the red-light and blue-light binocular imaging systems need to be calibrated before scanning. Specifically, this requires using the calibration board that comes with the color 3D scanner, which also has calibrated reflective markers. The purpose of the calibration process is to correct for parameter changes in the color 3D scanner caused by transportation and environmental variations, including calibrating the extrinsic parameter sets of the red-light binocular imaging system. The set of extrinsic parameters for a blue light binocular imaging system and transformation matrix .

[0046] During calibration, the red light supplementary illumination is turned on, and the color 3D scanner captures images of the calibration board at different angles and distances to ensure that the marker points cover as many positions as possible in the image. Subsequently, the transformation parameters between the two cameras of the red light binocular imaging system are obtained through adjustment based on the principle of minimizing the reprojection error of the marker points. This includes the rotation matrix from the right camera to the left camera. Translation matrix Since the red and blue light images are actually formed by the same color sensor, when updating the blue light binocular imaging system, the relative relationship between the red and blue light cameras in the same imaging system can be assumed to remain unchanged. That is, within the same imaging system, the transformation matrix... It is stable and unchanging, including the rotation matrix from the blue light binocular imaging system to the red light binocular imaging system in this imaging system. Translation matrix Therefore, the homogeneous coordinates of points in the left and right cameras of the red-blue binocular imaging system are... and , respectively represented as:

[0047]

[0048] Therefore, after obtaining the calibration parameters of the red light binocular imaging system Afterwards, the transformations of points in the left camera and points in the right camera satisfy the following formula:

[0049] Based on the above formula, we can obtain:

[0050] Therefore, the transformation matrix from the right camera to the left camera in a blue light binocular imaging system is:

[0051] Specifically, when the coordinate system of the left camera in the red / blue binocular imaging system is used as a reference, the transformation between the two binocular imaging systems... This is equal to the transformation matrix between the red and blue cameras in the left camera of the imaging system:

[0052] Therefore, before scanning the target object, the two imaging systems in the color 3D scanner were calibrated, and the camera parameters of the two imaging systems were obtained, thus obtaining the transformation matrix between the red and blue binocular imaging systems. .

[0053] The following is a detailed description of a three-dimensional reconstruction method provided in the embodiments of this application. Figure 3 This is a flowchart illustrating a three-dimensional reconstruction method provided in an embodiment of this application, as shown below. Figure 3 As shown, the method includes: S101. Acquire two sets of laser line images and two sets of marker point images acquired by the two imaging systems.

[0054] S102. Based on two sets of laser line images and two sets of marker point images, perform three-dimensional reconstruction of the target object to obtain a textureless three-dimensional model of the target object.

[0055] In this embodiment, before acquiring two sets of laser line images and two sets of marker point images from the two imaging systems, the processor needs to control the laser in the color 3D scanner to emit laser light, control the supplementary light to provide supplementary illumination, and control the color sensor to acquire laser line images and marker point images at each moment. Thus, the processor can acquire two sets of laser line images and two sets of marker point images from the color sensors of the two imaging systems.

[0056] Optionally, before acquiring the two sets of laser line images and two sets of marker point images acquired by the two imaging systems, the method further includes: when the color 3D scanner has a dual-channel filter, controlling the laser in the color 3D scanner to emit laser lines of the first monochromatic spectrum, and controlling the supplementary light in the supplementary light of the color 3D scanner to emit light of the second monochromatic spectrum.

[0057] Specifically, in this embodiment, taking the first monochromatic spectrum as the blue spectrum and the second monochromatic spectrum as the red spectrum as an example, when the color 3D scanner has a red and blue dual-channel filter, the laser is controlled to emit blue laser lines, and the red supplementary light is controlled to emit light to supplement the surface of the target object. This allows the red spectrum sensor of the color sensor to collect the marker point image at each moment, and the blue spectrum sensor to collect the laser line image at each moment. Thus, the color sensor in each imaging system collects a set of marker point images and a set of laser line images.

[0058] The processor processes two sets of laser line images and two sets of marker point images to perform three-dimensional reconstruction of the target object, resulting in a textureless three-dimensional model of the target object.

[0059] It should be noted that even without a dual-channel filter, the laser in the color 3D scanner is still controlled to emit a laser line of the first monochromatic spectrum, and the white supplementary light in the supplementary light lamp is also controlled to emit light. This can be understood as controlling the laser to emit a blue laser line and controlling the white supplementary light lamp to emit light to illuminate the surface of the target object. Similarly, the red spectrum sensor of the color sensor acquires the marker point image at each moment, the blue spectrum sensor acquires the laser line image at each moment, and the green spectrum sensor also acquires the object surface image at each moment. Thus, each color sensor in the imaging system acquires a set of marker point images, a set of laser line images, and a set of object surface images. The processor then uses these images to perform 3D reconstruction of the target object, obtaining a textured 3D model of the target object. However, compared to color 3D scanners with dual-channel filters, the use of white supplemental lighting and the lack of filtering for light other than red and blue light means that the acquired marker images are affected by other light sources. This affects the accuracy of marker extraction and consequently, the accuracy of the reconstructed 3D model of the target object. In contrast, color sensors acquire true-color texture images in the red, green, and blue bands, allowing for the reconstruction of textured 3D models.

[0060] In summary, this application provides a 3D reconstruction method applied to a processor in a color 3D scanner. The method includes: acquiring two sets of laser line images and two sets of marker point images from two imaging systems; and performing 3D reconstruction of a target object based on the two sets of laser line images and the two sets of marker point images to obtain a textureless 3D model of the target object. This 3D reconstruction method uses two sets of laser line images and two sets of marker point images to perform 3D reconstruction of the target object. Because the laser lines and marker points are not in the same image, the impact of laser line aliasing on marker point extraction is reduced, improving the reconstruction accuracy of the target object and the quality of the textureless 3D model.

[0061] This application also provides another possible implementation of the three-dimensional reconstruction method. Figure 4 A flowchart illustrating another three-dimensional reconstruction method provided in this application embodiment is shown below. Figure 4 As shown, a three-dimensional reconstruction of the target object is performed based on two sets of laser line images and two sets of marker point images, resulting in a textureless three-dimensional model of the target object, including: S201. Extract the marker points from the two sets of marker point images to obtain the first three-dimensional coordinate set.

[0062] The first three-dimensional coordinate set includes the three-dimensional coordinates of the same-named markers in the two sets of marker images.

[0063] In this embodiment, using the intrinsic and extrinsic parameters of the red-light binocular imaging system, the three-dimensional coordinates of all marker points on the current marker point image are calculated based on the triangulation principle, thus forming a point cloud set of marker points at the current moment. Then, based on the principle of triangulation, the three-dimensional coordinates of all marker points on the marker point image at each time point are calculated, and the point cloud set of the marker points at each time point can be obtained. Based on the point cloud set of the marker points at each time point, the first three-dimensional coordinate set is constructed.

[0064] S202. Determine the pose parameters of the two imaging systems based on the first three-dimensional coordinate set.

[0065] Optionally, based on the three-dimensional coordinates of the same-named markers in adjacent frames in the first three-dimensional coordinate set, the first transformation parameters of the same-named markers are calculated as the pose parameters of the corresponding imaging system.

[0066] Specifically, the three-dimensional coordinates of all marker points in the current time marker image are calculated, and the point cloud set constituting the current time marker points is represented as follows: Then, the three-dimensional coordinates of all marker points on the marker point image at the next time step are calculated, and the point cloud set of marker points at the next time step is represented as follows: For a set of point clouds in adjacent frames, where most points are the same spatial marker, the first transformation parameters between adjacent frames can be calculated based on the 3D coordinates of the same marker in adjacent frames. Including rotation matrix Translation matrix It satisfies the following relationship:

[0067] Then the first transformation parameter These are the pose parameters of the red light binocular imaging system.

[0068] S203. Extract laser lines from the two sets of laser line images to obtain the second three-dimensional coordinate set.

[0069] The second three-dimensional coordinate set includes the three-dimensional coordinates of each laser line in the two sets of laser line images.

[0070] Specifically, using the intrinsic and extrinsic parameters of the blue light binocular imaging system, the three-dimensional coordinates of the laser line points on the laser line image at the current moment are calculated based on the triangulation principle, thus obtaining the point cloud set of the laser line points at the current moment. Then, based on the principle of triangulation, the three-dimensional coordinates of all laser line points on the laser line image at each moment are calculated, and the point cloud set of laser line points at each moment can be obtained. Based on the point cloud set of laser line points at each moment, a second three-dimensional coordinate set is constructed.

[0071] S204. Based on the pose parameters of the two imaging systems, the camera parameters, and the second three-dimensional coordinate set, the target object is reconstructed in three dimensions to obtain a textureless three-dimensional model of the target object.

[0072] Specifically, based on the pose parameters and camera parameters of the two imaging systems, the second three-dimensional coordinate set is transformed to perform three-dimensional reconstruction of the target object, resulting in a textureless three-dimensional model of the target object.

[0073] In the method provided in this application embodiment, two sets of marker point images are processed to extract marker points, resulting in a first three-dimensional coordinate set, wherein the first three-dimensional coordinate set includes the three-dimensional coordinates of the same-name marker points in the two sets of marker point images; the pose parameters of the two imaging systems are determined based on the first three-dimensional coordinate set; laser lines are extracted from two sets of laser line images to obtain a second three-dimensional coordinate set, wherein the second three-dimensional coordinate set includes the three-dimensional coordinates of each laser line in the two sets of laser line images; based on the pose parameters of the two imaging systems, camera parameters, and the second three-dimensional coordinate set, a three-dimensional reconstruction of the target object is performed to obtain a textureless three-dimensional model of the target object. By processing the two sets of laser line images and the two sets of marker point images, a three-dimensional reconstruction of the target object is performed to obtain a textureless three-dimensional model of the target object.

[0074] This application also provides another possible implementation of the three-dimensional reconstruction method. Figure 5 A flowchart illustrating another three-dimensional reconstruction method provided in this application embodiment is shown below. Figure 5 As shown, based on the pose parameters of the two imaging systems, camera parameters, and the second 3D coordinate set, a 3D reconstruction of the target object is performed to obtain a textureless 3D model of the target object, including: S301. Construct a reference world coordinate system based on the three-dimensional coordinates of the marker points in the first three-dimensional coordinate set in the first frame.

[0075] S302. Using the reference world coordinate system as a benchmark, determine the transformation matrix of the corresponding imaging system.

[0076] The transformation matrix includes the second transformation parameters of the corresponding marker points in the first three-dimensional coordinate set.

[0077] In this embodiment, to ensure the pose parameters To ensure computational accuracy, the 3D coordinates of the marker points in the first frame of the first 3D coordinate set are generally used as the reference world coordinate system. For example, if the reference world coordinate system is established using the coordinates of the marker points determined by measuring equipment such as a laser rangefinder, then the transformation of the marker points in each frame needs to be multiplied by the transformation matrix into the world coordinate system. Using the first frame as the reference world coordinate system is equivalent to transformation. The rotation matrix is ​​an identity matrix, and the translation is 0. Therefore, the general formula for transforming each frame's marker point to the world coordinate system is:

[0078] Where, transformation matrix The second transformation parameter is for the same-named marker.

[0079] S303. Based on the transformation matrix, pose parameters, and camera parameters of the corresponding imaging system, transform the three-dimensional coordinates of each laser line in the second three-dimensional coordinate set to the reference world coordinate system to obtain the transformed second three-dimensional coordinate set.

[0080] S304. Based on the transformed second three-dimensional coordinate set, perform three-dimensional reconstruction of the target object to obtain a textureless three-dimensional model of the target object.

[0081] Specifically, the pose parameters based on the red light binocular system are... The transformation matrix from the corresponding imaging system to the world coordinate system is... and camera parameters Transform the laser line points of each frame of the blue light binocular system in the second three-dimensional coordinate set to the world coordinate system, and represent it as follows:

[0082] This yields the transformed second three-dimensional coordinate set. Based on the transformed second three-dimensional coordinate set, the target object is reconstructed in three dimensions to obtain a textureless three-dimensional model of the target object.

[0083] In the method provided in this application embodiment, a reference world coordinate system is constructed based on the three-dimensional coordinates of the marker points in the first three-dimensional coordinate set in the first frame; using the reference world coordinate system as a reference, the transformation matrix of the corresponding imaging system is determined, and the transformation matrix includes the second transformation parameters of the marker points with the same name in the first three-dimensional coordinate set; based on the transformation matrix, pose parameters and camera parameters of the corresponding imaging system, the three-dimensional coordinates of each laser line in the second three-dimensional coordinate set are transformed to the reference world coordinate system to obtain the transformed second three-dimensional coordinate set; finally, based on the transformed second three-dimensional coordinate set, the target object is reconstructed in three dimensions to obtain a textureless three-dimensional model of the target object, so that the laser line points of each frame of the blue light binocular system are transformed to the world coordinate system, and the target object is reconstructed in three dimensions using the transformed second three-dimensional coordinate set to obtain a textureless three-dimensional model of the target object.

[0084] This application also provides another possible implementation of the three-dimensional reconstruction method. Figure 6 A flowchart illustrating another three-dimensional reconstruction method provided in this application embodiment is shown below. Figure 6 As shown, the method also includes: S401. Acquire two sets of color texture images captured by two imaging systems.

[0085] S402. Based on the camera parameters of the two imaging systems, map the surface texture coordinates of the textureless 3D model to two sets of color texture images to obtain the color information of the surface texture coordinates.

[0086] S403. Render the textureless 3D model using the color information of the surface texture coordinates.

[0087] In this embodiment, since the target object is reconstructed in three dimensions based on only two sets of laser line images (blue light images) and two sets of marker point images (red light images) to obtain a textureless 3D model of the target object, and color rendering of the textureless 3D model is still required, texture UV unwrapping is performed on the textureless 3D model; uniform light and color processing is performed on the red and blue dual-channel false color images; using the intrinsic and extrinsic parameters of the two imaging systems of the color 3D scanner, the surface texture coordinates of the textureless 3D model are back-projected onto each color texture image to obtain the color information of the surface texture coordinates, which is used to perform color rendering of the textureless 3D model based on the color information of the surface texture coordinates.

[0088] Optionally, before acquiring two sets of color texture images acquired by the two imaging systems, the method further includes: With the dual-channel filter removed, the laser in the color 3D scanner is controlled to emit a laser line of the second monochromatic spectrum, and the white fill light in the fill light in the color 3D scanner is controlled to emit light.

[0089] Specifically, the laser is controlled to emit a blue laser line, and the white fill light is controlled to emit light, so that the color sensor can acquire true color texture images of the red, green and blue three channels. Then the two imaging systems acquire two sets of color texture images respectively.

[0090] The following continues to explain the three-dimensional reconstruction device provided in any of the above embodiments of this application. Its specific implementation process and the resulting technical effects are the same as those in the corresponding method embodiments. For the sake of brevity, parts not mentioned in this embodiment can be referred to the corresponding content in the method embodiments.

[0091] Figure 7 This is a schematic diagram of the functional modules of a three-dimensional reconstruction device provided in an embodiment of this application. Figure 7 As shown, the processor used in the color 3D scanner, the 3D reconstruction device 100 includes: The acquisition module 110 is used to acquire two sets of laser line images and two sets of marker point images collected by the two imaging systems; The reconstruction module 120 is used to perform three-dimensional reconstruction of the target object based on two sets of laser line images and two sets of marker point images, so as to obtain a textureless three-dimensional model of the target object.

[0092] Optionally, the reconstruction module 120 is further configured to extract landmarks from the two sets of landmark images to obtain a first three-dimensional coordinate set, wherein the first three-dimensional coordinate set includes: the three-dimensional coordinates of landmarks with the same name in the two sets of landmark images; determine the pose parameters of the two imaging systems based on the first three-dimensional coordinate set; extract laser lines from the two sets of laser line images to obtain a second three-dimensional coordinate set, wherein the second three-dimensional coordinate set includes: the three-dimensional coordinates of each laser line in the two sets of laser line images; and perform three-dimensional reconstruction of the target object based on the pose parameters of the two imaging systems, camera parameters, and the second three-dimensional coordinate set to obtain a textureless three-dimensional model of the target object.

[0093] Optionally, the reconstruction module 120 is further configured to calculate the first transformation parameters of the corresponding marker points as pose parameters of the corresponding imaging system based on the three-dimensional coordinates of the corresponding marker points in the first three-dimensional coordinate set in adjacent frames.

[0094] Optionally, the reconstruction module 120 is further configured to construct a reference world coordinate system based on the three-dimensional coordinates of the markers in the first three-dimensional coordinate set in the first frame; determine the transformation matrix of the corresponding imaging system based on the reference world coordinate system, the transformation matrix including: the second transformation parameters of the markers with the same name in the first three-dimensional coordinate set; transform the three-dimensional coordinates of each laser line in the second three-dimensional coordinate set to the reference world coordinate system according to the transformation matrix, pose parameters and camera parameters of the corresponding imaging system, to obtain the transformed second three-dimensional coordinate set; and perform three-dimensional reconstruction of the target object based on the transformed second three-dimensional coordinate set to obtain a textureless three-dimensional model of the target object.

[0095] Optionally, the device further includes: The acquisition module 110 is also used to acquire two sets of color texture images acquired by the two imaging systems; The mapping module is used to map the surface texture coordinates of the textureless 3D model to two sets of color texture images based on the camera parameters of the two imaging systems, so as to obtain the color information of the surface texture coordinates. The rendering module is used to render the color of a textureless 3D model based on the color information of the surface texture coordinates.

[0096] Optionally, the device further includes: The control module is used to control the laser in the color 3D scanner to emit a laser line of the first monochromatic spectrum, and to control the fill light in the fill light of the second monochromatic spectrum in the color 3D scanner, when the color 3D scanner has a dual-channel filter.

[0097] Optionally, the control module is also used to control the laser in the color 3D scanner to emit a laser line of a second monochromatic spectrum when the dual-channel filter is removed, and to control the white fill light in the fill light of the color 3D scanner to emit light.

[0098] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0099] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0100] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A color 3D scanner, characterized in that, The color 3D scanner includes: two rigidly connected imaging systems, a laser, and a processor. Each imaging system includes: a lens and a color sensor. The laser is fixedly disposed between the two imaging systems. The color sensor includes: multiple monochromatic spectral sensors arranged in an array. Both the color sensor and the laser are connected to the processor. The laser is used to emit a first monochromatic laser spectrum onto the surface of a target object under the control of the processor; The processor is used to acquire two sets of laser line images and two sets of marker point images collected by the two imaging systems, and to perform three-dimensional reconstruction of the target object based on the two sets of laser line images and the two sets of marker point images to obtain a textureless three-dimensional model of the target object. The two sets of laser line images are laser line images acquired by the monochromatic spectral sensor corresponding to the first monochromatic spectrum in the color sensors of the two imaging systems, and the two sets of marker point images are marker point images acquired by the monochromatic spectral sensor corresponding to the second monochromatic spectrum in the color sensors of the two imaging systems.

2. The color 3D scanner according to claim 1, characterized in that, Each imaging system further includes a dual-channel filter disposed between the lens and the color sensor, wherein the dual-channel filter is a filter for the first monochromatic spectrum and the second monochromatic spectrum.

3. The color 3D scanner according to claim 2, characterized in that, Each imaging system further includes a dual-channel fill light disposed around the lens, the dual-channel fill light being connected to the processor, wherein the dual-channel fill light includes a fill light with a second monochromatic spectrum and a white fill light.

4. A three-dimensional reconstruction method, characterized in that, The processor applied to the color 3D scanner of claim 3 above, the method comprising: Acquire two sets of laser line images and two sets of marker point images acquired by the two imaging systems; The target object is reconstructed in three dimensions based on the two sets of laser line images and the two sets of marker point images to obtain a textureless three-dimensional model of the target object.

5. The method according to claim 4, characterized in that, The step of performing three-dimensional reconstruction of the target object based on the two sets of laser line images and the two sets of marker point images to obtain a textureless three-dimensional model of the target object includes: Marker points are extracted from the two sets of marker point images to obtain a first three-dimensional coordinate set, wherein the first three-dimensional coordinate set includes the three-dimensional coordinates of the same-name marker points in the two sets of marker point images; Based on the first three-dimensional coordinate set, determine the pose parameters of the two imaging systems; Laser lines are extracted from the two sets of laser line images to obtain a second three-dimensional coordinate set, wherein the second three-dimensional coordinate set includes the three-dimensional coordinates of each laser line in the two sets of laser line images; Based on the pose parameters of the two imaging systems, the camera parameters, and the second three-dimensional coordinate set, the target object is reconstructed in three dimensions to obtain a textureless three-dimensional model of the target object.

6. The method according to claim 5, characterized in that, The step of determining the pose parameters of the two imaging systems based on the first three-dimensional coordinate set includes: Based on the three-dimensional coordinates of the corresponding markers in the first three-dimensional coordinate set in adjacent frames, the first transformation parameter of the corresponding markers is calculated as the pose parameter of the corresponding imaging system.

7. The method according to claim 5, characterized in that, The step of reconstructing the target object in three dimensions based on the pose parameters of the two imaging systems, the camera parameters, and the second three-dimensional coordinate set to obtain a textureless three-dimensional model of the target object includes: Based on the three-dimensional coordinates of the marker points in the first three-dimensional coordinate set in the first frame, a reference world coordinate system is constructed; Using the reference world coordinate system as a reference, the transformation matrix of the corresponding imaging system is determined. The transformation matrix includes: the second transformation parameters of the corresponding marker points in the first three-dimensional coordinate set. Based on the transformation matrix, pose parameters, and camera parameters of the corresponding imaging system, the three-dimensional coordinates of each laser line in the second three-dimensional coordinate set are transformed to the reference world coordinate system to obtain the transformed second three-dimensional coordinate set. Based on the transformed second three-dimensional coordinate set, the target object is reconstructed in three dimensions to obtain a textureless three-dimensional model of the target object.

8. The method according to claim 4, characterized in that, The method further includes: Acquire two sets of color texture images captured by the two imaging systems; Based on the camera parameters of the two imaging systems, the surface texture coordinates of the textureless 3D model are mapped to the two sets of color texture images to obtain the color information of the surface texture coordinates; The textureless 3D model is rendered using color information based on the surface texture coordinates.

9. The method according to claim 4, characterized in that, Before acquiring the two sets of laser line images and two sets of marker point images acquired by the two imaging systems, the method further includes: When the color 3D scanner has the dual-channel filter, the laser in the color 3D scanner is controlled to emit a laser line of the first monochromatic spectrum, and the fill light in the fill light of the color 3D scanner is controlled to emit light of the second monochromatic spectrum.

10. The method according to claim 8, characterized in that, Before acquiring the two sets of color texture images captured by the two imaging systems, the method further includes: When the dual-channel filter is removed, the laser in the color 3D scanner is controlled to emit a laser line of the second monochromatic spectrum, and the white fill light in the fill light in the color 3D scanner is controlled to emit light.

Images