A color structured light 3D camera using coaxial light paths and a calibration method thereof

Abstract

This invention discloses a color structured light 3D camera using a coaxial optical path and its calibration method, relating to the field of structured light cameras. It includes a camera body positioned directly above the object being measured, the camera body integrating an imaging structure, a color modulation structure, and an embedded processing chip. Compared to existing structured light 3D cameras, this invention uses an upright camera layout to eliminate parallax errors caused by oblique installation, improving the spatial positioning accuracy of 2D color information; through a coaxial optical path design, it achieves optical path multiplexing of the structured light channel and color channel without increasing the device size; it selects a CMOS monochrome high-speed sensor, avoiding the resolution loss problem caused by interpolation of three-channel information from a Bayer array sensor, and simultaneously achieves high-precision acquisition of color information through time-division exposure.

Description

Technical Field

[0001] This invention relates to the field of structured light cameras, and more particularly to a color structured light 3D camera using a coaxial optical path and its calibration method. Background Technology

[0002] Structured light 3D cameras use a projector to project a pre-designed pattern with a special structure onto the surface of the object to be measured. Then, the camera observes the deformation of the pattern on the surface. If the object surface is a plane, the observed pattern is similar to the projected pattern and there is no deformation. Once the surface has changes in height, the projected pattern will be modulated by the height information. By analyzing this deformation information, the structured light 3D camera can reconstruct the 3D shape.

[0003] To obtain accurate 3D information, structured light 3D cameras typically use monochrome CMOS sensors in high-precision applications. Therefore, they can only acquire height information and grayscale information of a single color. Moreover, most cameras are tilted, so the grayscale information is often not very accurate. For defect detection scenarios on complex surfaces, it is often necessary to make a comprehensive judgment by combining color 2D information with 3D information.

[0004] To simultaneously satisfy the requirements of camera orientation and the combination of color 2D information, this invention proposes a color structured light 3D camera using a coaxial optical path and its calibration method. Summary of the Invention

[0005] The present invention provides a color structured light 3D camera using a coaxial optical path, comprising: a camera body 1 placed directly above the object being measured, wherein the camera body 1 integrates an imaging structure 11, a color modulation structure 12 and an embedded processing chip.

[0006] The imaging structure 11 consists of a CMOS monochrome high-speed sensor 111 and an imaging lens 112, and the embedded processing chip is electrically connected to the CMOS monochrome high-speed sensor 111.

[0007] The color modulation structure 12 includes an optical protective glass 127, a semi-transparent and semi-reflective mirror group, and an LED light source group. The semi-transparent and semi-reflective mirror group includes a first semi-transparent and semi-reflective mirror 121, a second semi-transparent and semi-reflective mirror 123, and a third semi-transparent and semi-reflective mirror 125. The LED light source group includes a red LED light source 122, a green LED light source 124, and a blue LED light source 126.

[0008] The first semi-transparent and semi-reflective mirror 121, the second semi-transparent and semi-reflective mirror 123, and the third semi-transparent and semi-reflective mirror 125 are arranged vertically from top to bottom, each at a 45° angle to the horizontal direction. Their bottom surfaces are coated with anti-reflective films and their top surfaces are coated with anti-reflective films to achieve separation of reflection and transmission of light in a specific direction. The red LED light source 122, the green LED light source 124, and the blue LED light source 126 are respectively arranged horizontally on the left side of the first semi-transparent and semi-reflective mirror 121, the second semi-transparent and semi-reflective mirror 123, and the third semi-transparent and semi-reflective mirror 125.

[0009] As described above, a color structured light 3D camera using a coaxial optical path is used, wherein the measurement process of the color structured light 3D camera is as follows:

[0010] The external DLP projector is activated to project a preset coded pattern onto the surface of the object being measured, and the CMOS monochrome high-speed sensor (111) simultaneously acquires the structured light image;

[0011] The external DLP projector is turned off, and the three-color LED light source is started sequentially using a time-sharing exposure strategy. The CMOS monochrome high-speed sensor (111) synchronously acquires the R, G, and B three-channel images.

[0012] The embedded processing chip is based on the structured light coding and parsing principle. It performs phase calculation and 3D coordinate reconstruction on the acquired structured light image. At the same time, it performs pixel-level registration, fusion and color correction on the R, G and B three-channel images. Finally, it correlates and maps the 3D geometric data with the color data to generate high-precision color 3D point cloud data containing X, Y and Z coordinates and RGB three-color information, and outputs it to the host computer system through the gigabit Ethernet interface.

[0013] As described above, a color structured light 3D camera using a coaxial optical path is used in which an external DLP projector projects a preset coded pattern onto the surface of the object being measured. The structured light signal reflected from the surface of the object penetrates the optical protective glass (127) and passes through the third semi-transparent mirror (125), the second semi-transparent mirror (123), and the first semi-transparent mirror (121) in sequence along the vertical direction to form a coaxial optical path and enter the imaging lens (112). The imaging lens (112) converges the optical signal onto the photosensitive surface of the CMOS monochrome high-speed sensor (111) to complete the structured light image acquisition. This image is used for subsequent three-dimensional reconstruction calculations.

[0014] The specific process of the time-division exposure strategy in the color structured light 3D camera using a coaxial optical path, as described above, is as follows:

[0015] After the red LED light source (122) is turned on, the emitted light is reflected vertically downward through the anti-reflection film on the bottom surface of the first semi-transparent and semi-reflective mirror (121), and then passes through the anti-reflection film on the top surface of the second semi-transparent and semi-reflective mirror (123), the third semi-transparent and semi-reflective mirror (125), and the optical protective glass (127) to be projected onto the object under test; the red light reflected by the object returns to the CMOS monochrome high-speed sensor (111) along the coaxial optical path to complete the acquisition of the red channel image;

[0016] The red LED light source (122) is turned off and the green LED light source (124) is turned on. The emitted light is reflected vertically downward through the anti-reflection film on the bottom surface of the second semi-transparent and semi-reflective mirror (123), and is projected onto the object under test through the anti-reflection film on the top surface of the third semi-transparent and semi-reflective mirror (125) and the optical protective glass (127). The reflected light returns along the coaxial optical path, completing the green channel image acquisition.

[0017] The green LED light source (124) is turned off and the blue LED light source (126) is turned on. The emitted light is reflected vertically downward by the anti-reflective coating on the bottom surface of the third semi-transparent mirror (125) and projected onto the object under test through the optical protective glass (127). The reflected light returns along the coaxial optical path, completing the acquisition of the blue channel image.

[0018] The present invention also provides a calibration method for a color structured light 3D camera using a coaxial optical path, applied to an embedded processing chip of the color structured light 3D camera using a coaxial optical path as described in any of the above claims, comprising:

[0019] Step S210: Define a set of calibration parameters that are compatible with the imaging structure and color modulation structure;

[0020] Step S220: Control the imaging structure to sequentially acquire the image of the integrated calibration board under projection and the RGB three-channel image;

[0021] Step S230: Set initial values ​​for the calibration parameter set based on the acquired images;

[0022] Step S240: Using the geometry-color integrated collaborative calibration algorithm, starting from the set initial value, iteratively optimize the calibration parameter set to achieve the optimal value;

[0023] Step S250: Store the optimal calibration parameter set obtained by iterative optimization, and automatically read and load the stored parameter set when the 3D camera is restarted.

[0024] The calibration method for a color structured light 3D camera using a coaxial optical path, as described above, utilizes a geometry-color integrated collaborative calibration algorithm. Starting with pre-set initial values, iteratively optimizes the calibration parameter set to achieve its optimal state. Specifically, it includes the following sub-steps:

[0025] Construct the objective function for the geometry-color integrated collaborative calibration algorithm;

[0026] The calibration parameter set is iterated based on the objective function to achieve its optimal state.

[0027] As described above, a calibration method for a color structured light 3D camera using a coaxial optical path includes an integrated calibration board whose substrate surface is coated with a white coating and has a checkerboard pattern. Within the checkerboard area, 18 RGB standard color blocks are printed, and each color block is engraved with a micro QR code, the encoding content of which is the standard sRGB value and physical location of the color block. At the four corners and the center of the four sides of the calibration board, eight micro photodiodes are embedded. The photosensitive surface of each micro photodiode is flush with the surface of the calibration board, and the surface is covered with a diffuse reflection protective window that is consistent with the spectral characteristics of the surrounding coating.

[0028] The beneficial effects achieved by this invention are as follows: Compared with existing structured light 3D cameras, this invention adopts an upright camera layout to eliminate parallax errors caused by oblique installation and improve the spatial positioning accuracy of 2D color information; through coaxial optical path design, optical path multiplexing of structured light channel and color channel is achieved without increasing the size of the device; by selecting a CMOS monochrome high-speed sensor, the resolution loss problem caused by the interpolation of three-channel information of Bayer array sensor is avoided, and high-precision acquisition of color information is achieved through time-division exposure. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0030] Figure 1 This is a schematic diagram of a color structured light 3D camera using a coaxial optical path, provided in Embodiment 1 of this application;

[0031] Figure 2 This is a flowchart of a calibration method for a color structured light 3D camera using a coaxial optical path, provided in Embodiment 2 of this application. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Example 1

[0034] like Figure 1 As shown, Embodiment 1 of this application provides a color structured light 3D camera using a coaxial optical path, including a camera body 1 placed directly above the object being measured. The camera body 1, together with an external DLP projector 2 and the object being measured 3, form a color point cloud measurement system. The external DLP projector 2 projects a preset coded pattern onto the surface of the object being measured 3, and the camera body 1 captures a structured light image that has been highly modulated by the object being measured 3.

[0035] The camera body 1 integrates an imaging structure 11, a color modulation structure 12, and an embedded processing chip.

[0036] The imaging structure 11 consists of a CMOS monochrome high-speed sensor 111 and an imaging lens 112. The embedded processing chip is electrically connected to the CMOS monochrome high-speed sensor (111). The CMOS monochrome high-speed sensor 111 adopts a global shutter mode, with a pixel size of not less than 3.75μm, and can achieve image acquisition at 120fps@1280×1024 resolution to ensure imaging quality in dynamic scenes. The imaging lens 112 is a low-distortion industrial lens with a distortion rate ≤0.1% and a focal length range of 12-25mm to meet different measurement distance requirements.

[0037] The color modulation structure 12 includes an optical protective glass 127, a semi-transparent mirror assembly, and an LED light source assembly. The semi-transparent mirror assembly includes a first semi-transparent mirror 121, a second semi-transparent mirror 123, and a third semi-transparent mirror 125. The LED light source assembly includes a red LED light source 122, a green LED light source 124, and a blue LED light source 126. The first semi-transparent mirror 121, the second semi-transparent mirror 123, and the third semi-transparent mirror 125 are arranged vertically from top to bottom, each forming a 45° angle with the horizontal direction. The surface is coated with an anti-reflective coating (reflectivity ≥95%) and the top surface is coated with an anti-transmittance coating (transmittance ≥92%) to achieve separation of reflection and transmission in a specific direction of the light path; red LED light source 122, green LED light source 124 and blue LED light source 126 are respectively arranged horizontally on the left side of the first semi-transparent semi-reflective mirror 121, the second semi-transparent semi-reflective mirror 123 and the third semi-transparent semi-reflective mirror 125, with peak wavelengths of 625nm, 525nm and 470nm respectively, and all are equipped with collimating lenses to ensure that the parallelism error of the emitted light is ≤0.5°.

[0038] The measurement process in this embodiment is as follows:

[0039] The external DLP projector 2 is activated to project a preset coded pattern onto the surface of the object being measured. The structured light signal reflected from the object surface penetrates the optical protective glass 127 and passes through the third semi-transparent mirror 125, the second semi-transparent mirror 123, and the first semi-transparent mirror 121 in sequence along the vertical direction, forming a coaxial optical path that enters the imaging lens 112. The imaging lens 112 focuses the optical signal onto the photosensitive surface of the CMOS monochrome high-speed sensor 111 to complete the structured light image acquisition. This image is used for subsequent three-dimensional reconstruction calculations.

[0040] With the external DLP projector 2 turned off, the three-color LED light source was activated sequentially using a time-sharing exposure strategy:

[0041] After the red LED light source 122 is turned on, the emitted light is reflected vertically downward through the anti-reflection film on the bottom surface of the first semi-transparent and semi-reflective mirror 121, and then passes through the anti-reflection film on the top surface of the second semi-transparent and semi-reflective mirror 123, the third semi-transparent and semi-reflective mirror 125 and the optical protective glass 127 to be projected onto the object under test; the red light reflected by the object returns to the CMOS monochrome high-speed sensor 111 along the coaxial optical path to complete the acquisition of the red channel image.

[0042] The red LED light source 122 is turned off and the green LED light source 124 is turned on. The emitted light is reflected vertically downward through the anti-reflective coating on the bottom surface of the second semi-transparent mirror 123, and then projected onto the object under test through the anti-reflective coating on the top surface of the third semi-transparent mirror 125 and the optical protective glass 127. The reflected light returns along the coaxial optical path, completing the green channel image acquisition.

[0043] The green LED light source 124 is turned off and the blue LED light source 126 is turned on. The emitted light is reflected vertically downward by the anti-reflective film on the bottom surface of the third semi-transparent and semi-reflective mirror 125, and is projected onto the object under test through the optical protective glass 127. The reflected light returns along the coaxial optical path, completing the acquisition of the blue channel image.

[0044] The embedded processing chip is based on the structured light coding and parsing principle. It performs phase calculation and 3D coordinate reconstruction on the acquired structured light image. At the same time, it performs pixel-level registration, fusion and color correction on the R, G and B three-channel images. Finally, it correlates and maps the 3D geometric data with the color data to generate high-precision color 3D point cloud data containing X, Y and Z coordinates and RGB three-color information, and outputs it to the host computer system through the gigabit Ethernet interface.

[0045] This embodiment achieves optical path multiplexing for structured light measurement and color imaging through coaxial optical path design. Combined with upright mounting and a monochrome high-speed sensor, it obtains high-resolution color texture information without parallax while ensuring the accuracy of three-dimensional measurement (Z-axis repeatability ≤2μm), effectively meeting the comprehensive judgment requirements of complex surface detection.

[0046] Example 2

[0047] like Figure 2 As shown, Embodiment 2 of this application provides a calibration method for a color structured light 3D camera using a coaxial optical path. This method is applied to an embedded processing chip of a color structured light 3D camera using a coaxial optical path. Its core lies in solving the pain points of large registration errors and the inability to calibrate time-division imaging timing deviations caused by the separate calibration of geometric and color parameters in existing calibration technologies through a geometric-color integrated collaborative calibration algorithm. This provides key technical support for the output of high-precision color 3D point cloud data from the 3D camera, including:

[0048] Step S210: Define a set of calibration parameters that are compatible with the imaging structure and color modulation structure;

[0049] Define the calibration parameter set as ,in These are geometric parameters, including the camera intrinsic matrix K, distortion coefficients d, extrinsic parameters R (rotation matrix) and t (offset vector), used to ensure that the images acquired by the imaging structure can reconstruct accurate three-dimensional geometric coordinates; For color parameters, including one A color correction matrix M and a The bias vector b is used to convert the RGB three-channel grayscale image into standard sRGB color values ​​in real time. The timing deviation parameter includes the delayed acquisition time of the R, G, and B channels. This is used to compensate for the corresponding delay time between the LED lighting command and the camera exposure trigger signal during each time-division acquisition of R, G, and B images, ensuring that each image acquisition is performed under completely stable lighting conditions.

[0050] Step S220: Control the imaging structure to sequentially acquire the image of the integrated calibration board under projection and the RGB three-channel image;

[0051] This embodiment designs a geometry-color integrated calibration board for a color structured light 3D camera using a coaxial optical path. This calibration board integrates geometric calibration, color calibration, and timing synchronization functions. It uses a highly stable material as the substrate, specifically a ceramic substrate or an optical glass substrate. The substrate surface is sprayed to form a white coating with a diffuse reflectance of not less than 95%, ensuring that structured light and colored light form uniform and stable reflection signals on its surface. A checkerboard pattern structure is set on the surface of the substrate, with a checkerboard size of [missing information]. There are 10 squares, each with a physical size of 1. Additionally, 18 RGB standard color blocks (selected from international standard color charts) are printed within the checkerboard area, each color block measuring [size missing]. A miniature QR code is engraved next to the color block, which encodes the standard sRGB value and physical location of the color block. Eight miniature photodiodes (PDs) are also embedded in the four corners and the center of the four sides of the calibration board. The photosensitive surface of each PD is flush with the surface of the calibration board and is covered with a diffuse reflection protective window with the same spectral characteristics as the surrounding coating, ensuring that it only senses the intensity of the illumination light and does not interfere with the geometric and color characteristics of the calibration board.

[0052] The integrated calibration plate is fixed directly below the camera to be calibrated using a high-precision six-dimensional adjustment frame. The orientation of the calibration plate is adjusted by using the camera preview screen to make it fill the field of view and produce a clear image.

[0053] Turn on the external DLP projector to project uniform white light, and control the camera to capture the image of the calibration board under the projection. Turn off the external DLP projector and turn on the red (R), green (G), and blue (B) LED light sources sequentially and individually. After each light source is turned on for 3ms (an initial delay time is given to ensure the quality of the captured image), control the camera to perform an image capture, and record the lighting time of each light source, as well as the time when the light intensity detected by each photodiode exceeds the threshold after the light source is turned on. Finally, the image of the calibration board under the projection, the RGB three-channel monochrome image, and a time set of the RGB light sources are obtained.

[0054] Step S230: Set initial values ​​for the calibration parameter set based on the acquired images;

[0055] Geometric parameters can be solved using traditional camera calibration methods based on the projected image of the calibration board. The initial value; using the initialized value The physical coordinates of each color block are projected onto three monochrome RGB images, and the initial average grayscale value is extracted. Let $\mathbf{j}$ be the average grayscale values ​​of the j-th color patch in the R, G, and B channel images, respectively, where $j$ is the color patch index and $\mathbf{j}$ is the value of the j-th color patch. , To determine the total number of color blocks on the calibration board, the linear least squares method is then used to solve for the number of color blocks. The minimum color correction matrix M and bias vector b are used as color parameters. The initial value, where The standard sRGB value of the j-th color patch; timing deviation parameter. That is, the delay acquisition time of the three channels R, G, and B. Initialize to 0.

[0056] Step S240: Using the geometry-color integrated collaborative calibration algorithm, starting from the set initial value, iteratively optimize the calibration parameter set to achieve the optimal value;

[0057] First, we construct the objective function for the geometry-color integrated collaborative calibration algorithm;

[0058] The objective function is expressed as: ,in For geometric consistency loss term, For color fidelity loss items, For time synchronization loss items, To balance the hyperparameters of the three loss terms, so as to avoid the difference in magnitude of the return values ​​of the three loss terms being too large and affecting the balance of the algorithm; , To calibrate the physical coordinates of the i-th feature point on the board (in this embodiment, the corner point of the chessboard), Let i be the image coordinates of the i-th feature point on the projected calibration board image, where i takes values ​​from 1 to... , The total number of feature points. Together they form the geometric parameters to be calibrated. , This is a camera projection model, used to determine the parameters. physical coordinates Mapped to the camera coordinate system, for Robust kernel function, used to suppress the effects of mismatches; , Let j be the standard sRGB value of the j-th color patch. Let be the physical coordinates (center position) of the j-th color block in the calibration plate. and All can be obtained via the QR code next to the color block. , , These are used to return the average grayscale value of the j-th color patch in an RGB three-channel monochrome image. For the color parameters to be calibrated, For color correction functions, , , yes The inverse of the covariance matrix, where j takes values ​​from 1 to... , To determine the total number of color blocks on the calibration board;

[0059] It should be noted that, The calculation process for the functions is consistent, with... For example, firstly according to Determine the physical location of the j-th color block, then determine its four corner points based on its size data (preset values), and finally use the camera projection model. Project the positions of the four corner points onto the camera coordinate system to obtain the image coordinates of the four corner points. Based on the image coordinates, the pixel area of ​​the color block on the R channel image can be determined. Finally, calculate the average gray value of the pixels in the pixel area and return it. , Let be the lighting time of the k-th LED light source. Let k be the delay acquisition time for the k-th channel. It is the median time when the light intensity detected by the 8 photodiodes on the k-th channel exceeds the threshold. , The timing deviation parameter to be calibrated.

[0060] Then, the Levenberg-Marquardt (LM) nonlinear least squares optimization algorithm is used to iteratively optimize the calibration parameter set. Each iteration includes the following steps: ① Based on the current calibration parameter set... Calculate the objective function The return value; ② The L-M algorithm is based on The return value is for the calibration parameter set. Update the parameters to obtain the next set of calibration parameters. ③If If the value is less than the threshold, stop the iteration and reset the current calibration parameter set. Output the optimal solution; otherwise, calibrate the parameter set. As the current calibration parameter set Return to step ①.

[0061] Step S250: Store the optimal calibration parameter set obtained by iterative optimization, and automatically read and load the stored parameter set when the 3D camera is restarted;

[0062] After the 3D camera restarts, the stored geometric parameters (intrinsic parameter matrix K and distortion coefficient d) are directly loaded into the processing unit of the 3D reconstruction algorithm. When performing 3D point cloud computing, all the original image coordinates acquired by the CMOS sensor are used to perform distortion correction and perspective projection transformation, thereby ensuring that the reconstructed 3D geometric coordinates (X, Y, Z) are accurate.

[0063] Stored timing parameters (delay time of each channel) The data is loaded into the light source driving circuit. During each time-division acquisition of R, G, and B images, the control circuit compensates for the corresponding delay time between the LED lighting command and the camera exposure trigger signal, ensuring that each image acquisition is performed under a completely stable lighting condition.

[0064] The stored color parameters (correction matrix M and bias vector b) are loaded into the color correction pipeline. For a three-channel grayscale image after geometric registration, the grayscale value vector of each pixel is... All will pass The transformation converts the color values ​​to standard sRGB values ​​in real time.

[0065] Corresponding to the above embodiments, the present invention provides a computer storage medium, including: at least one memory and at least one processor;

[0066] The memory is used to store one or more program instructions;

[0067] A processor for running one or more program instructions to execute a calibration method for a color structured light 3D camera using a coaxial optical path.

[0068] Corresponding to the above embodiments, this embodiment of the invention provides a computer-readable storage medium containing one or more program instructions, which are executed by a processor to provide a calibration method for a color structured light 3D camera using a coaxial optical path.

[0069] The embodiments disclosed in this invention provide a computer-readable storage medium storing computer program instructions that, when executed on a computer, cause the computer to perform the above-described calibration method for a color structured light 3D camera using a coaxial optical path.

[0070] In this embodiment of the invention, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0071] The various methods, steps, and logic diagrams disclosed in the embodiments of this invention can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the storage medium and, in conjunction with its hardware, completes the steps of the above methods.

[0072] The storage medium can be memory, such as volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0073] Among them, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory.

[0074] Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (Synchlink DRAM, SLDRAM), and direct memory bus RAM (DRRAM).

[0075] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0076] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using a combination of hardware and software. When applied as software, the corresponding functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0077] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A color structured light 3D camera using a coaxial optical path, characterized in that, include: The camera body (1) is placed directly above the object being measured. The camera body (1) integrates an imaging structure (11), a color modulation structure (12), and an embedded processing chip. The imaging structure (11) consists of a CMOS monochrome high-speed sensor (111) and an imaging lens (112), and the embedded processing chip is electrically connected to the CMOS monochrome high-speed sensor (111). The color modulation structure (12) includes an optical protective glass (127), a semi-transparent mirror assembly and an LED light source assembly, wherein the semi-transparent mirror assembly includes a first semi-transparent mirror (121), a second semi-transparent mirror (123) and a third semi-transparent mirror (125), and the LED light source assembly includes a red LED light source (122), a green LED light source (124) and a blue LED light source (126). The first semi-transparent and semi-reflective mirror (121), the second semi-transparent and semi-reflective mirror (123), and the third semi-transparent and semi-reflective mirror (125) are arranged vertically from top to bottom, each at a 45° angle to the horizontal direction. Their bottom surfaces are coated with anti-reflective coatings and their top surfaces are coated with anti-reflective coatings to achieve separation of reflection and transmission of light in a specific direction. Red LED light source (122), green LED light source (124), and blue LED light source (126) are respectively arranged horizontally on the left side of the first semi-transparent and semi-reflective mirror (121), the second semi-transparent and semi-reflective mirror (123), and the third semi-transparent and semi-reflective mirror (125). The embedded processing chip is used to perform the following calibration method: Step S210: Define a set of calibration parameters that are compatible with the imaging structure and color modulation structure; Define the calibration parameter set as ,in These are geometric parameters, including the camera intrinsic matrix K, distortion coefficients d, rotation matrix R, and offset vector t, used to ensure that the images acquired by the imaging structure can reconstruct accurate three-dimensional geometric coordinates; For color parameters, including one A color correction matrix M and a The bias vector b is used to convert the RGB three-channel grayscale image into standard sRGB color values ​​in real time. This refers to timing deviation parameters, including the delayed acquisition times of the R, G, and B channels. This is used to compensate for the corresponding delay time between the LED lighting command and the camera exposure trigger signal when acquiring R, G, and B images in each time-division multiplexing, ensuring that each image acquisition is performed under completely stable lighting conditions. Step S220: Control the imaging structure to sequentially acquire the image of the integrated calibration board under projection and the RGB three-channel image; Step S230: Set initial values ​​for the calibration parameter set based on the acquired images; Step S240: Using the geometry-color integrated collaborative calibration algorithm, starting from the set initial values, iteratively optimize the calibration parameter set to achieve the optimal value. This specifically includes the following sub-steps: Construct the objective function for the geometry-color integrated collaborative calibration algorithm; The objective function is expressed as: ,in For geometric consistency loss term, For color fidelity loss items, For time synchronization loss items, To balance the hyperparameters of the three loss terms, so as to avoid the difference in magnitude of the return values ​​of the three loss terms being too large and affecting the balance of the algorithm; , To determine the physical coordinates of the i-th feature point on the calibration board, Let i be the image coordinates of the i-th feature point on the projected calibration board image, where i takes the value of i. , The total number of feature points. Together they form the geometric parameters to be calibrated. , This is a camera projection model, used to determine the parameters. physical coordinates Mapped to the camera coordinate system, This is the Cauchy robust kernel function, used to suppress the effects of mismatches; , Let j be the standard sRGB value of the j-th color patch. Let J be the physical coordinates of the j-th color patch on the calibration board. and All can be obtained via the QR code next to the color block. , , These are used to return the average grayscale value of the j-th color patch in an RGB three-channel monochrome image. For the color parameters to be calibrated, For color correction functions, , yes The inverse of the covariance matrix, where j takes values ​​of , To determine the total number of color blocks on the calibration board; , Let be the lighting time of the k-th LED light source. Let k be the delay acquisition time for the k-th channel. It is the median time when the light intensity detected by the 8 photodiodes on the k-th channel exceeds the threshold. , The timing deviation parameter to be calibrated; The calibration parameter set is iterated based on the objective function to achieve its optimal state. Step S250: Store the optimal calibration parameter set obtained by iterative optimization, and automatically read and load the stored parameter set when the 3D camera is restarted.

2. The color structured light 3D camera using a coaxial optical path according to claim 1, characterized in that, The measurement process of the color structured light 3D camera is as follows: The external DLP projector is activated to project a preset coded pattern onto the surface of the object being measured, and the CMOS monochrome high-speed sensor (111) simultaneously acquires the structured light image; The external DLP projector is turned off, and the three-color LED light source is started sequentially using a time-sharing exposure strategy. The CMOS monochrome high-speed sensor (111) synchronously acquires the R, G, and B three-channel images. The embedded processing chip is based on the structured light coding and parsing principle. It performs phase calculation and 3D coordinate reconstruction on the acquired structured light image. At the same time, it performs pixel-level registration, fusion and color correction on the R, G and B three-channel images. Finally, it correlates and maps the 3D geometric data with the color data to generate high-precision color 3D point cloud data containing X, Y and Z coordinates and RGB three-color information, and outputs it to the host computer system through the gigabit Ethernet interface.

3. The color structured light 3D camera using a coaxial optical path according to claim 2, characterized in that, After the external DLP projector projects a preset coded pattern onto the surface of the object being measured, the structured light signal reflected from the object surface penetrates the optical protective glass (127) and passes through the third semi-transparent mirror (125), the second semi-transparent mirror (123), and the first semi-transparent mirror (121) in sequence along the vertical direction, forming a coaxial optical path that enters the imaging lens (112). The imaging lens (112) converges the optical signal onto the photosensitive surface of the CMOS monochrome high-speed sensor (111) to complete the structured light image acquisition, which is used for subsequent three-dimensional reconstruction calculations.

4. The color structured light 3D camera using a coaxial optical path according to claim 2, characterized in that, The specific process of the time-sharing exposure strategy is as follows: After the red LED light source (122) is turned on, the emitted light is reflected vertically downward through the anti-reflection film on the bottom surface of the first semi-transparent and semi-reflective mirror (121), and then passes through the anti-reflection film on the top surface of the second semi-transparent and semi-reflective mirror (123), the third semi-transparent and semi-reflective mirror (125), and the optical protective glass (127) to be projected onto the object under test; the red light reflected by the object returns to the CMOS monochrome high-speed sensor (111) along the coaxial optical path to complete the acquisition of the red channel image; The red LED light source (122) is turned off and the green LED light source (124) is turned on. The emitted light is reflected vertically downward through the anti-reflection film on the bottom surface of the second semi-transparent and semi-reflective mirror (123), and is projected onto the object under test through the anti-reflection film on the top surface of the third semi-transparent and semi-reflective mirror (125) and the optical protective glass (127). The reflected light returns along the coaxial optical path, completing the green channel image acquisition. The green LED light source (124) is turned off and the blue LED light source (126) is turned on. The emitted light is reflected vertically downward by the anti-reflective coating on the bottom surface of the third semi-transparent mirror (125) and projected onto the object under test through the optical protective glass (127). The reflected light returns along the coaxial optical path, completing the acquisition of the blue channel image.

5. The calibration method for a color structured light 3D camera using a coaxial optical path according to claim 1, characterized in that, The integrated calibration board has a white coating on its substrate surface and a checkerboard pattern. Within the checkerboard area, 18 RGB standard color blocks are printed. Each color block is engraved with a micro QR code, which encodes the standard sRGB value and physical location of the color block. Eight micro photodiodes are also embedded in the four corners and the center of the four sides of the calibration board. The photosensitive surface of each micro photodiode is flush with the surface of the calibration board and is covered with a diffuse reflection protective window that has the same spectral characteristics as the surrounding coating.

Images