Depth and color fused true color three-dimensional point cloud acquisition method and system

Abstract

The invention discloses a depth and color fused true color three-dimensional point cloud acquisition method and system, and the method comprises the steps: setting a plurality of lasers to emit specific spectrum laser, enabling the specific spectrum laser to be reflected by the surface of an object, and filtering the specific spectrum laser through a narrow-band optical filter of a corresponding wave band, thereby enabling a corresponding sensor to only receive a single-spectrum laser line emitted by a corresponding laser emitter. A laser center point position and an intensity response value are extracted through a sub-pixel-level laser line refinement technology; calibrating the unified measurement coordinate system through sensor pose calibration; according to the light transmittance of an optical filter, spectral response characteristics of an image sensor and the like, correcting and compensating the intensity response value, calculating a compensation factor to obtain a color value of a measurement point, fusing a three-dimensional coordinate with corrected color information, outputting true color point cloud data, and presenting depth and color information of an object at the same time. The three-dimensional information and the color information of the object are synchronously obtained, so that the measurement data contain rich spectral characteristics.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional measurement technology, and more specifically, relates to a method and system for acquiring true color three-dimensional point clouds with minimum depth and color fusion. Background Art

[0002] In modern industrial production and scientific research, line structured light 3D measurement technology, as a non-contact measurement method, has gained widespread application due to its unique advantages. This technology primarily utilizes a specialized laser light source to generate linear structured light, which is then projected onto the surface of the object being measured. When the structured light is projected onto a surface of varying shapes, the light is deformed by the surface's uneven topography. This deformation directly reflects the surface's geometric information. A camera positioned at a specific angle then captures the image of the object illuminated by the structured light. A series of image processing algorithms analyze the captured image to calculate the 3D coordinates of the object's surface. With its significant advantages of high precision, high efficiency, and ease of operation, line structured light 3D measurement technology plays an important role in numerous fields, including industrial inspection, cultural heritage preservation, medical imaging, and virtual reality. Because the measurement process requires no direct contact with the object being measured, potential damage to fragile or dangerous objects is effectively avoided, significantly expanding the application scope of this measurement technology.

[0003] However, existing line structured light 3D measurement technology still faces numerous challenges. First, existing structured light sensors are limited in functionality, capable only of acquiring 3D measurement information of an object. Their output is typically a geometric point cloud consisting of 3D coordinates (X, Y, Z). In actual measurement, using a single-wavelength laser as the structured light source is limited by its spectral characteristics and cannot stimulate the reflective properties of other visible light bands on the object's surface. This results in the measurement data containing only geometric information describing the object's 3D dimensions, while a wealth of important information related to the object, such as color, texture, and material, is not effectively captured. This makes the measurement results incapable of accurately and accurately describing the actual state of the scanned object. In practical applications, measurement data acquired using single-wavelength structured light can only calculate object geometric parameters; it cannot analyze the object's material through multispectral reflectance, such as distinguishing between metal and plastic or detecting coating thickness. Furthermore, it struggles with automated color-based classification tasks, such as polarity identification of electronic components. Consequently, it simply cannot meet practical measurement requirements in applications involving combined geometric and optical analysis.

[0004] Secondly, traditional color point cloud acquisition solutions mostly rely on taking pictures with a camera, then constructing an object model through a 3D reconstruction algorithm, and then coloring the point cloud during subsequent processing. However, this solution has obvious technical flaws. Its spatial resolution and measurement accuracy are relatively low, and ultimately only rough model data can be generated, making it difficult to accurately and individually color high-resolution measurement points. The accuracy of this solution is largely limited by the resolution of the camera itself, and in the process of matching camera coordinates with 3D coordinates, the lack of an accurate correspondence calibration method results in large errors in the coordinate conversion process, resulting in the final obtained color point cloud data being unable to accurately reflect the true color and detailed features of the object.

[0005] In summary, existing line-structured light 3D measurement technologies and color point cloud acquisition solutions have significant deficiencies in terms of comprehensiveness, accuracy, and precision. With increasing demands for the completeness and accuracy of object measurement information in industrial production and scientific research, there is an urgent need for innovative technical solutions that can achieve simultaneous, high-precision acquisition of 3D and color information to meet the measurement needs of increasingly complex application scenarios. Summary of the Invention

[0006] This invention aims to overcome the shortcomings of existing line-structured light 3D measurement technologies, such as their inability to obtain object color information and the low accuracy of traditional color point cloud acquisition schemes. Through an innovative design, the color intensity response value of the corresponding location is simultaneously acquired when calculating the object's depth information. After compensation and correction, the color and depth information are integrated to produce a real-time, high-precision true-color point cloud that accurately reflects the object's texture, color, and size. This provides reliable data for a variety of fields, including industrial inspection and cultural relic preservation, and expands the application of 3D measurement technology.

[0007] In response to the above-mentioned defects or improvement needs of the prior art, as a first aspect of the present invention, the present invention provides a method for acquiring a true color 3D point cloud with depth and color fusion, comprising: S1. Multiple lasers are configured to emit lasers in specific spectral bands. The laser beams are reflected from the surface of an object, and narrowband filters corresponding to the laser wavelengths are placed in front of the corresponding sensors. S2. By replacing the laser line with a single pixel and sub-pixel width light; S3. Calculate the laser center position and the intensity response value at that position; S4. Calibrate the sensor pose to ensure that the laser and imaging system are in the same measurement coordinate system so that their measurement results reflect the same depth information. S5. Correct and compensate the intensity response values ​​obtained by different structured light sensors based on factors including filter transmittance and image sensor spectral response characteristics. Calibrate the channel gain parameters based on filter transmittance and quantum efficiency, and calculate the compensation factor to obtain the color value at the measurement point. S6. Fuse the three-dimensional coordinate measurement information of each point with the corrected and compensated color information to output true color point cloud data containing both depth and color information.

[0008] Furthermore, the specific method for solving the laser center point position and the intensity response value at this position in S3 is: Since the cross-sectional light intensity of the laser stripe on the imaging plane approximately obeys the Gaussian distribution, the distribution formula is: , in, is the pixel value of pixel x in the image, I0 is the peak intensity at the center of the fringe, that is, the intensity response value at the center of the laser fringe; μ is the ordinate of the actual fringe center, σ is the fringe width; x is the position coordinate along the cross section of the laser fringe, which is used to describe the light intensity measurement points at different positions on the cross section; Use Gaussian fitting algorithm to solve, take the natural logarithm of the above formula, let , expand the square term to get the polynomial expression ,in: , , Where, Are in different locations x value; , is the light intensity at the i-th measurement point; After bringing in the points near the laser line in the image, the coefficients a, b, and c can be solved using the least squares algorithm, and then the position of the laser center point and the intensity response value at that position can be calculated: , Based on the above formula, the intensity response value I0 corresponding to the coordinate μ of the center point of the laser stripe can be solved, denoted as V(λ). V(λ) is the original intensity response value output by the sensor at the specific wavelength λ in subsequent calculations; then the coordinate μ of the center of the laser stripe in the image sensor is calculated.

[0009] Furthermore, the factors affecting the transmittance of the filter in S5 are quantified in the following way: Regarding the transmittance of the filter, the transmittance characteristics of the filter are expressed by the following formula: , Where, is the transmittance at a specific wavelength λ, is the intensity of light passing through the filter, is the intensity of light incident on the filter; the transmittance of N filters is expressed as T(λ1), T(λ2) to T(λ N ) to indicate.

[0010] Furthermore, the factors affecting the spectral response characteristics of the image sensor in S5 are specifically quantified as follows: The sensitivity of image sensors to light of different wavelengths varies, which is expressed in quantum efficiency. QE (λ) is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np. The specific formula is: , Where I is the photocurrent, h is Planck's constant, c is the speed of light, e is the elementary charge, P is the incident light power, and λ is the incident light wavelength.

[0011] Furthermore, the specific method of correcting and compensating the intensity response values ​​obtained by different structured light sensors in S5 is: Different image sensors have their own QE-λ curves, and the quantum efficiency QE(λ) corresponding to a certain wavelength λ is obtained; the quantum efficiencies of N spectra in the image sensor are recorded as QE(λ1), QE(λ2) to QE(λ N ); The formula for the compensated band intensity P(λ) is: , Where V(λ) is the raw intensity response value output by the sensor at a specific wavelength λ, T(λ) is the transmittance of the filter at a specific wavelength λ, and k(λ) is the channel gain parameter of the image sensor at a specific wavelength λ, which is related to hardware parameters including integration time and area.

[0012] Furthermore, the calculation method of the compensation factor in S5 is: , Where S(λ) represents the intensity response value compensation factor.

[0013] Furthermore, the calibration method of the channel gain parameter in S5 is: , Under standard white light environment, the intensity of each band after compensation is the same, P(λ1)=P(λ2)=P(λ N ), thereby calibrating the channel gain parameter k(λ).

[0014] Furthermore, the calculation method of the color value of the measurement point in S5 is: The compensation factors of the intensity response values ​​corresponding to N spectra are recorded as S(λ1), S(λ2) to S(λ N ); finally, the color value of each measurement point is obtained by calculation. , Finally, the three-dimensional coordinate measurement information (X, Y, Z) of each point is combined to output a true color point cloud (X, Y, Z, C) with depth and color information.

[0015] As a second aspect of the present invention, a true color 3D point cloud acquisition system with depth and color fusion is provided, comprising: The multi-spectral laser projection and filtering receiving unit is used to set up multiple lasers to emit lasers of specific spectrum bands respectively. The laser beams are reflected after irradiating the surface of the object, and a narrow-band filter with the corresponding wavelength band of the laser is set in front of the corresponding sensor; A laser line sub-pixel refinement processing unit for replacing laser lines with single-pixel and sub-pixel width rays; Laser center point and intensity response calculation unit, used to solve the laser center point position and the intensity response value at that position; The measurement system pose calibration unit is used to calibrate the sensor pose so that the combination of the laser and imaging system is in the same measurement coordinate system, so that the measurement results reflect the same depth information; The information correction, compensation, and color synthesis unit is used to correct and compensate the intensity response values ​​obtained by different structured light sensors based on factors including filter transmittance and image sensor spectral response characteristics. It combines filter transmittance and quantum efficiency to calibrate channel gain parameters and calculate compensation factors to obtain the color value of the measurement point. The depth and color information fusion output unit is used to fuse the three-dimensional coordinate measurement information of each point with the corrected and compensated color information, and output true color point cloud data containing both depth and color information.

[0016] As a third aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program is stored, characterized in that the computer program is executed by a processor to execute any step of the method for acquiring true color three-dimensional point cloud with depth and color fusion.

[0017] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art: 1. The method for acquiring true-color three-dimensional point clouds with depth and color fusion of the present invention is to set up multiple lasers that can emit lasers in specific spectral bands. After calibration, the line lasers are projected onto the surface of the object in a collinear manner. The reflected light is filtered by a narrow-band filter corresponding to the wavelength of the laser and then enters the imaging sensor, thereby realizing the precise projection and reception of multi-spectral lasers. This technical feature ensures that lasers of different wavelengths can accurately illuminate the same position of the object, effectively reducing the interference of ambient light, so that the imaging sensor only collects the laser line image of the corresponding spectrum, laying the foundation for the subsequent acquisition of the surface contour and color information of the object. Its technical effect is to break through the limitations of traditional single-wavelength laser measurement, provide the hardware conditions for the simultaneous acquisition of three-dimensional information and color information of the object, and make the measurement data contain rich spectral characteristics.

[0018] 2. The method for acquiring true-color three-dimensional point clouds with depth and color fusion of the present invention adopts a laser stripe center point extraction algorithm, replaces laser lines with single-pixel and sub-pixel width light to achieve laser line refinement, and combines algorithms such as Gaussian fitting to solve the laser center point position and intensity response value, thereby achieving high-precision image acquisition and coordinate calculation. This technical feature can accurately extract the center point coordinates from laser stripes with a certain pixel width, achieving sub-pixel accuracy, and greatly improving the accuracy of three-dimensional information acquisition. At the same time, by calculating the intensity response values ​​of lasers of different wavelengths at the center point, quantitative data is provided for color information extraction. Its technical effect is reflected in significantly improving the spatial resolution and measurement accuracy of three-dimensional point clouds, so that the point cloud data can more accurately reflect the geometric morphology and detailed features of the object surface.

[0019] 3. The method for acquiring true-color three-dimensional point clouds with depth and color fusion of the present invention comprehensively considers factors such as filter transmittance and image sensor spectral response characteristics, corrects and compensates the intensity response value, and fuses the corrected color information with the three-dimensional coordinate information to output true-color point cloud data, thereby achieving accurate calculation of color information and fusion of depth information. This technical feature eliminates the impact of differences in filter and sensor characteristics on color acquisition by establishing a color compensation model, ensuring that the intensity response values ​​of lasers of different wavelengths are converted into accurate color values. Its technical effect is that the generated true-color point cloud not only has high-precision three-dimensional coordinates, but can also truly restore the surface color of objects, providing high-quality data with both depth and color for fields such as industrial inspection and cultural relics protection, effectively expanding the application scenarios and practicality of three-dimensional measurement technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of a method for acquiring true color 3D point clouds with depth and color fusion according to an embodiment of the present invention; Figure 2 This is a measurement principle diagram of a depth color fusion sensor according to an embodiment of the present invention; Figure 3A schematic diagram of a depth and color fusion sensor according to an embodiment of the present invention; Figure 4 This is a workflow diagram of a depth color fusion sensor according to an embodiment of the present invention; Figure 5 2 is a diagram of system units according to an embodiment of the present invention.

[0021] In all the drawings, the same reference numerals represent the same technical features, specifically: 1 - object to be measured; 2 - laser; 3 - laser line; 4 - filter; 5 - optical lens; 6 - sensor. DETAILED DESCRIPTION

[0022] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0023] Example 1 Please refer to Figure 1 This embodiment 1 provides a method for acquiring a true color 3D point cloud with depth and color fusion, including: S1. Set up multiple lasers, each emitting lasers in a specific spectral band. The laser beams illuminate the surface of an object and then reflect. Narrowband filters corresponding to the laser wavelengths are placed in front of the corresponding sensors, so that the corresponding sensors only receive the laser beams in the corresponding spectrum. S2. Replacing laser lines with single-pixel and sub-pixel widths allows for thinning of the laser line to improve the accuracy of 3D information acquisition. S3. Calculate the laser center position and the intensity response value at that position; S4. Calibrate the sensor pose to ensure that the laser and imaging system are in the same measurement coordinate system so that their measurement results reflect the same depth information. S5. Correct and compensate the intensity response values ​​obtained by different structured light sensors based on factors including filter transmittance and image sensor spectral response characteristics. Calibrate the channel gain parameters based on filter transmittance and quantum efficiency, and calculate the compensation factor to obtain the color value at the measurement point. S6. Fuse the three-dimensional coordinate measurement information of each point with the corrected and compensated color information to output true color point cloud data containing both depth and color information.

[0024] This embodiment 1 further explains the above steps.

[0025] Please refer to Figure 2In general, this embodiment 1 includes: Laser Module: The laser can emit laser light in a specific wavelength band as needed, ultimately outputting line lasers with different spectra. Through installation and calibration, the laser module's line laser output is aligned to ensure the same position on the object is measured.

[0026] Multiple line lasers of different spectral bands are irradiated onto the surface of an object, reflected and collected by the imaging sensor through the optical imaging lens. A filter with a specific band bandwidth is installed in front of or behind the lens. Through frequency selection, only laser lines of a specific spectral band can enter the imaging sensor.

[0027] The laser image captured by the imaging sensor is processed and calculated to extract the laser center to obtain the position of the measurement point in the image coordinate system (image coordinates). The laser center coordinates are calibrated for distance measurement, and the image coordinates are converted to object coordinates to obtain the three-dimensional information of the object to be measured.

[0028] Based on the method of extracting the coordinates of the laser center image plane, the intensity response value of the laser wavelength corresponding to the coordinate position is calculated. Multiple measurement modules can obtain multiple sets of channel data containing three-dimensional information and monochrome information for the same position in the object space in the overlapping area. The three-dimensional depth information and color information of the same location are synthesized after compensation and correction to obtain the color information corresponding to the location, and a true color point cloud is obtained.

[0029] Please refer to Figure 3 as well as Figure 4 , where 1 is the object to be measured; 2 is the laser. The three lasers in the system emit lasers of specific spectrum bands respectively; 3 is the divergent laser line, which propagates in space and irradiates the surface of the object 1; 4 is a filter of specific wavelength, which only allows the corresponding laser wavelength light to pass through; 5 is an optical lens. After passing through the lens, the laser line of the specific wavelength band can finally enter the imaging sensor 6 for subsequent line structured light sensor data processing and fusion.

[0030] The laser beam that strikes the surface of an object is reflected and received by the optical imaging system. To specifically filter the laser beams in different spectral bands and reduce ambient light interference, narrowband filters with bandwidths corresponding to the laser's wavelength are installed before and after the optical lens. A filter is an optical option that selectively passes light within specific wavelength ranges, such as the required red, green, and blue light.

[0031] Then, for different sensor units, since filters with set wavelengths are installed, only laser lines with corresponding spectra can be received, and clear images are formed on the imaging sensor, forming a laser stripe with a width of several pixels. The shape of the laser stripe reflects the contour characteristics of the surface of the object to be measured.

[0032] Using a laser streak center point extraction algorithm, the sensor's imaging coordinates are obtained. Calibration technology then converts these coordinates to the object-space coordinates of the real-world object being measured. Sensor pose calibration aligns the three laser and imaging system pairs in the same measurement coordinate system, ensuring that the measurement results reflect consistent depth information.

[0033] The processing unit processes the effect of the laser line and extracts the precise position of the laser centerline. Since the laser stripes captured by the image sensor are typically a few to dozens of pixels wide, directly using the laser stripe image will inevitably result in significant errors and precision loss. Therefore, it is necessary to calculate the coordinates of the center point that accurately reflects the object's contour features from a laser line of a certain width. This allows the laser line to be replaced with a single-pixel or even sub-pixel width of light, achieving "thinning" of the laser line and improving the accuracy of 3D information acquisition.

[0034] In addition, since the cross-sectional light intensity of the laser stripe on the imaging plane approximately obeys the Gaussian distribution, the distribution formula is: , in, is the pixel value of pixel x in the image, I0 is the peak intensity at the center of the stripe, that is, the intensity response value at the center of the laser stripe; μ is the vertical coordinate of the actual stripe center point, σ is the stripe width; x is the position coordinate along the cross section of the laser stripe, which is used to describe the light intensity measurement points at different positions on the cross section. Use the Gaussian fitting algorithm to solve, take the natural logarithm of the above formula, and let , expand the square term to get the polynomial expression ,in: , , Where, Are in different locations x value; , is the light intensity at the i-th measurement point; After bringing in the points near the laser line in the image, the coefficients a, b, and c can be solved using the least squares algorithm, and then the position of the laser center point and the intensity response value at that position can be calculated: , Based on the above formula, the intensity response value I0 corresponding to the coordinate μ of the center point of the laser stripe can be solved, denoted as V(λ). V(λ) is the original intensity response value output by the sensor at a specific wavelength λ in subsequent calculations. Then, the coordinate μ of the center of the laser stripe in the image sensor is calculated. The subdivision coefficient is K, which can be 32, 64, etc. Taking 32 subdivisions as an example, the sub-pixel coordinate accuracy obtained is 1 / 32 pixel.

[0035] A calibration algorithm is used to obtain the object-space 3D coordinates (X, Y, Z) corresponding to the image sensor's image-space coordinates. Normalization is performed using an M-bit color depth. For example, using M=8, the intensity response range is 0-255. Before multi-channel color information fusion, the intensity response values ​​obtained by the N structured light sensors must be corrected and compensated, taking into account factors such as the filter transmittance and the image sensor's spectral response.

[0036] First, regarding the transmittance of the filter, the transmittance characteristics of the filter can be approximately expressed by the following formula: , Where, is the transmittance at a specific wavelength λ, is the intensity of light passing through the filter, is the intensity of light incident on the filter. The transmittance of N filters is expressed as T(λ1), T(λ2) to T(λ N ) to indicate.

[0037] In addition, since image sensors have different sensitivities to light of different wavelengths, quantum efficiency (QE) is usually used to represent it. QE (λ) is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np. The specific formula is expanded as follows: , Where I is the photocurrent, h is Planck's constant, c is the speed of light, e is the elementary charge, P is the incident light power, and λ is the incident light wavelength. Different image sensors have their own QE-λ curves, which can be used to obtain the quantum efficiency QE(λ) corresponding to a certain wavelength λ. The quantum efficiencies of N spectra in the image sensor are denoted as QE(λ1), QE(λ2), QE(λ3), QE(λ4), QE(λ5), QE(λ6), QE(λ7), QE(λ8), QE(λ9), QE(λ11), QE(λ12), QE(λ13), QE(λ14), QE(λ15), QE(λ16), QE(λ17), QE(λ18), QE(λ19), QE(λ21), QE(λ22), QE(λ N ).

[0038] The formulas for the compensated band intensity P(λ) and intensity response value compensation factor S(λ) are respectively , , Where V(λ) is the raw intensity response value output by the sensor at a specific wavelength λ, T(λ) is the transmittance of the filter at a specific wavelength λ, k(λ) is the channel gain parameter of the image sensor at a specific wavelength λ, which is related to hardware parameters such as integration time and area; QE(λ) is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np.

[0039] In a preferred embodiment, the calibration method of the channel gain parameter is: , Under standard white light environment, the intensity of each band after compensation is the same, P(λ1)=P(λ2)=P(λ N ), thereby calibrating the channel gain parameter k(λ).

[0040] The compensation factors of the intensity response values ​​corresponding to N spectra are recorded as S(λ1), S(λ2) to S(λ N ). Finally, the color value of each measurement point is obtained by calculation. , Finally, the three-dimensional coordinate measurement information (X, Y, Z) of each point is combined to output a true color point cloud (X, Y, Z, C) with depth and color information.

[0041] Taking the most common RGB three-channel color synthesis as an example, for the red laser, the intensity response value corresponding to the sub-pixel coordinate is obtained and normalized using an N-bit color depth, denoted as V(R). Similarly, the intensity response values ​​of the other two structured light sensors are V(G) (green laser) and V(B) (blue laser). The filter transmittances of red light, green light, and blue light are denoted as T(R), T(G), and T(B), respectively. The quantum efficiency of red light, green light, and blue light on the image sensor are denoted as QE(R), QE(G), and QE(B), respectively. Experiments are conducted under a standard white light environment, and the channel gain parameters are calibrated to obtain k(R), k(G), and k(B), respectively. Referring to the compensation factor formulas above, the compensation factors S(R), S(G), and S(B) for red light, green light, and blue light are calculated, respectively, to obtain the color value at the measurement point: , Finally, the three-dimensional coordinate measurement information (X, Y, Z) of each point is combined and the RGB color synthesis is output, which contains the true color point cloud (X, Y, Z, C RGB ).

[0042] Example 2 Referring to FIG. 5 , this embodiment 2 provides a true color 3D point cloud acquisition system with depth and color fusion, including: The multi-spectral laser projection and filtering receiving unit is used to set up multiple lasers to emit lasers of specific spectrum bands respectively. The laser beams are reflected after irradiating the surface of the object. A narrow-band filter with the corresponding wavelength band of the laser is set in front of the corresponding sensor so that the corresponding sensor can only receive the laser beam of the corresponding spectrum. A laser line sub-pixel refinement processing unit is used to replace the laser line with light of single pixel and sub-pixel width to achieve laser line refinement and improve the accuracy of 3D information acquisition; Laser center point and intensity response calculation unit, used to solve the laser center point position and the intensity response value at that position; The measurement system pose calibration unit is used to calibrate the sensor pose so that the combination of the laser and imaging system is in the same measurement coordinate system, so that the measurement results reflect the same depth information; The information correction, compensation, and color synthesis unit is used to correct and compensate the intensity response values ​​obtained by different structured light sensors based on factors including filter transmittance and image sensor spectral response characteristics. It combines filter transmittance and quantum efficiency to calibrate channel gain parameters and calculate compensation factors to obtain the color value of the measurement point. The depth and color information fusion output unit is used to fuse the three-dimensional coordinate measurement information of each point with the corrected and compensated color information, and output true color point cloud data containing both depth and color information.

[0043] Example 3 This embodiment 3 further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, any step of a method for acquiring a true color three-dimensional point cloud with depth and color fusion can be implemented.

[0044] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., which can store program codes.

[0045] For an introduction to the computer-readable storage medium provided in this application, please refer to the above method embodiment, and this application will not go into details here.

[0046] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for acquiring true color 3D point clouds with depth and color fusion, characterized in that: include: S1. Set up multiple lasers to emit lasers of specific spectral bands. The laser beams illuminate the surface of an object and then reflect. Set narrowband filters corresponding to the laser bands in front of the corresponding sensors. S2. By replacing the laser line with a single pixel and sub-pixel width light; S3. Calculate the laser center position and the intensity response value at that position; S4. Calibrate the sensor pose to align the laser and imaging system in the same measurement coordinate system so that the measurement results reflect the same depth information. S5. Correct and compensate the intensity response values ​​obtained by different structured light sensors based on factors including filter transmittance and image sensor spectral response characteristics. Calibrate the channel gain parameters based on filter transmittance and quantum efficiency, and calculate the compensation factor to obtain the color value at the measurement point. S6. Fuse the three-dimensional coordinate measurement information of each point with the corrected and compensated color information to output true color point cloud data containing both depth and color information.

2. The method for acquiring true color 3D point cloud with depth and color fusion according to claim 1, characterized in that: The specific method for solving the laser center point position and the intensity response value at this position in S3 is: Since the cross-sectional light intensity of the laser stripe on the imaging plane approximately obeys the Gaussian distribution, the distribution formula is: ， in, is the pixel value of pixel x in the image, I0 is the peak intensity at the center of the fringe, that is, the intensity response value at the center of the laser fringe; μ is the ordinate of the actual fringe center, σ is the fringe width; x is the position coordinate along the cross section of the laser fringe, which is used to describe the light intensity measurement points at different positions on the cross section; Use Gaussian fitting algorithm to solve, take the natural logarithm of the above formula, let , expand the square term to get the polynomial expression ,in: ， ， Where, Are in different locations x value; , is the light intensity at the i-th measurement point; After bringing in the points near the laser line in the image, the coefficients a, b, and c can be solved using the least squares algorithm, and then the position of the laser center point and the intensity response value at that position can be calculated: ， According to the above formula, the intensity response value corresponding to the coordinate μ of the center point of the laser stripe can be solved: , denoted as V(λ), where V(λ) is the original intensity response value output by the sensor at a specific wavelength λ in subsequent calculations; the coordinate μ of the center of the laser stripe in the image sensor is then calculated.

3. The method for acquiring true color 3D point cloud with depth and color fusion according to claim 1, characterized in that: The factors affecting the transmittance of the filter in S5 are quantified as follows: Regarding the transmittance of the filter, the transmittance characteristics of the filter are expressed by the following formula: ， Where, is the transmittance at a specific wavelength λ, is the intensity of light passing through the filter, is the intensity of light incident on the filter; the transmittance of N filters is expressed as T(λ1), T(λ2) to T(λ N ) to indicate.

4. The method for acquiring true color 3D point cloud with depth and color fusion according to claim 1, characterized in that: The factors affecting the spectral response characteristics of the image sensor in S5 are quantified as follows: The sensitivity of image sensors to light of different wavelengths varies, which is expressed in quantum efficiency. QE (λ) is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np. The specific formula is: ， Where I is the photocurrent, h is Planck's constant, c is the speed of light, e is the elementary charge, P is the incident light power, and λ is the incident light wavelength.

5. A method for acquiring true color 3D point clouds with depth and color fusion according to claim 3 or 4, characterized in that: The specific method for correcting and compensating the intensity response values ​​obtained by different structured light sensors in S5 is: Different image sensors have their own QE-λ curves, and the quantum efficiency QE(λ) corresponding to a certain wavelength λ is obtained; the quantum efficiencies of N spectra in the image sensor are recorded as QE(λ1), QE(λ2) to QE(λ N ); The formula for the compensated band intensity P(λ) is: ， Where V(λ) is the raw intensity response value output by the sensor at a specific wavelength λ, T(λ) is the transmittance of the filter at a specific wavelength λ, k(λ) is the channel gain parameter of the image sensor at a specific wavelength λ, and is related to hardware parameters including integration time and area. QE(λ) is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np.

6. A method for acquiring true color 3D point clouds with depth and color fusion according to claim 3 or 4, characterized in that: The calculation method of the compensation factor in S5 is: ， Where T(λ) is the transmittance of the filter at a specific wavelength λ, k(λ) is the channel gain parameter of the image sensor at a specific wavelength λ, QE(λ) is the ratio of the number of electrons Ne generated by the sensor to the number of incident photons Np, and S(λ) represents the intensity response value compensation factor.

7. The method for acquiring true color 3D point cloud with depth and color fusion according to claim 5, characterized in that: The calibration method of the channel gain parameter in S5 is: ， Under standard white light environment, the intensity of each band after compensation is the same, P(λ1)=P(λ2)=P(λ N ), thereby calibrating the channel gain parameter k(λ).

8. The method for acquiring true color 3D point cloud with depth and color fusion according to claim 6, characterized in that: The calculation method of the color value of the measurement point in S5 is: The compensation factors of the intensity response values ​​corresponding to N spectra are recorded as S(λ1), S(λ2) to S(λ N ); finally, the color value of each measurement point is obtained by calculation. ， Finally, the three-dimensional coordinate measurement information (X, Y, Z) of each point is combined to output a true color point cloud (X, Y, Z, C) with depth and color information.

9. A true color 3D point cloud acquisition system with depth and color fusion, characterized by: include: The multi-spectral laser projection and filtering receiving unit is used to set up multiple lasers to emit lasers of specific spectrum bands respectively. The laser beams are reflected after irradiating the surface of the object, and a narrow-band filter with the corresponding wavelength band of the laser is set in front of the corresponding sensor; A laser line sub-pixel refinement processing unit for replacing laser lines with single-pixel and sub-pixel width rays; Laser center point and intensity response calculation unit, used to solve the laser center point position and the intensity response value at that position; The measurement system pose calibration unit is used to calibrate the sensor pose so that the combination of the laser and imaging system is in the same measurement coordinate system, so that the measurement results reflect the same depth information; The information correction, compensation, and color synthesis unit is used to correct and compensate the intensity response values ​​obtained by different structured light sensors based on factors including filter transmittance and image sensor spectral response characteristics. It combines filter transmittance and quantum efficiency to calibrate channel gain parameters and calculate compensation factors to obtain the color value of the measurement point. The depth and color information fusion output unit is used to fuse the three-dimensional coordinate measurement information of each point with the corrected and compensated color information, and output true color point cloud data containing both depth and color information.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the method for acquiring true color three-dimensional point cloud with depth and color fusion as described in any one of claims 1-8.

Images